From patchwork Thu Mar 23 09:53:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 666456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3489C6FD1C for ; Thu, 23 Mar 2023 09:57:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230004AbjCWJ5q (ORCPT ); Thu, 23 Mar 2023 05:57:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229951AbjCWJ5B (ORCPT ); Thu, 23 Mar 2023 05:57:01 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E56830EB0; Thu, 23 Mar 2023 02:56:04 -0700 (PDT) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32N4wUNe012451; Thu, 23 Mar 2023 09:54:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=xehSbS+s0c5yXxqmHCnm4b44HXDj2CwqiJavRokDY8o=; b=K4hcvWOZhV5MYW/waDmIPtOdvd5kIzhxP8Xw3cNSdGh175ZiK4LJo4Cx0Yi5iRUqHoEZ I0UkPI0VU50UrZOSJ3+cygCDsmJ67vBXSbpg/MdJHUjnyJ347fKkstJdEy5VdRVvErVc fbBxdzGxMEicgCK4UDEz2gyjo57WjUPFZjAQZHYTsCUy7s5owzVzrrdOHVDbWbdQk1nW 0GgiFIokuXf6ofT5nPQYDsHSTn108Eb38tDfUUoGsgG0ZssweF9JFawOR393KtxwYQaK DqIsQUIVBZn9bIn4u7W0WrD0Vl6OyOLfYz4Lkitydtgp06+5XZyaQ8ndQ1UIKcwDlkP5 Lw== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3pgbf715s7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Mar 2023 09:54:26 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 32N9sPIJ023604 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Mar 2023 09:54:25 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 23 Mar 2023 02:54:25 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , Arthur Simchaev , Eric Biggers , Kiwoong Kim , open list Subject: [RFC PATCH v3 1/5] ufs: mcq: Add supporting functions for mcq abort Date: Thu, 23 Mar 2023 02:53:31 -0700 Message-ID: <90b7b370269674fd7c0f505b59e0993c5b95a316.1679564391.git.quic_nguyenb@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: io4UVUNdW3JE3cEOpicUCtRoqtrQYZb3 X-Proofpoint-GUID: io4UVUNdW3JE3cEOpicUCtRoqtrQYZb3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-22_21,2023-03-22_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1011 adultscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303150002 definitions=main-2303230074 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add supporting functions to handle ufs abort in mcq mode. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufs-mcq.c | 189 +++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 10 +++ drivers/ufs/core/ufshcd.c | 4 +- include/ufs/ufshci.h | 17 ++++ 4 files changed, 218 insertions(+), 2 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 31df052..29d0e37 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -12,6 +12,9 @@ #include #include #include "ufshcd-priv.h" +#include +#include +#include #define MAX_QUEUE_SUP GENMASK(7, 0) #define UFS_MCQ_MIN_RW_QUEUES 2 @@ -27,6 +30,9 @@ #define MCQ_ENTRY_SIZE_IN_DWORD 8 #define CQE_UCD_BA GENMASK_ULL(63, 7) +/* Max mcq register polling time in millisecond unit */ +#define MCQ_POLL_MS 500 + static int rw_queue_count_set(const char *val, const struct kernel_param *kp) { return param_set_uint_minmax(val, kp, UFS_MCQ_MIN_RW_QUEUES, @@ -429,3 +435,186 @@ int ufshcd_mcq_init(struct ufs_hba *hba) host->host_tagset = 1; return 0; } + +static int ufshcd_mcq_poll_register(void __iomem *reg, u32 mask, + u32 val, unsigned long timeout_ms) +{ + unsigned long timeout = jiffies + msecs_to_jiffies(timeout_ms); + int err = 0; + + /* ignore bits that we don't intend to wait on */ + val = val & mask; + + while ((readl(reg) & mask) != val) { + usleep_range(10, 50); + if (time_after(jiffies, timeout)) { + err = -ETIMEDOUT; + break; + } + } + + return err; +} + +static int ufshcd_mcq_sq_stop(struct ufs_hba *hba, struct ufs_hw_queue *hwq) +{ + void __iomem *reg; + u32 i = hwq->id; + int err; + + writel(SQ_STOP, mcq_opr_base(hba, OPR_SQD, i) + REG_SQRTC); + reg = mcq_opr_base(hba, OPR_SQD, i) + REG_SQRTS; + err = ufshcd_mcq_poll_register(reg, SQ_STS, SQ_STS, MCQ_POLL_MS); + if (err) + dev_err(hba->dev, "%s: failed. hwq-id=%d, err=%d\n", + __func__, i, err); + return err; +} + +static int ufshcd_mcq_sq_start(struct ufs_hba *hba, struct ufs_hw_queue *hwq) +{ + void __iomem *reg; + u32 i = hwq->id; + int err; + + writel(SQ_START, mcq_opr_base(hba, OPR_SQD, i) + REG_SQRTC); + reg = mcq_opr_base(hba, OPR_SQD, i) + REG_SQRTS; + err = ufshcd_mcq_poll_register(reg, SQ_STS, 0, MCQ_POLL_MS); + if (err) + dev_err(hba->dev, "%s: failed. hwq-id=%d, err=%d\n", + __func__, i, err); + return err; +} + +/** + * ufshcd_mcq_sq_cleanup - Clean up Submission Queue resources + * associated with the pending command. + * @hba - per adapter instance. + * @task_tag - The command's task tag. + * @result - Result of the Clean up operation. + * + * Returns 0 and result on completion. Returns error code if + * the operation fails. + */ +int ufshcd_mcq_sq_cleanup(struct ufs_hba *hba, int task_tag, int *result) +{ + struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; + struct scsi_cmnd *cmd = lrbp->cmd; + struct ufs_hw_queue *hwq; + void __iomem *reg, *opr_sqd_base; + u32 nexus, i; + int err; + + if (task_tag != hba->nutrs - UFSHCD_NUM_RESERVED) { + if (!cmd) + return FAILED; + hwq = ufshcd_mcq_req_to_hwq(hba, scsi_cmd_to_rq(cmd)); + } else { + hwq = hba->dev_cmd_queue; + } + + i = hwq->id; + + spin_lock(&hwq->sq_lock); + + /* stop the SQ fetching before working on it */ + err = ufshcd_mcq_sq_stop(hba, hwq); + if (err) + goto unlock; + + /* SQCTI = EXT_IID, IID, LUN, Task Tag */ + nexus = lrbp->lun << 8 | task_tag; + opr_sqd_base = mcq_opr_base(hba, OPR_SQD, i); + writel(nexus, opr_sqd_base + REG_SQCTI); + + /* SQRTCy.ICU = 1 */ + writel(SQ_ICU, opr_sqd_base + REG_SQRTC); + + /* Poll SQRTSy.CUS = 1. Return result from SQRTSy.RTC */ + reg = opr_sqd_base + REG_SQRTS; + err = ufshcd_mcq_poll_register(reg, SQ_CUS, SQ_CUS, MCQ_POLL_MS); + if (err) { + dev_err(hba->dev, "%s: failed. hwq=%d, lun=0x%x, tag=%d\n", + __func__, i, lrbp->lun, task_tag); + *result = FIELD_GET(SQ_ICU_ERR_CODE_MASK, readl(reg)); + } + + err = ufshcd_mcq_sq_start(hba, hwq); + +unlock: + spin_unlock(&hwq->sq_lock); + return err; +} + +static u64 ufshcd_mcq_get_cmd_desc_addr(struct ufs_hba *hba, + int task_tag) +{ + struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; + __le32 hi = lrbp->utr_descriptor_ptr->command_desc_base_addr_hi; + __le32 lo = lrbp->utr_descriptor_ptr->command_desc_base_addr_lo; + + return le64_to_cpu((__le64)hi << 32 | lo); +} + +/** + * ufshcd_mcq_nullify_cmd - Nullify utrd. Host controller does not fetch + * transfer with Command Type = 0xF. post the Completion Queue with OCS=ABORTED. + * @hba - per adapter instance. + * @hwq - Hardware Queue of the nullified utrd. + */ +static void ufshcd_mcq_nullify_cmd(struct ufs_hba *hba, struct ufs_hw_queue *hwq) +{ + struct utp_transfer_req_desc *utrd; + u32 dword_0; + + utrd = (struct utp_transfer_req_desc *)(hwq->sqe_base_addr + + hwq->id * sizeof(struct utp_transfer_req_desc)); + dword_0 = le32_to_cpu(utrd->header.dword_0); + dword_0 &= ~UPIU_COMMAND_TYPE_MASK; + dword_0 |= FIELD_PREP(UPIU_COMMAND_TYPE_MASK, 0xF); + utrd->header.dword_0 = cpu_to_le32(dword_0); +} + +/** + * ufshcd_mcq_sqe_search - Search for the cmd in the Submission Queue (SQ) + * @hba - per adapter instance. + * @hwq - Hardware Queue to be searched. + * @task_tag - The command's task tag. + * + * Returns true if the SQE containing the command is present in the SQ + * (not fetched by the controller); returns false if the SQE is not in the SQ. + */ +static bool ufshcd_mcq_sqe_search(struct ufs_hba *hba, + struct ufs_hw_queue *hwq, int task_tag) +{ + struct utp_transfer_req_desc *utrd; + u32 mask = hwq->max_entries - 1; + bool ret = false; + u64 addr, match; + u32 sq_head_slot; + + spin_lock(&hwq->sq_lock); + + ufshcd_mcq_sq_stop(hba, hwq); + sq_head_slot = ufshcd_mcq_get_sq_head_slot(hwq); + if (sq_head_slot == hwq->sq_tail_slot) + goto out; + + addr = ufshcd_mcq_get_cmd_desc_addr(hba, task_tag); + while (sq_head_slot != hwq->sq_tail_slot) { + utrd = (struct utp_transfer_req_desc *)(hwq->sqe_base_addr + + sq_head_slot * sizeof(struct utp_transfer_req_desc)); + match = le64_to_cpu((__le64)utrd->command_desc_base_addr_hi << 32 | + utrd->command_desc_base_addr_lo) & CQE_UCD_BA; + if (addr == match) { + ret = true; + goto out; + } + sq_head_slot = (sq_head_slot + 1) & mask; + } + +out: + ufshcd_mcq_sq_start(hba, hwq); + spin_unlock(&hwq->sq_lock); + return ret; +} diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index 529f850..1a40cf7 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -78,6 +78,8 @@ struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); +int ufshcd_mcq_sq_cleanup(struct ufs_hba *hba, int task_tag, int *result); + #define UFSHCD_MCQ_IO_QUEUE_OFFSET 1 #define SD_ASCII_STD true #define SD_RAW false @@ -403,4 +405,12 @@ static inline struct cq_entry *ufshcd_mcq_cur_cqe(struct ufs_hw_queue *q) return cqe + q->cq_head_slot; } + +static inline u32 ufshcd_mcq_get_sq_head_slot(struct ufs_hw_queue *q) +{ + u32 val = readl(q->mcq_sq_head); + + return val / sizeof(struct utp_transfer_req_desc); +} + #endif /* _UFSHCD_PRIV_H_ */ diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 629442c..504f041 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -56,7 +56,8 @@ #define NOP_OUT_RETRIES 10 /* Timeout after 50 msecs if NOP OUT hangs without response */ #define NOP_OUT_TIMEOUT 50 /* msecs */ - +/* Maximum MCQ registers polling time */ +#define MCQ_POLL_TIMEOUT 500 /* Query request retries */ #define QUERY_REQ_RETRIES 3 /* Query request timeout */ @@ -173,7 +174,6 @@ EXPORT_SYMBOL_GPL(ufshcd_dump_regs); enum { UFSHCD_MAX_CHANNEL = 0, UFSHCD_MAX_ID = 1, - UFSHCD_NUM_RESERVED = 1, UFSHCD_CMD_PER_LUN = 32 - UFSHCD_NUM_RESERVED, UFSHCD_CAN_QUEUE = 32 - UFSHCD_NUM_RESERVED, }; diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h index 11424bb..ceb8664 100644 --- a/include/ufs/ufshci.h +++ b/include/ufs/ufshci.h @@ -99,6 +99,9 @@ enum { enum { REG_SQHP = 0x0, REG_SQTP = 0x4, + REG_SQRTC = 0x8, + REG_SQCTI = 0xC, + REG_SQRTS = 0x10, }; enum { @@ -111,12 +114,26 @@ enum { REG_CQIE = 0x4, }; +enum { + SQ_START = 0x0, + SQ_STOP = 0x1, + SQ_ICU = 0x2, +}; + +enum { + SQ_STS = 0x1, + SQ_CUS = 0x2, +}; + +#define SQ_ICU_ERR_CODE_MASK GENMASK(7, 4) +#define UPIU_COMMAND_TYPE_MASK GENMASK(31, 28) #define UFS_MASK(mask, offset) ((mask) << (offset)) /* UFS Version 08h */ #define MINOR_VERSION_NUM_MASK UFS_MASK(0xFFFF, 0) #define MAJOR_VERSION_NUM_MASK UFS_MASK(0xFFFF, 16) +#define UFSHCD_NUM_RESERVED 1 /* * Controller UFSHCI version * - 2.x and newer use the following scheme: From patchwork Thu Mar 23 09:53:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 666458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13763C6FD1D for ; Thu, 23 Mar 2023 09:56:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231146AbjCWJ4s (ORCPT ); Thu, 23 Mar 2023 05:56:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229451AbjCWJ4Z (ORCPT ); Thu, 23 Mar 2023 05:56:25 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3961A12076; Thu, 23 Mar 2023 02:55:32 -0700 (PDT) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32N4dSk3001753; Thu, 23 Mar 2023 09:55:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=qto8nirKrDnq9b3yZUDtZmlx+sU1DsaWYCzYeW33bXA=; b=f5XqQrQEYxOlVKdXccT41XDzH5dUGsTdcoFHy2M9XaUJ0DkFTd9WXj02mTjHCtrvurNj RDk6pcvnMGaRaL+dzLicsLofvzqkQcLCNC0K3e1WtpnUPxPDcmnCDvQ/wS+eLRGJzHlg rXxADzV6H3nGRgWMvhplJ9MTmXGAaRqeG8rcwNzoRpsicDGzsfLMENdvoTFKFNSG99MQ 3j9QZzPqUpc0pN61lo3Gax3/bCvamtBZNxW5OmZSQtGz1XrCHmbyqnOi/pRXiLEAIviB Dyus6XasxHk8FwqQTkJSNhG3GqdAi+0/q7bg63dXLFGQ0deL859KiiWfHtxCmwKbLJ5i bg== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3pg5tthyyr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Mar 2023 09:55:18 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 32N9tHhb004853 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Mar 2023 09:55:17 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 23 Mar 2023 02:55:16 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , Andy Gross , Bjorn Andersson , Konrad Dybcio , "Arthur Simchaev" , open list , "open list:UNIVERSAL FLASH STORAGE HOST CONTROLLER DRIVER..." Subject: [RFC PATCH v3 4/5] ufs: mcq: Use ufshcd_mcq_poll_cqe_lock() in mcq mode Date: Thu, 23 Mar 2023 02:53:34 -0700 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: dp28vabEYDRpBIPX7Pp4Pu_q_WZlSXaR X-Proofpoint-ORIG-GUID: dp28vabEYDRpBIPX7Pp4Pu_q_WZlSXaR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-22_21,2023-03-22_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 clxscore=1015 mlxscore=0 phishscore=0 spamscore=0 lowpriorityscore=0 priorityscore=1501 impostorscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303150002 definitions=main-2303230074 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In preparation for adding mcq error handler support, update the mcq code to use the ufshcd_mcq_poll_cqe_lock() in interrupt context instead of using ufshcd_mcq_poll_cqe_nolock(). This is to keep synchronization between mcq interrupt and error handler contexts because both need to access the mcq hardware in separate contexts. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufs-mcq.c | 6 +++--- drivers/ufs/core/ufshcd-priv.h | 2 -- drivers/ufs/core/ufshcd.c | 2 +- drivers/ufs/host/ufs-qcom.c | 2 +- include/ufs/ufshcd.h | 2 +- 5 files changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 5c417a1..613d302 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -283,8 +283,8 @@ static void ufshcd_mcq_process_cqe(struct ufs_hba *hba, ufshcd_compl_one_cqe(hba, tag, cqe); } -unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, - struct ufs_hw_queue *hwq) +static unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, + struct ufs_hw_queue *hwq) { unsigned long completed_reqs = 0; @@ -300,7 +300,6 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, return completed_reqs; } -EXPORT_SYMBOL_GPL(ufshcd_mcq_poll_cqe_nolock); unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq) @@ -313,6 +312,7 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, return completed_reqs; } +EXPORT_SYMBOL_GPL(ufshcd_mcq_poll_cqe_lock); void ufshcd_mcq_make_queues_operational(struct ufs_hba *hba) { diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index ef66151..70b7ffc 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -71,8 +71,6 @@ void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds); void ufshcd_mcq_select_mcq_mode(struct ufs_hba *hba); u32 ufshcd_mcq_read_cqis(struct ufs_hba *hba, int i); void ufshcd_mcq_write_cqis(struct ufs_hba *hba, u32 val, int i); -unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, - struct ufs_hw_queue *hwq); struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba, struct request *req); unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 359747a..3c28627 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -6756,7 +6756,7 @@ static irqreturn_t ufshcd_handle_mcq_cq_events(struct ufs_hba *hba) ufshcd_mcq_write_cqis(hba, events, i); if (events & UFSHCD_MCQ_CQIS_TAIL_ENT_PUSH_STS) - ufshcd_mcq_poll_cqe_nolock(hba, hwq); + ufshcd_mcq_poll_cqe_lock(hba, hwq); } return IRQ_HANDLED; diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index 34fc453..c686fb8 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -1558,7 +1558,7 @@ static irqreturn_t ufs_qcom_mcq_esi_handler(int irq, void *__hba) struct ufs_hw_queue *hwq = &hba->uhq[id]; ufshcd_mcq_write_cqis(hba, 0x1, id); - ufshcd_mcq_poll_cqe_nolock(hba, hwq); + ufshcd_mcq_poll_cqe_lock(hba, hwq); return IRQ_HANDLED; } diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index 05e4164..5d7f9b2 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1244,7 +1244,7 @@ void ufshcd_update_evt_hist(struct ufs_hba *hba, u32 id, u32 val); void ufshcd_hba_stop(struct ufs_hba *hba); void ufshcd_schedule_eh_work(struct ufs_hba *hba); void ufshcd_mcq_write_cqis(struct ufs_hba *hba, u32 val, int i); -unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba, +unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, struct ufs_hw_queue *hwq); void ufshcd_mcq_enable_esi(struct ufs_hba *hba); void ufshcd_mcq_config_esi(struct ufs_hba *hba, struct msi_msg *msg); From patchwork Thu Mar 23 09:53:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bao D. Nguyen" X-Patchwork-Id: 666457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E313C6FD1C for ; Thu, 23 Mar 2023 09:57:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231253AbjCWJ5Z (ORCPT ); Thu, 23 Mar 2023 05:57:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230455AbjCWJ4o (ORCPT ); Thu, 23 Mar 2023 05:56:44 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EC7612BC9; Thu, 23 Mar 2023 02:55:54 -0700 (PDT) Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32N5m8k1031101; Thu, 23 Mar 2023 09:55:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=9nK1QVzD5wgTB9KziIcEmOW4Vd3VDIQjJ2WXRImwQhE=; b=ADy2+ubK4QRmpMuE4OvPCtcUstI1gtnAf0cix+HImMN3r4EOKLEpdcWlMeyklBZjVxQM CJDJlR2UnT3292V0e2vYdB+PKvMOLAHLL/zd5jfmLZa7CJ6eKaJGOKTpwfm1e9f7HVbx ccAllrH33L5ox9WwkHyQbi0FDRWYQx6xUjzSgJwxC9Djnzgqp+dFjnp9w+b3/GUtEPOJ yQcwhVcNBcwlZoFV9eMm61RZmXPg5ukVhYl8YF+svazOsDE7qeMX54O1ATcgLRdOqNt0 DqRVGOfz1/nZPYPzDeq5qZKuzgi87WF95o/S6qmo/OETw7Mxqu2DCzUQv0a4bFBhWGHs DQ== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3pg3c9aese-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Mar 2023 09:55:23 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 32N9tMM0004593 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Mar 2023 09:55:22 GMT Received: from stor-berry.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 23 Mar 2023 02:55:21 -0700 From: "Bao D. Nguyen" To: , , , , , , , , CC: , "Bao D. Nguyen" , Alim Akhtar , "James E.J. Bottomley" , open list Subject: [RFC PATCH v3 5/5] ufs: core: Add error handling for MCQ mode Date: Thu, 23 Mar 2023 02:53:35 -0700 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: G-EjhWkYiT1xtAbhj34cFfXy7Ajk-6Za X-Proofpoint-ORIG-GUID: G-EjhWkYiT1xtAbhj34cFfXy7Ajk-6Za X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-22_21,2023-03-22_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 mlxlogscore=984 bulkscore=0 priorityscore=1501 mlxscore=0 malwarescore=0 clxscore=1015 suspectscore=0 adultscore=0 impostorscore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303150002 definitions=main-2303230074 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add support for error handling for MCQ mode. Signed-off-by: Bao D. Nguyen --- drivers/ufs/core/ufshcd.c | 81 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 70 insertions(+), 11 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 3c28627..be4299e 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -3104,6 +3104,12 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT; dev_dbg(hba->dev, "%s: dev_cmd request timedout, tag %d\n", __func__, lrbp->task_tag); + + /* MCQ mode */ + if (is_mcq_enabled(hba)) + return ufshcd_clear_cmds(hba, 1U << lrbp->task_tag); + + /* SDB mode */ if (ufshcd_clear_cmds(hba, 1U << lrbp->task_tag) == 0) { /* successfully cleared the command, retry if needed */ err = -EAGAIN; @@ -5536,6 +5542,10 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) */ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) { + struct ufshcd_lrb *lrbp; + u32 hwq_num, utag; + int tag; + /* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. * In order to prevent other interrupts starvation the DB is read once @@ -5554,7 +5564,22 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) * Ignore the ufshcd_poll() return value and return IRQ_HANDLED since we * do not want polling to trigger spurious interrupt complaints. */ - ufshcd_poll(hba->host, UFSHCD_POLL_FROM_INTERRUPT_CONTEXT); + if (!is_mcq_enabled(hba)) { + ufshcd_poll(hba->host, UFSHCD_POLL_FROM_INTERRUPT_CONTEXT); + goto out; + } + + /* mcq mode */ + for (tag = 0; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; + if (lrbp->cmd) { + utag = blk_mq_unique_tag(scsi_cmd_to_rq(lrbp->cmd)); + hwq_num = blk_mq_unique_tag_to_hwq(utag); + ufshcd_poll(hba->host, hwq_num); + } + } + +out: return IRQ_HANDLED; } @@ -6323,18 +6348,36 @@ static bool ufshcd_abort_all(struct ufs_hba *hba) bool needs_reset = false; int tag, ret; - /* Clear pending transfer requests */ - for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { - ret = ufshcd_try_to_abort_task(hba, tag); - dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, - hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, - ret ? "failed" : "succeeded"); - if (ret) { - needs_reset = true; - goto out; + if (is_mcq_enabled(hba)) { + struct ufshcd_lrb *lrbp; + int tag; + + for (tag = 0; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; + if (lrbp->cmd) { + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + } + if (ret) { + needs_reset = true; + goto out; + } + } + } else { + /* Clear pending transfer requests */ + for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + if (ret) { + needs_reset = true; + goto out; + } } } - /* Clear pending task management requests */ for_each_set_bit(tag, &hba->outstanding_tasks, hba->nutmrs) { if (ufshcd_clear_tm_cmd(hba, tag)) { @@ -7266,6 +7309,8 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) unsigned long flags, pending_reqs = 0, not_cleared = 0; struct Scsi_Host *host; struct ufs_hba *hba; + struct ufs_hw_queue *hwq; + struct ufshcd_lrb *lrbp; u32 pos; int err; u8 resp = 0xF, lun; @@ -7281,6 +7326,20 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) goto out; } + if (is_mcq_enabled(hba)) { + for (pos = 0; pos < hba->nutrs; pos++) { + lrbp = &hba->lrb[pos]; + if (lrbp->cmd && lrbp->lun == lun) { + ufshcd_clear_cmds(hba, 1U << pos); + hwq = ufshcd_mcq_req_to_hwq(hba, + scsi_cmd_to_rq(lrbp->cmd)); + ufshcd_mcq_poll_cqe_lock(hba, hwq); + } + } + err = 0; + goto out; + } + /* clear the commands that were pending for corresponding LUN */ spin_lock_irqsave(&hba->outstanding_lock, flags); for_each_set_bit(pos, &hba->outstanding_reqs, hba->nutrs)