From patchwork Thu Feb 18 13:19:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 384712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D8BFC433E9 for ; Thu, 18 Feb 2021 16:59:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 62FF364EB8 for ; Thu, 18 Feb 2021 16:59:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234100AbhBRQ5w (ORCPT ); Thu, 18 Feb 2021 11:57:52 -0500 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:2071 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233264AbhBRN0l (ORCPT ); Thu, 18 Feb 2021 08:26:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654800; x=1645190800; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GGiQnfmLQo8dIg2skv/t5+JKGNT/ObkSr0cIYyWrUzQ=; b=Z7Q0JMq1wI4fx31SWw3XpKuI3hNrQC4X37csrUPGudZzO0sn0lZMETTj 2Q4ytT4vHeMoLzQ14dNLyqNp7EVhuEJZ74woMyDiO+E0FPdcvhjYIKJpQ hfogxF2wV0gO7xdi4HHHkSx2Pv+zL0NoSPFVowAtbAxcw3jUuPM0aCa67 PynnDgN0yiIYKq62QcGT8gOB7IuIvMIlphW/UREv68MCVXYY5EUXh/EHG UdxEJDi4NWjlf7xfnrTTYaUKmnWCl9AXvbi2PEY0uLr93JQcF49eJxZ95 dTKO/zmDcY28oKmYJ9bsc5Pryhndgm/418upeNPqNLcb7U8K/2SlQOLqH g==; IronPort-SDR: qavi5vJWZr8EI/g4jjBZfatZ27XMgIHMPTjhLSnkS2yHD1ZsFLW8fWDQ1lTJnv5CqfW0dmHA69 DdzXUiC3VLOvK4ikV2kmg+CCaHSMA4Qhg/EvENoC9bJo8EYRI+UPl5HA2vMpA5QN+WjD+k1BHD LtsdWAJJdnTck0PGWhS9vWp+0sYjyy428653LcPFQcNCbtL/f9pTyjmowT/lHMS727bvD3d85U wdBljYovcIGaosiWLMNRsRyFHRwawjsXhz8Rrj3sqMivnsQYPgdTHpjQjaeYoAIlXQuYiKXTIL t7M= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="164746392" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:20:14 +0800 IronPort-SDR: QQo8CDMWhrEjaPs4yKdBu5QpQU88qx41VZmT/0wYMJMWRkoAJlHnFfB26m8QYwQOEDL3SwfIfJ fkudOFdnw030mEEzprfBE+zvoPXjmVzQJyyzVgUx7muutUNuYRxS42f/4Uvtr1fy0aatjUMKOi 5vCnTqHq69zqlIH83HdAs68KyHPc74D9yV447foEgQ8pZlEg1FnmLZ7/NG2QaHYeBJ/SXA2ThI ZhWV+Yy6YC8n9zYIjmMxUcS9ZYPFEWdcr5naisaqgphtOddbrqhf/Fu+oi1KWnU1F0SLbgGnrR 30/+S2SetLJ8Y+0vmPcHeNIY Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:03:40 -0800 IronPort-SDR: zciRZvXO8859SfE+LG2hlno/Hn2q2UMteYKVnt+GrJBVkPTQlwvsQiyp1TwH5UDTW6dEun8T6Q B2iSeurFhoCsYRZeYJQiqXnSvhS7x2fgKPsp/mGqMeUNpBwRelCcDd/XXtCnvQEmcFeLD8QM+R nxwp5/cZS9PSPHBY4QECN/Diiv63FeirATbIy2JilFLuE2Wbxw7z278BYfijbRfcnJZheUtEcb grJiC3gBb8ize+yWM2yxzkweq2TZnA9KoibYP0NGMRi5wh9ZY+bwweqQlohuxOTES3qL6Crjqn PI0= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:20:10 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 1/9] scsi: ufshpb: Cache HPB Control mode on init Date: Thu, 18 Feb 2021 15:19:24 +0200 Message-Id: <20210218131932.106997-2-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org We will use it later, when we'll need to differentiate between device and host control modes. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshcd.h | 2 ++ drivers/scsi/ufs/ufshpb.c | 8 +++++--- drivers/scsi/ufs/ufshpb.h | 2 ++ 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ufs/ufshcd.h b/drivers/scsi/ufs/ufshcd.h index 49bfebf366c7..1a8dcdaa357a 100644 --- a/drivers/scsi/ufs/ufshcd.h +++ b/drivers/scsi/ufs/ufshcd.h @@ -655,6 +655,7 @@ struct ufs_hba_variant_params { * @slave_conf_cnt: counter to check all lu finished initialization * @hpb_disabled: flag to check if HPB is disabled * @max_hpb_single_cmd: maximum size of single HPB command + * @control_mode: either host or device */ struct ufshpb_dev_info { int num_lu; @@ -663,6 +664,7 @@ struct ufshpb_dev_info { atomic_t slave_conf_cnt; bool hpb_disabled; int max_hpb_single_cmd; + u8 control_mode; }; #endif diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index e97a78400f68..f9140f3a4eed 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -1522,6 +1522,9 @@ static void ufshpb_lu_parameter_init(struct ufs_hba *hba, % (hpb->srgn_mem_size / HPB_ENTRY_SIZE); hpb->pages_per_srgn = DIV_ROUND_UP(hpb->srgn_mem_size, PAGE_SIZE); + + if (hpb_dev_info->control_mode == HPB_HOST_CONTROL) + hpb->is_hcm = true; } static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) @@ -2205,11 +2208,10 @@ void ufshpb_get_dev_info(struct ufs_hba *hba, u8 *desc_buf) { struct ufshpb_dev_info *hpb_dev_info = &hba->ufshpb_dev; int version, ret; - u8 hpb_mode; u32 max_hpb_sigle_cmd = 0; - hpb_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CONTROL]; - if (hpb_mode == HPB_HOST_CONTROL) { + hpb_dev_info->control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CONTROL]; + if (hpb_dev_info->control_mode == HPB_HOST_CONTROL) { dev_err(hba->dev, "%s: host control mode is not supported.\n", __func__); hpb_dev_info->hpb_disabled = true; diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index eb8366d47d8a..f55a71b250eb 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -223,6 +223,8 @@ struct ufshpb_lu { u32 entries_per_srgn_shift; u32 pages_per_srgn; + bool is_hcm; + struct ufshpb_stats stats; struct ufshpb_params params; From patchwork Thu Feb 18 13:19:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 385157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3CD6C433E0 for ; Thu, 18 Feb 2021 17:00:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A304864EAF for ; Thu, 18 Feb 2021 17:00:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234159AbhBRQ6i (ORCPT ); Thu, 18 Feb 2021 11:58:38 -0500 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:2868 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233283AbhBRN13 (ORCPT ); Thu, 18 Feb 2021 08:27:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654849; x=1645190849; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yom8YghvRy3JpvoVcsokM4ePBjNz+gLkJ3acr37Efdk=; b=PpiGFAWAcXRnH+vBXRD+o/tCDRR19LFYpcWIa/MxiaIsdx0v4J5Fl3mK YO+yRrmNRI7w2MSviACoxpeQlWXrVnG/GZnKpCyogzLAVGGYOWD9V2mpt MaAOsTjN1BQUTq2inQYotoHsiChjGMwWoG30Y2CdvmEy8X3xWNY/n4H2S IBXR3AfKeK2iqFFJKpmqnAdS3xq1urONGNmUdlp1Hl7Ks5JhJj+rPo6Ex 67XgU3BYwZejjS46J/N8LXA6QJpam+tNcV0fekEtJr4wC/Hb2AHnFNFe7 /w9DBQBDJX9/YYN4Fv6tPjEEWLsD2nDFS3qIYQVDamyc4PmL89mFj18LS g==; IronPort-SDR: 2byqv33m54iO/TB8GpPY3PuSL38Xj6/fee5L9X6yKjCUDnLILE7KLV77HI4KiWqEGYEnWgtECs WRCH2pBluEcGP7g+5vYP3tMFUerA+8geRAkHUGJGoQECuvWm03MuXbIU6AJgOkzppRp6sec95k Ryr2IkVHILAAxP+Nb4/rmtjiXlcr6T7awKebvXawpVyafC6TdbB/fWsahzL4lg45oD3D2Xfo32 rLAQWo+b5UqX0D/NiglGBnFczfNql4a2xTZhUYZj22CCYz8IEjt++sXGwcJOXfXVL+vpewRKAe 61Y= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="161433332" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:20:26 +0800 IronPort-SDR: f8hu879gzibLoSfp/qQY1mxxnbptR8Z3UNDEYKDq1OPbkrj44NrJiLV3D6rmlTHw56FzD+tRyn 7K0Dx91vCrMiMOGvKM+yjXBJg6qY9epZxRJ/pgGWkcaZFd6PwdXaFejSfasP4gnQvfn6x4ckW4 y+zy2Jw4/oUMBjdgYR2AwIqN4tajSNmINEpDjP0qtj+25a6qEAIFqQY8OxYAVMt7repIRNfJZK qns+CS49pOz3RIxTMAhJT3uwGHRgQs0FwysZENxHEMoxU2xxdLvyKul0BiJsmUcBB4OQZZkRWj 9KsqpUPjK3WTwZvFG8ijcB8M Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:03:51 -0800 IronPort-SDR: QcR3SqhsaGKhpkT+YOmErbYkbZ347LMzRkyEjeH1X+LWLF4j8y57iM8KGBOZLuxHs8zwgqlyzb 9ZzOEtzz/u45AQAdI7O/zyRDn3ws7yipFsHZ8JfcT09MUWlBd4qz6GJyIJ/k+u+aQKF7A9V8qg rsAUp2AzzwaJ6IB/Ci/Ifdi+422RzTvtS9WK1/TM5+X3ScxDOsVFoSUoFd+Xxfyxa4cA5QRqdA emiQi5IjWvlgeyE9hchpplRjwhsWc9vaa6IP8nE8+lWVy4EiB9ZW2S30JeDrwDBBWztTysAhR3 8Uw= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:20:21 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 2/9] scsi: ufshpb: Add host control mode support to rsp_upiu Date: Thu, 18 Feb 2021 15:19:25 +0200 Message-Id: <20210218131932.106997-3-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In device control mode, the device may recommend the host to either activate or inactivate a region, and the host should follow. Meaning those are not actually recommendations, but more of instructions. On the contrary, in host control mode, the recommendation protocol is slightly changed: a) The device may only recommend the host to update a subregion of an already-active region. And, b) The device may *not* recommend to inactivate a region. Furthermore, in host control mode, the host may choose not to follow any of the device's recommendations. However, in case of a recommendation to update an active and clean subregion, it is better to follow those recommendation because otherwise the host has no other way to know that some internal relocation took place. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 34 +++++++++++++++++++++++++++++++++- drivers/scsi/ufs/ufshpb.h | 2 ++ 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index f9140f3a4eed..e052260868ad 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -156,6 +156,8 @@ static void ufshpb_set_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, else set_bit_len = cnt; + set_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + if (rgn->rgn_state != HPB_RGN_INACTIVE && srgn->srgn_state == HPB_SRGN_VALID) bitmap_set(srgn->mctx->ppn_dirty, srgn_offset, set_bit_len); @@ -222,6 +224,11 @@ static bool ufshpb_test_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, return false; } +static inline bool is_rgn_dirty(struct ufshpb_region *rgn) +{ + return test_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); +} + static int ufshpb_fill_ppn_from_page(struct ufshpb_lu *hpb, struct ufshpb_map_ctx *mctx, int pos, int len, u64 *ppn_buf) @@ -715,6 +722,7 @@ static void ufshpb_put_map_req(struct ufshpb_lu *hpb, static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, struct ufshpb_subregion *srgn) { + struct ufshpb_region *rgn; u32 num_entries = hpb->entries_per_srgn; WARN_ON(!srgn->mctx); @@ -723,6 +731,10 @@ static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, num_entries = hpb->last_srgn_entries; bitmap_zero(srgn->mctx->ppn_dirty, num_entries); + + rgn = hpb->rgn_tbl + srgn->rgn_idx; + clear_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + return 0; } @@ -1171,6 +1183,18 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, srgn_i = be16_to_cpu(rsp_field->hpb_active_field[i].active_srgn); + rgn = hpb->rgn_tbl + rgn_i; + if (hpb->is_hcm && + (rgn->rgn_state != HPB_RGN_ACTIVE || is_rgn_dirty(rgn))) { + /* + * in host control mode, subregion activation + * recommendations are only allowed to active regions. + * Also, ignore recommendations for dirty regions - the + * host will make decisions concerning those by himself + */ + continue; + } + dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, "activate(%d) region %d - %d\n", i, rgn_i, srgn_i); @@ -1178,7 +1202,6 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, ufshpb_update_active_info(hpb, rgn_i, srgn_i); spin_unlock(&hpb->rsp_list_lock); - rgn = hpb->rgn_tbl + rgn_i; srgn = rgn->srgn_tbl + srgn_i; /* blocking HPB_READ */ @@ -1189,6 +1212,14 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, hpb->stats.rb_active_cnt++; } + if (hpb->is_hcm) { + /* + * in host control mode the device is not allowed to inactivate + * regions + */ + goto out; + } + for (i = 0; i < rsp_field->inactive_rgn_cnt; i++) { rgn_i = be16_to_cpu(rsp_field->hpb_inactive_field[i]); dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, @@ -1213,6 +1244,7 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, hpb->stats.rb_inactive_cnt++; } +out: dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, "Noti: #ACT %u #INACT %u\n", rsp_field->active_rgn_cnt, rsp_field->inactive_rgn_cnt); diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index f55a71b250eb..8e6c2a5b80a0 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -115,6 +115,8 @@ struct ufshpb_region { /* below information is used by lru */ struct list_head list_lru_rgn; + unsigned long rgn_flags; +#define RGN_FLAG_DIRTY 0 }; #define for_each_sub_region(rgn, i, srgn) \ From patchwork Thu Feb 18 13:19:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 385160 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FD8FC433E6 for ; Thu, 18 Feb 2021 16:59:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DA8F64EAD for ; Thu, 18 Feb 2021 16:59:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234084AbhBRQ5Z (ORCPT ); Thu, 18 Feb 2021 11:57:25 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:5492 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233255AbhBRN0I (ORCPT ); Thu, 18 Feb 2021 08:26:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654923; x=1645190923; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7Z5gobiy/8FkE/TYphPMCbzPJlRPUDrnk+T3bO1VJgs=; b=YI7BvkHVjF32+WKpHi8KkSM9crKY71B006eWSfCuR+oWl38pW47+I9dD /Vf5HxSX39s5gDhSFJVwOQNR1tIGjtMQjGaLq9H7dJ8mIBFfIL/bkWNwY hMbiMZ3r01jV36YTFPyip/id5lEXBpZm9/t+6yTIv1AiwqlDDWk0vwFJM ylxRbM4UTaRfpszSvTODhqSO4Ka/49pfXsQX9RL5R0Lb2Z4oO8oU08x9l 34Wto74afGl+cZVY7PbMQpUnjfoYjq30wppUyucboCyQMb0E0Vgnp5b2o YCKOUy+eGLdZdYFngkADJDrUHVdnF7dmXf3mpgTK0AK2JTrM/ku+WDZ8R A==; IronPort-SDR: VPiZQ2urwXMOxw1xhwe9Q+1Ii7Q/NVQP2TgNZcrvVrsmu4l82tRjFkSC3ZJBl1f2zsv7cplfOL e9Gu+i78UEJD5OxpPvUaP9KTRgJS0brfKH8m3veM7B2I+VNV5oevcwy+9uWUflH71Z3Z4vNb5I /rdxKtuwym3K1SG4fZwGHLsdunTGivOI/lwsZMC9Y5JM1kmGQdhnJ8BZVamrmQYcMwdW4dTU7o qPebyZIOgNAlMugmqtg+2dwLgH/4xxYY0Za26R8N/Wt74bKuLlzZcdMa7tKf/QRTNMW7beFK8b NZc= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="264440596" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:40:45 +0800 IronPort-SDR: J92JhHaq4y8c2VzWPsnaT5il+c99rUTE0yJw4xso0e0mc5Pw3wphkEQgc2fIPpB4niDcziCtD4 ZbhZPronLCvTVWktI/fM3tFeN60qqj2EomTNNlP5ZTrgCLqZc0Gxe2mlNtCOssDDyfQ0r6RpqD /RzaOQ7c46UWhPbbZ+Yuei8o2Z2Qc+JcjeDjtIb5aflqPwGtp943o4Dj5kXMQlrdQd9xCxPg9Q BQ9eLX+Bg1PA+av6RDW48O+mNqTHJdONBtUrvWq2PkemPsxNFby5xzdZp0Js3yQ3lNTUgBZ5b7 TG3Qoz8Oou92FhAq0/AwcXuZ Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:02:18 -0800 IronPort-SDR: SuMQNxoMJZmionB4ubMvOKoud5sKbJoDcoDMqEtkGcOC9fvtLG/jYDzy3onjSMWfV+g0wFtGj5 d6I1CRNX+GughXuENc36PNhXVLrjqb/rrtX6Sn8+iJ5VYpBOlmgIt2Vz0TQOvYKzVePaRC6sxa cnHIBFjT19XlGgDT2nKj8U2FhbYjZPYyDh2b3VzXz5Eples7zMv0gFSK+LsDQEsOA9SylQ0QLN OMMWWEhvmX0tQcfLfn8vh84AqYH1lhtQ5EUdvyHlMTxTIL+xyyPFDi0IeBTy3RPSTiE1HX3OAe b8Y= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:20:34 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 3/9] scsi: ufshpb: Add region's reads counter Date: Thu, 18 Feb 2021 15:19:26 +0200 Message-Id: <20210218131932.106997-4-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host control mode, reads are the major source of activation trials. Keep track of those reads counters, for both active as well inactive regions. We reset the read counter upon write - we are only interested in "clean" reads. less intuitive however, is that we also reset it upon region's deactivation. Region deactivation is often due to the fact that eviction took place: a region become active on the expense of another. This is happening when the max-active-regions limit has crossed. If we don’t reset the counter, we will trigger a lot of trashing of the HPB database, since few reads (or even one) to the region that was deactivated, will trigger a re-activation trial. Keep those counters normalized, as we are using those reads as a comparative score, to make various decisions. If during consecutive normalizations an active region has exhaust its reads - inactivate it. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 108 ++++++++++++++++++++++++++++++++------ drivers/scsi/ufs/ufshpb.h | 7 +++ 2 files changed, 100 insertions(+), 15 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index e052260868ad..348185964c32 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -16,6 +16,8 @@ #include "ufshpb.h" #include "../sd.h" +#define ACTIVATION_THRESHOLD 4 /* 4 IOs */ + /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; static mempool_t *ufshpb_mctx_pool; @@ -570,6 +572,21 @@ static int ufshpb_issue_pre_req(struct ufshpb_lu *hpb, struct scsi_cmnd *cmd, return ret; } +static void ufshpb_update_active_info(struct ufshpb_lu *hpb, int rgn_idx, + int srgn_idx) +{ + struct ufshpb_region *rgn; + struct ufshpb_subregion *srgn; + + rgn = hpb->rgn_tbl + rgn_idx; + srgn = rgn->srgn_tbl + srgn_idx; + + list_del_init(&rgn->list_inact_rgn); + + if (list_empty(&srgn->list_act_srgn)) + list_add_tail(&srgn->list_act_srgn, &hpb->lh_act_srgn); +} + /* * This function will set up HPB read command using host-side L2P map data. */ @@ -616,12 +633,45 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) ufshpb_set_ppn_dirty(hpb, rgn_idx, srgn_idx, srgn_offset, transfer_len); spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + + if (hpb->is_hcm) { + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads = 0; + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + } + return 0; } if (!ufshpb_is_support_chunk(hpb, transfer_len)) return 0; + if (hpb->is_hcm) { + bool activate = false; + /* + * in host control mode, reads are the main source for + * activation trials. + */ + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads++; + if (rgn->reads == ACTIVATION_THRESHOLD) + activate = true; + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + if (activate) { + spin_lock_irqsave(&hpb->rsp_list_lock, flags); + ufshpb_update_active_info(hpb, rgn_idx, srgn_idx); + hpb->stats.rb_active_cnt++; + spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); + dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, + "activate region %d-%d\n", rgn_idx, srgn_idx); + } + + /* keep those counters normalized */ + if (rgn->reads > hpb->entries_per_srgn && + !test_and_set_bit(WORK_PENDING, &hpb->work_data_bits)) + schedule_work(&hpb->ufshpb_normalization_work); + } + spin_lock_irqsave(&hpb->rgn_state_lock, flags); if (ufshpb_test_ppn_dirty(hpb, rgn_idx, srgn_idx, srgn_offset, transfer_len)) { @@ -738,21 +788,6 @@ static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, return 0; } -static void ufshpb_update_active_info(struct ufshpb_lu *hpb, int rgn_idx, - int srgn_idx) -{ - struct ufshpb_region *rgn; - struct ufshpb_subregion *srgn; - - rgn = hpb->rgn_tbl + rgn_idx; - srgn = rgn->srgn_tbl + srgn_idx; - - list_del_init(&rgn->list_inact_rgn); - - if (list_empty(&srgn->list_act_srgn)) - list_add_tail(&srgn->list_act_srgn, &hpb->lh_act_srgn); -} - static void ufshpb_update_inactive_info(struct ufshpb_lu *hpb, int rgn_idx) { struct ufshpb_region *rgn; @@ -998,6 +1033,14 @@ static void __ufshpb_evict_region(struct ufshpb_lu *hpb, ufshpb_cleanup_lru_info(lru_info, rgn); + if (hpb->is_hcm) { + unsigned long flags; + + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads = 0; + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + } + for_each_sub_region(rgn, srgn_idx, srgn) ufshpb_purge_active_subregion(hpb, srgn); } @@ -1420,6 +1463,36 @@ static void ufshpb_run_inactive_region_list(struct ufshpb_lu *hpb) spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); } +static void ufshpb_normalization_work_handler(struct work_struct *work) +{ + struct ufshpb_lu *hpb; + int rgn_idx; + + hpb = container_of(work, struct ufshpb_lu, ufshpb_normalization_work); + + for (rgn_idx = 0; rgn_idx < hpb->rgns_per_lu; rgn_idx++) { + struct ufshpb_region *rgn = hpb->rgn_tbl + rgn_idx; + + if (rgn->reads) { + unsigned long flags; + + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads = (rgn->reads >> 1); + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + } + + if (rgn->rgn_state != HPB_RGN_ACTIVE || rgn->reads) + continue; + + /* if region is active but has no reads - inactivate it */ + spin_lock(&hpb->rsp_list_lock); + ufshpb_update_inactive_info(hpb, rgn->rgn_idx); + spin_unlock(&hpb->rsp_list_lock); + } + + clear_bit(WORK_PENDING, &hpb->work_data_bits); +} + static void ufshpb_map_work_handler(struct work_struct *work) { struct ufshpb_lu *hpb = container_of(work, struct ufshpb_lu, map_work); @@ -1810,6 +1883,9 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&hpb->list_hpb_lu); INIT_WORK(&hpb->map_work, ufshpb_map_work_handler); + if (hpb->is_hcm) + INIT_WORK(&hpb->ufshpb_normalization_work, + ufshpb_normalization_work_handler); hpb->map_req_cache = kmem_cache_create("ufshpb_req_cache", sizeof(struct ufshpb_req), 0, 0, NULL); @@ -1909,6 +1985,8 @@ static void ufshpb_discard_rsp_lists(struct ufshpb_lu *hpb) static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { + if (hpb->is_hcm) + cancel_work_sync(&hpb->ufshpb_normalization_work); cancel_work_sync(&hpb->map_work); } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 8e6c2a5b80a0..0a41df919f06 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -117,6 +117,10 @@ struct ufshpb_region { struct list_head list_lru_rgn; unsigned long rgn_flags; #define RGN_FLAG_DIRTY 0 + + /* region reads - for host mode */ + spinlock_t rgn_lock; + unsigned int reads; }; #define for_each_sub_region(rgn, i, srgn) \ @@ -207,6 +211,9 @@ struct ufshpb_lu { /* for selecting victim */ struct victim_select_info lru_info; + struct work_struct ufshpb_normalization_work; + unsigned long work_data_bits; +#define WORK_PENDING 0 /* pinned region information */ u32 lu_pinned_start; From patchwork Thu Feb 18 13:19:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 385159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DE9BC433E0 for ; Thu, 18 Feb 2021 16:59:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 670B664EB6 for ; Thu, 18 Feb 2021 16:59:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234123AbhBRQ6H (ORCPT ); Thu, 18 Feb 2021 11:58:07 -0500 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:30076 "EHLO esa5.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233384AbhBRN0l (ORCPT ); Thu, 18 Feb 2021 08:26:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654800; x=1645190800; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1LJBjYU69njB65xIZ3s0koQEhLEywwRydvSxE8Dq5TY=; b=grck9ATkjstu8IvNtZ3q0vZDe5cgNbBd9p06RDram/UeqxNij4+ldI5U O3VOKEwJyD5bEJHKeDSeNanYDQ/A6RLwR9qeSACEy8Twzi++eZ53vUm3L eDrAe4S689uACHVy96JXj0WNrpNuehhTQJDxbuzNt7R7f4T9OxjO5TKRM RDIWt4RyAB8/nqPY7+2d5kZYHJZrZrQN9Kk+WTvVkUbTD8FvoRNK+FYB+ P/okWHAJWaJl6pwXlpcH6RUpldUSd5PJYr6FwGTBzKpBMYY+RybkT1arV uk7FT04Yn/wt30EvEPU7lDdyz2ZT2boCkRm1FcLSgmSJLb1j5329NzhWt g==; IronPort-SDR: bu4wAJL+C1W/ZBQmrbXUvppPWxXU96xqITadjDdYRIGmsYwrAkAoJ54UgOC4q0aTX2dluKvx1b Uq8HYOWD+C6z+LwkN0TTD1YR0w0hJwcdkRH1sHykIr6I14qauX5yeQyqJ6L82Q90ANJ9Xs9zih /xUyWcufPHwlyIJotsXOhjgyLs74u5QRfLODYoc/FsJsyEMLk50s8SVpINC+Nw+3xGk6WhJ6T7 ruGzw5PMiHwpopC5qvsM5c3dZTbYO9rIlEYV/CAMbm7sVKjKT70GXDtkXiDD5EcQ+uNzLjRGGg o70= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="160277218" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:20:48 +0800 IronPort-SDR: 36/tyPWbgdWIH/fiYJJx45b/cQBZgQczIHP3JEBtV1JMrEIJiUnBjyMpVcU6BBuSrp3JhUTkSS RSdFLhOYAB6mkZjfYyul7SCDqZndOvH96UrtCoYPtUpadQs5fchanxBTr7bMVDUWw1y/1PMM66 oBVHhvKrRE2MYD9WtkRoL+JADpcD3/Xzn5jCabTE8zd9u1mdqj0wOAstm1a2qj2vIPWmqjPOus Zslju5l/7icHO3nvMBLuELyGA4QEsE+7BxibU9D4Hr0qP5X8qY6206N8ytIAPX6+nRo3XsPInW bqks2kSH2/wS3PM6I245tCAD Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:04:14 -0800 IronPort-SDR: 6CM1oLgCbxZF9hau8fAPMMnyHg/vcrf7CtpVj7B+45WR+T8vgoJkAOCIKC/KFy1a2b0ECax2Cl KI45lmjwSjal6obqSe6Qp1E/h8jGTJDtGj6qffGte/Xd6Au007NJ8HFrMrZ+I1mwIR6K/LuiE+ IYPara+PDuRnT0GF7th6vNA/U5vAowe0LsXI6yJcF1NoBsFIeozNLXAlSrAlSF2QBHSJopIL7f U6AbezCo9nZZT+ca92HisUboJuXy1TN2A/3ponFJf2yGs64LrL30X3FDL6tMt5z25A3LbZQgEH UkA= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:20:44 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 4/9] scsi: ufshpb: Make eviction depends on region's reads Date: Thu, 18 Feb 2021 15:19:27 +0200 Message-Id: <20210218131932.106997-5-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host mode, eviction is considered an extreme measure. verify that the entering region has enough reads, and the exiting region has much less reads. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 348185964c32..9cb17c97cf93 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -17,6 +17,7 @@ #include "../sd.h" #define ACTIVATION_THRESHOLD 4 /* 4 IOs */ +#define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 6) /* 256 IOs */ /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; @@ -995,6 +996,13 @@ static struct ufshpb_region *ufshpb_victim_lru_info(struct ufshpb_lu *hpb) if (ufshpb_check_srgns_issue_state(hpb, rgn)) continue; + /* + * in host control mode, verify that the exiting region + * has less reads + */ + if (hpb->is_hcm && rgn->reads > (EVICTION_THRESHOLD >> 1)) + continue; + victim_rgn = rgn; break; } @@ -1150,7 +1158,7 @@ static int ufshpb_issue_map_req(struct ufshpb_lu *hpb, static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) { - struct ufshpb_region *victim_rgn; + struct ufshpb_region *victim_rgn = NULL; struct victim_select_info *lru_info = &hpb->lru_info; unsigned long flags; int ret = 0; @@ -1178,6 +1186,16 @@ static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) * because the device could detect this region * by not issuing HPB_READ */ + + /* + * in host control mode, verify that the entering + * region has enough reads + */ + if (hpb->is_hcm && rgn->reads < EVICTION_THRESHOLD) { + ret = -EACCES; + goto out; + } + victim_rgn = ufshpb_victim_lru_info(hpb); if (!victim_rgn) { dev_warn(&hpb->sdev_ufs_lu->sdev_dev, From patchwork Thu Feb 18 13:19:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 384710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0960EC432C3 for ; Thu, 18 Feb 2021 16:59:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C4B7F64EBB for ; Thu, 18 Feb 2021 16:59:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234134AbhBRQ6O (ORCPT ); Thu, 18 Feb 2021 11:58:14 -0500 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:2097 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232308AbhBRN0m (ORCPT ); Thu, 18 Feb 2021 08:26:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654800; x=1645190800; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2SzdTpY078B1u2ZtuNLU5TzOEvdFKNJ83j2vTItjfjA=; b=dzEInwJmY5JCHIUz62C5VhK7g/DFfu7HbdMJr9fOHxU1pSyMNuU3FBKR RaGzTVG+QZAITlniv8ZfN1CkwawtSAv3ZDGGdjkOkHeD9lovj2CELSHmw UcdoGUI0WlQXT0+RTiVSmxGRYn3ClKPbcNn1Xl0uLMNAPgTPY8nJwJUpd y4U8NKkcJkMmVP2WZRdkSsNQH779pT+suYIMeQ6M35z3xBrj56c29cXVq tQPc06kAOnXYaYKjWQ4PH1HUsSgSkzibRHwpH96l4/JHU68ONVHHkQWKJ ZcLY6VsvF1zwD/ym7U/Cqc863nbeQ8wqLon+UEmBCP0CgT2FiHQWCHpSv Q==; IronPort-SDR: qdfiHXqaZh4mqJ/Owqc9Wj5N/SOOp8ZfApNOqzoLg6t++Q79JJvVBUo44L0cxDqa4OKC6n9r5O nBpAm9EQEglkuA0/h2io4AXW7ctOKIMIyqK1FWm1Q+9IsIhwmXa/OC91Xs4VHdtl6kMqHCXB38 7D/msdG83uyQzrF14RB0ekuOGY7ZIBSBNq7l8u30rN6ApgHv5HOlc7miquVB5BkQLRNkhgkQFO 8cZle2d364zhtZ05rEBzDMUPys9BHtEQI0crMdQ7PYsDoOjFWeSdvIYtPH28sAehUuTRAHYBav 2+w= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="164746437" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:20:59 +0800 IronPort-SDR: LpUXv1CzAh3Dmejfj1zL4mydo2elw6pcf29t1z037TSj4WLey7ASW/x18UVbMMDqb6uiJpQcwk iE400JyEfwCZNKLki7uJ3a5ijV8u1Al+BppG/irMMzDmuZx/O0egmjvm3TzSYXAJlSML32dEkW vqrHAYJeuiou9TC2AA5UizKW31J3Wbcg3gNnsj6CUT5UT5ss0vmRZpH7N2g5Yl2PbaBBaMe0AP zqdGdeEbIJu9OmIF1hr1vUaSMChWI6RBftOpcmieNLTPakDvALpiKaUrSlpgoh0zdA64rVby1z SFxWEy4WiMd/r0iBlbhy8loL Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:02:39 -0800 IronPort-SDR: JUoKiuFrdzc4tXp6MCw7/MeVyMJ/dcAWbWXru3ZOdiFVB1k75+y7AgzQgZi+hHY4dLphPoBB+9 w6/CRVfT6EFAGkFBFr/Z5MXDcsmPCgpdka1t/vLUpzqjwh1bEwxRZah2Htacji5RMEtXDr7SpI Q4wwSO815BLQe+EtbB+hOaiT2UqhwCP4MxiD7Khd76OZjsUjuSYD8r5C89qA0c6xecbbzIF/kJ AR8Q0FZa9F7IqQABNZWBvFuUkEpeVTGHv1XBTX8y6E5kOPUBTTdl/xmOKcb0CbA+57RkIKAyeR PIo= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:20:55 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 5/9] scsi: ufshpb: Region inactivation in host mode Date: Thu, 18 Feb 2021 15:19:28 +0200 Message-Id: <20210218131932.106997-6-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org I host mode, the host is expected to send HPB-WRITE-BUFFER with buffer-id = 0x1 when it inactivates a region. Use the map-requests pool as there is no point in assigning a designated cache for umap-requests. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 120 ++++++++++++++++++++++++++++++++------ drivers/scsi/ufs/ufshpb.h | 5 +- 2 files changed, 105 insertions(+), 20 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 9cb17c97cf93..3435a5507853 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -717,16 +717,15 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) return 0; } -static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb, - struct ufshpb_subregion *srgn) +static struct ufshpb_req *ufshpb_get_req(struct ufshpb_lu *hpb, + int rgn_idx, enum req_opf dir) { - struct ufshpb_req *map_req; + struct ufshpb_req *rq; struct request *req; - struct bio *bio; int retries = HPB_MAP_REQ_RETRIES; - map_req = kmem_cache_alloc(hpb->map_req_cache, GFP_KERNEL); - if (!map_req) + rq = kmem_cache_alloc(hpb->map_req_cache, GFP_KERNEL); + if (!rq) return NULL; retry: @@ -738,36 +737,57 @@ static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb, goto retry; } + req = blk_get_request(hpb->sdev_ufs_lu->request_queue, + dir, 0); if (IS_ERR(req)) - goto free_map_req; + goto free_rq; + + rq->hpb = hpb; + rq->req = req; + rq->rb.rgn_idx = rgn_idx; + + return rq; + +free_rq: + kmem_cache_free(hpb->map_req_cache, rq); + return NULL; +} + +static void ufshpb_put_req(struct ufshpb_lu *hpb, struct ufshpb_req *rq) +{ + blk_put_request(rq->req); + kmem_cache_free(hpb->map_req_cache, rq); +} + +static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb, + struct ufshpb_subregion *srgn) +{ + struct ufshpb_req *map_req; + struct bio *bio; + + map_req = ufshpb_get_req(hpb, srgn->rgn_idx, REQ_OP_SCSI_IN); + if (!map_req) + return NULL; bio = bio_alloc(GFP_KERNEL, hpb->pages_per_srgn); if (!bio) { - blk_put_request(req); - goto free_map_req; + ufshpb_put_req(hpb, map_req); + return NULL; } - map_req->hpb = hpb; - map_req->req = req; map_req->bio = bio; - map_req->rb.rgn_idx = srgn->rgn_idx; map_req->rb.srgn_idx = srgn->srgn_idx; map_req->rb.mctx = srgn->mctx; return map_req; - -free_map_req: - kmem_cache_free(hpb->map_req_cache, map_req); - return NULL; } static void ufshpb_put_map_req(struct ufshpb_lu *hpb, - struct ufshpb_req *map_req) + struct ufshpb_req *map_req) { bio_put(map_req->bio); - blk_put_request(map_req->req); - kmem_cache_free(hpb->map_req_cache, map_req); + ufshpb_put_req(hpb, map_req); } static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, @@ -829,6 +849,13 @@ static void ufshpb_activate_subregion(struct ufshpb_lu *hpb, srgn->srgn_state = HPB_SRGN_VALID; } +static void ufshpb_umap_req_compl_fn(struct request *req, blk_status_t error) +{ + struct ufshpb_req *umap_req = (struct ufshpb_req *)req->end_io_data; + + ufshpb_put_req(umap_req->hpb, umap_req); +} + static void ufshpb_map_req_compl_fn(struct request *req, blk_status_t error) { struct ufshpb_req *map_req = (struct ufshpb_req *) req->end_io_data; @@ -847,6 +874,14 @@ static void ufshpb_map_req_compl_fn(struct request *req, blk_status_t error) ufshpb_put_map_req(map_req->hpb, map_req); } +static void ufshpb_set_unmap_cmd(unsigned char *cdb, int rgn_idx) +{ + cdb[0] = UFSHPB_WRITE_BUFFER; + cdb[1] = UFSHPB_WRITE_BUFFER_ID; + put_unaligned_be16(rgn_idx, &cdb[2]); + cdb[9] = 0x00; +} + static void ufshpb_set_read_buf_cmd(unsigned char *cdb, int rgn_idx, int srgn_idx, int srgn_mem_size) { @@ -860,6 +895,27 @@ static void ufshpb_set_read_buf_cmd(unsigned char *cdb, int rgn_idx, cdb[9] = 0x00; } +static int ufshpb_execute_umap_req(struct ufshpb_lu *hpb, + struct ufshpb_req *umap_req) +{ + struct request_queue *q; + struct request *req; + struct scsi_request *rq; + + q = hpb->sdev_ufs_lu->request_queue; + req = umap_req->req; + req->timeout = 0; + req->end_io_data = (void *)umap_req; + rq = scsi_req(req); + ufshpb_set_unmap_cmd(rq->cmd, umap_req->rb.rgn_idx); + rq->cmd_len = HPB_WRITE_BUFFER_CMD_LENGTH; + + blk_execute_rq_nowait(q, NULL, req, 1, ufshpb_umap_req_compl_fn); + + hpb->stats.umap_req_cnt++; + return 0; +} + static int ufshpb_execute_map_req(struct ufshpb_lu *hpb, struct ufshpb_req *map_req, bool last) { @@ -1028,6 +1084,25 @@ static void ufshpb_purge_active_subregion(struct ufshpb_lu *hpb, } } +static int ufshpb_issue_umap_req(struct ufshpb_lu *hpb, + struct ufshpb_region *rgn) +{ + struct ufshpb_req *umap_req; + + umap_req = ufshpb_get_req(hpb, rgn->rgn_idx, REQ_OP_SCSI_OUT); + if (!umap_req) + return -ENOMEM; + + if (ufshpb_execute_umap_req(hpb, umap_req)) + goto free_umap_req; + + return 0; + +free_umap_req: + ufshpb_put_req(hpb, umap_req); + return -EAGAIN; +} + static void __ufshpb_evict_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) { @@ -1035,6 +1110,10 @@ static void __ufshpb_evict_region(struct ufshpb_lu *hpb, struct ufshpb_subregion *srgn; int srgn_idx; + + if (hpb->is_hcm && ufshpb_issue_umap_req(hpb, rgn)) + return; + lru_info = &hpb->lru_info; dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, "evict region %d\n", rgn->rgn_idx); @@ -1759,6 +1838,7 @@ ufshpb_sysfs_attr_show_func(rb_noti_cnt); ufshpb_sysfs_attr_show_func(rb_active_cnt); ufshpb_sysfs_attr_show_func(rb_inactive_cnt); ufshpb_sysfs_attr_show_func(map_req_cnt); +ufshpb_sysfs_attr_show_func(umap_req_cnt); static struct attribute *hpb_dev_stat_attrs[] = { &dev_attr_hit_cnt.attr, @@ -1767,6 +1847,7 @@ static struct attribute *hpb_dev_stat_attrs[] = { &dev_attr_rb_active_cnt.attr, &dev_attr_rb_inactive_cnt.attr, &dev_attr_map_req_cnt.attr, + &dev_attr_umap_req_cnt.attr, NULL, }; @@ -1881,6 +1962,7 @@ static void ufshpb_stat_init(struct ufshpb_lu *hpb) hpb->stats.rb_active_cnt = 0; hpb->stats.rb_inactive_cnt = 0; hpb->stats.map_req_cnt = 0; + hpb->stats.umap_req_cnt = 0; } static void ufshpb_param_init(struct ufshpb_lu *hpb) diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 0a41df919f06..13ec49ba260d 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -36,11 +36,13 @@ /* hpb vender defined opcode */ #define UFSHPB_READ 0xF8 #define UFSHPB_READ_BUFFER 0xF9 -#define UFSHPB_READ_BUFFER_ID 0x01 #define UFSHPB_WRITE_BUFFER 0xFA +#define UFSHPB_READ_BUFFER_ID 0x01 +#define UFSHPB_WRITE_BUFFER_ID 0x01 #define UFSHPB_WRITE_BUFFER_PREFETCH_ID 0x02 #define MAX_HPB_READ_ID 0x7F #define HPB_READ_BUFFER_CMD_LENGTH 10 +#define HPB_WRITE_BUFFER_CMD_LENGTH 10 #define LU_ENABLED_HPB_FUNC 0x02 #define HPB_RESET_REQ_RETRIES 10 @@ -182,6 +184,7 @@ struct ufshpb_stats { u64 rb_inactive_cnt; u64 map_req_cnt; u64 pre_req_cnt; + u64 umap_req_cnt; }; struct ufshpb_lu { From patchwork Thu Feb 18 13:19:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 384709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8215CC433E0 for ; Thu, 18 Feb 2021 17:00:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2FD5664EAF for ; Thu, 18 Feb 2021 17:00:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234150AbhBRQ6b (ORCPT ); Thu, 18 Feb 2021 11:58:31 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:5138 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232486AbhBRN05 (ORCPT ); Thu, 18 Feb 2021 08:26:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654998; x=1645190998; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B6DrFT1ZEou0D3mCrFnzhgoBXsLD8u2OUEwahoURYsM=; b=aP7nb1ehSbbtB5f6QgipOFXC55pvXI2JJ8U7v/2Oh8gzDzm+3shBiBGg yeRDI9Y+2v8hjRjwuhW0p/teJR0ILP5Ur2n94hCnMwpeAb4sA8fYiloqp sQoa036oncUAu4HvZmRHGT3G8H3BziK2cAxuCL3LEmEUwNgVB0O+4yfCc espxBt080vYdvr6KeXs3gz0yuqSxLNHtGTfqHyNn0hw/mOAPS9QEg7Bgv a4CDnV5eRyvAbEvwYOgSLuYmZK6aS9ds+diBwjn7vYQeLWCrJXfQI5Ycx S8Czcq5FlHljvwT+Z7kClAv8upPv1hYF76llSgaSM8v1aQnijoyy+z1dh w==; IronPort-SDR: 8j0F21sGdJ+nXyFN63y4iknYslEbYO/QX8hYCflJy/9ip7sn9rJVqLz0f7U8ROKrkHRWx3KwEr li3BQyKkM2iK+5GETt2usqmxhpKb/Lwwn2wQPrcyIddl4HOQJwObLJGoVJ7smebxaA0vknORck k4CB+4mSJyZDjEictJQPruX+saJDmuac91gDkkRE8oeeo2yS9oJCe1G4wjn2X/Ac7dNAOxZaz/ I2YPWtaIcZWp+Bc9XXrmToPCyqsTv32vkgrYOXUvHYbGciFlaxtof/VzAz0NzS8/h711q930Ed o64= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="264440629" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:21:28 +0800 IronPort-SDR: pMUYDV1ZU7fa0tH4VV3s1dqpctWB+gdLlMPsmTFJK+8em6V8OAbyd0kmo0pm5Z/aBJDkygqph+ K4GSJ6DtcNxR6D8FN0OWtkheAEpEuwC408r97Cui+oU40VGgRMYtl4akG+anJXLeA3D+26aii/ RO+DBQ0ZBETPGmbugFrNC9vMOwqn4lZcY5jmNBI9gJmEQucLIRAJmJfp58+m+VYb2FbrNNC8YX o5ky63tL2vybPnDEg5f5awhEKAfanH8uQ013yQaHn5zyW97aGp8i84qwXh5PlcHzgDNfNzio/n dogV252AXiDG/1RmCnIGdbUf Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:02:58 -0800 IronPort-SDR: 8RJqXVcXLIET9cQsn2Kr9SkEwo0WPQ86vFh+bqYevi6u9GbJpabEPf/Eu6jreatZRtKKwnK4hI 9Cn5gKvbn72iOkazNwTzg1JsP9H9+9BSFRNLHtORkSTBCiwiock9ltADDfjzAhpK5D+h1Ea/7r IC6mBa3d3BfRiDRvzRXbEViBTqi8Px2dxfXKWEXTCA+YUpuefY2+3pxg8Ef7sU9ZKgaklJsz+g XyIhmFMzDyEnSEaRghHPF/avorSikGUbJ9NCaf195j1uy+P7Dcs6XFZ7wFvJZRwi2mMG+0mM9g fEQ= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:21:14 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 6/9] scsi: ufshpb: Add hpb dev reset response Date: Thu, 18 Feb 2021 15:19:29 +0200 Message-Id: <20210218131932.106997-7-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The spec does not define what is the host's recommended response when the device send hpb dev reset response (oper 0x2). We will update all active hpb regions: mark them and do that on the next read. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 40 ++++++++++++++++++++++++++++++++++++--- drivers/scsi/ufs/ufshpb.h | 3 +++ 2 files changed, 40 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 3435a5507853..15755f677097 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -658,7 +658,8 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) if (rgn->reads == ACTIVATION_THRESHOLD) activate = true; spin_unlock_irqrestore(&rgn->rgn_lock, flags); - if (activate) { + if (activate || + test_and_clear_bit(RGN_FLAG_UPDATE, &rgn->rgn_flags)) { spin_lock_irqsave(&hpb->rsp_list_lock, flags); ufshpb_update_active_info(hpb, rgn_idx, srgn_idx); hpb->stats.rb_active_cnt++; @@ -1446,6 +1447,11 @@ void ufshpb_rsp_upiu(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) case HPB_RSP_DEV_RESET: dev_warn(&hpb->sdev_ufs_lu->sdev_dev, "UFS device lost HPB information during PM.\n"); + + if (hpb->is_hcm && + !test_and_set_bit(RESET_PENDING, &hpb->work_data_bits)) + schedule_work(&hpb->ufshpb_lun_reset_work); + break; default: dev_notice(&hpb->sdev_ufs_lu->sdev_dev, @@ -1560,6 +1566,27 @@ static void ufshpb_run_inactive_region_list(struct ufshpb_lu *hpb) spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); } +static void ufshpb_reset_work_handler(struct work_struct *work) +{ + struct ufshpb_lu *hpb; + struct victim_select_info *lru_info; + struct ufshpb_region *rgn; + unsigned long flags; + + hpb = container_of(work, struct ufshpb_lu, ufshpb_lun_reset_work); + + lru_info = &hpb->lru_info; + + spin_lock_irqsave(&hpb->rgn_state_lock, flags); + + list_for_each_entry(rgn, &lru_info->lh_lru_rgn, list_lru_rgn) + set_bit(RGN_FLAG_UPDATE, &rgn->rgn_flags); + + spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + + clear_bit(RESET_PENDING, &hpb->work_data_bits); +} + static void ufshpb_normalization_work_handler(struct work_struct *work) { struct ufshpb_lu *hpb; @@ -1770,6 +1797,8 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) } else { rgn->rgn_state = HPB_RGN_INACTIVE; } + + rgn->rgn_flags = 0; } return 0; @@ -1983,9 +2012,12 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&hpb->list_hpb_lu); INIT_WORK(&hpb->map_work, ufshpb_map_work_handler); - if (hpb->is_hcm) + if (hpb->is_hcm) { INIT_WORK(&hpb->ufshpb_normalization_work, ufshpb_normalization_work_handler); + INIT_WORK(&hpb->ufshpb_lun_reset_work, + ufshpb_reset_work_handler); + } hpb->map_req_cache = kmem_cache_create("ufshpb_req_cache", sizeof(struct ufshpb_req), 0, 0, NULL); @@ -2085,8 +2117,10 @@ static void ufshpb_discard_rsp_lists(struct ufshpb_lu *hpb) static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { - if (hpb->is_hcm) + if (hpb->is_hcm) { + cancel_work_sync(&hpb->ufshpb_lun_reset_work); cancel_work_sync(&hpb->ufshpb_normalization_work); + } cancel_work_sync(&hpb->map_work); } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 13ec49ba260d..a3b85d2df224 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -119,6 +119,7 @@ struct ufshpb_region { struct list_head list_lru_rgn; unsigned long rgn_flags; #define RGN_FLAG_DIRTY 0 +#define RGN_FLAG_UPDATE 1 /* region reads - for host mode */ spinlock_t rgn_lock; @@ -215,8 +216,10 @@ struct ufshpb_lu { /* for selecting victim */ struct victim_select_info lru_info; struct work_struct ufshpb_normalization_work; + struct work_struct ufshpb_lun_reset_work; unsigned long work_data_bits; #define WORK_PENDING 0 +#define RESET_PENDING 1 /* pinned region information */ u32 lu_pinned_start; From patchwork Thu Feb 18 13:19:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 385158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59C79C43219 for ; Thu, 18 Feb 2021 16:59:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2043A64EAD for ; Thu, 18 Feb 2021 16:59:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234140AbhBRQ6Y (ORCPT ); Thu, 18 Feb 2021 11:58:24 -0500 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:29743 "EHLO esa5.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232544AbhBRN05 (ORCPT ); Thu, 18 Feb 2021 08:26:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654816; x=1645190816; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x18Znh/eyS+r6PchW22MgvwY6wKosNieDBsoZ+mEnvA=; b=HDX9DHTgeTJQsN+0QdmskZ9hkLPVF45QRHlFbE4PVb3qpFzAUSglLQXA ejC9dv2ISi+M9bXLbi0R7TnwfLGiPspyqI0Tel5twYMX7SORIGjnOuT9W z2lYmQvpUwH12IDGaf5Dx2zs02Gn+sSO147d6/1bXfzoIdHKJ0vnWsAbu rGI3AmMmNU8tH6nVtKWjdILLy0LyQ5ZUBxb8WSEq6bTzqMgbopwcnbJIa zIlpVkRBv16320y2jN3+gO4HZoeNz/4BABvZ5dn5Jia0aKfzj1V1jSbn0 P42BWL21Ekzif2Q3VAIB3fZFJuFRhojPGWqcerZnAVcRJmlzYIH1rPyG5 Q==; IronPort-SDR: NeTuhu519lS22iRcqC0bjRg9csyabwQenq8Dt7F8Ca4tqIHF6+VXA52jwP3fcTCWG709epD2E0 nSBGau2TWRo0HqRDPiJE+GXfauLxo+RskGnuiUTiPDjwsJBkBFeBB4zlFeAK1hdiAjOzLwPKzs 7IVWxQnHZmBnAuKoEo+RfSvNOLpgyeBH0iqGmroTfUMXspzKCc4SSmnyQgmQ9YnKHoNNh7z7Kx PKk+q8sGjAt+ku9C2j3oXR7qFKd6UHOUHc3mhHRgLVmkBF9mXz1yElx3MCKequTot+cMdmIRkl 17U= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="160277233" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:21:28 +0800 IronPort-SDR: oEDoTX5Fry1I8kZZiRf9Cl2SpnL7MDWm0Tk6EGft/1d12iUiYY/PJnJqRAHQLbCOw8HEwWxYrT H5a4Ic93/wT9ZBDgEAiaYBW+dvkNgJc+UBTxG+iVUap6Hqlg76j5mAA0q2/tQfEzV5Kd4e99Vr pmoMoa2SkFDCXLJwRqVKfm/2GJK9fv0QiGPBVMwGENRWo9rjEDLOZ/VaLDnLRDggUpwKzskdYK tlh+mlJPlwJTYQxKEp2gm8KbIGBPZURqCSLxnxCg5rx8J9PdL328kNRoZxBd3B8s50tzPG28nX u3XmtZhAA9fKZxh8TAOdIgT4 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:04:54 -0800 IronPort-SDR: Zx0QSQ/A8K9zgRhd1SXrrWcVe9csB48I/+oAF1LzTO7INkTaaioD37VBXB/fWs76j5C8VE/Vr1 PL4a4TJ5imT0xKGDfyWUqaKtIsuYbu4bZT8LBuT/GfOj71FOEHLc4g9QCX+TmyMdyRKSOWDtpY Ri6r1wwbjNUxjz/p0KwGqLUkagDsG93wvBI1S1EykoQ+Rl9OX1u9ed6Yh2CFLx6t+YUbPCtkUI iPIvEYnCGV6zZhjW6Y053mUZw6xXF5+2KqQpCC4acdcELUOcheXk/pVwFm8euWQt7XWAenVlDE s+k= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:21:23 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 7/9] scsi: ufshpb: Add "Cold" regions timer Date: Thu, 18 Feb 2021 15:19:30 +0200 Message-Id: <20210218131932.106997-8-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In order not to hang on to “cold” regions, we shall inactivate a region that has no READ access for a predefined amount of time - READ_TO_MS. For that purpose we shall monitor the active regions list, polling it on every POLLING_INTERVAL_MS. On timeout expiry we shall add the region to the "to-be-inactivated" list, unless it is clean and did not exhaust its READ_TO_EXPIRIES - another parameter. All this does not apply to pinned regions. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 70 ++++++++++++++++++++++++++++++++++++++- drivers/scsi/ufs/ufshpb.h | 7 ++++ 2 files changed, 76 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 15755f677097..9a1c525204f7 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -18,6 +18,9 @@ #define ACTIVATION_THRESHOLD 4 /* 4 IOs */ #define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 6) /* 256 IOs */ +#define READ_TO_MS 1000 +#define READ_TO_EXPIRIES 100 +#define POLLING_INTERVAL_MS 200 /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; @@ -1029,12 +1032,64 @@ static int ufshpb_check_srgns_issue_state(struct ufshpb_lu *hpb, return 0; } +static void ufshpb_read_to_handler(struct work_struct *work) +{ + struct delayed_work *dwork = to_delayed_work(work); + struct ufshpb_lu *hpb; + struct victim_select_info *lru_info; + struct ufshpb_region *rgn; + unsigned long flags; + LIST_HEAD(expired_list); + + hpb = container_of(dwork, struct ufshpb_lu, ufshpb_read_to_work); + + if (test_and_set_bit(TIMEOUT_WORK_PENDING, &hpb->work_data_bits)) + return; + + spin_lock_irqsave(&hpb->rgn_state_lock, flags); + + lru_info = &hpb->lru_info; + + list_for_each_entry(rgn, &lru_info->lh_lru_rgn, list_lru_rgn) { + bool timedout = ktime_after(ktime_get(), rgn->read_timeout); + + if (timedout) { + rgn->read_timeout_expiries--; + if (is_rgn_dirty(rgn) || + rgn->read_timeout_expiries == 0) + list_add(&rgn->list_expired_rgn, &expired_list); + else + rgn->read_timeout = ktime_add_ms(ktime_get(), + READ_TO_MS); + } + } + + spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + + list_for_each_entry(rgn, &expired_list, list_expired_rgn) { + list_del_init(&rgn->list_expired_rgn); + spin_lock_irqsave(&hpb->rsp_list_lock, flags); + ufshpb_update_inactive_info(hpb, rgn->rgn_idx); + hpb->stats.rb_inactive_cnt++; + spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); + } + + clear_bit(TIMEOUT_WORK_PENDING, &hpb->work_data_bits); + + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); +} + static void ufshpb_add_lru_info(struct victim_select_info *lru_info, - struct ufshpb_region *rgn) + struct ufshpb_region *rgn) { rgn->rgn_state = HPB_RGN_ACTIVE; list_add_tail(&rgn->list_lru_rgn, &lru_info->lh_lru_rgn); atomic_inc(&lru_info->active_cnt); + if (rgn->hpb->is_hcm) { + rgn->read_timeout = ktime_add_ms(ktime_get(), READ_TO_MS); + rgn->read_timeout_expiries = READ_TO_EXPIRIES; + } } static void ufshpb_hit_lru_info(struct victim_select_info *lru_info, @@ -1778,6 +1833,7 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&rgn->list_inact_rgn); INIT_LIST_HEAD(&rgn->list_lru_rgn); + INIT_LIST_HEAD(&rgn->list_expired_rgn); if (rgn_idx == hpb->rgns_per_lu - 1) { srgn_cnt = ((hpb->srgns_per_lu - 1) % @@ -1799,6 +1855,7 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) } rgn->rgn_flags = 0; + rgn->hpb = hpb; } return 0; @@ -2017,6 +2074,8 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) ufshpb_normalization_work_handler); INIT_WORK(&hpb->ufshpb_lun_reset_work, ufshpb_reset_work_handler); + INIT_DELAYED_WORK(&hpb->ufshpb_read_to_work, + ufshpb_read_to_handler); } hpb->map_req_cache = kmem_cache_create("ufshpb_req_cache", @@ -2051,6 +2110,10 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) ufshpb_stat_init(hpb); ufshpb_param_init(hpb); + if (hpb->is_hcm) + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); + return 0; release_pre_req_mempool: @@ -2118,6 +2181,7 @@ static void ufshpb_discard_rsp_lists(struct ufshpb_lu *hpb) static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { if (hpb->is_hcm) { + cancel_delayed_work_sync(&hpb->ufshpb_read_to_work); cancel_work_sync(&hpb->ufshpb_lun_reset_work); cancel_work_sync(&hpb->ufshpb_normalization_work); } @@ -2228,6 +2292,10 @@ void ufshpb_resume(struct ufs_hba *hba) continue; ufshpb_set_state(hpb, HPB_PRESENT); ufshpb_kick_map_work(hpb); + if (hpb->is_hcm) + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); + } } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index a3b85d2df224..75093271d12b 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -107,6 +107,7 @@ struct ufshpb_subregion { }; struct ufshpb_region { + struct ufshpb_lu *hpb; struct ufshpb_subregion *srgn_tbl; enum HPB_RGN_STATE rgn_state; int rgn_idx; @@ -124,6 +125,10 @@ struct ufshpb_region { /* region reads - for host mode */ spinlock_t rgn_lock; unsigned int reads; + /* region "cold" timer - for host mode */ + ktime_t read_timeout; + unsigned int read_timeout_expiries; + struct list_head list_expired_rgn; }; #define for_each_sub_region(rgn, i, srgn) \ @@ -217,9 +222,11 @@ struct ufshpb_lu { struct victim_select_info lru_info; struct work_struct ufshpb_normalization_work; struct work_struct ufshpb_lun_reset_work; + struct delayed_work ufshpb_read_to_work; unsigned long work_data_bits; #define WORK_PENDING 0 #define RESET_PENDING 1 +#define TIMEOUT_WORK_PENDING 2 /* pinned region information */ u32 lu_pinned_start; From patchwork Thu Feb 18 13:19:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 384713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26053C433E6 for ; Thu, 18 Feb 2021 16:57:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D41C364EB7 for ; Thu, 18 Feb 2021 16:57:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233509AbhBRQ41 (ORCPT ); Thu, 18 Feb 2021 11:56:27 -0500 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:6374 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233468AbhBRNYP (ORCPT ); Thu, 18 Feb 2021 08:24:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654654; x=1645190654; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jQxGAGS6CiCS/3erGLa+UPWM+FLQsPmPWWd5FPecgfw=; b=ZA45jntjAF/x1QDW8z/sM0zSTT3iDP6AN5YBlHEAq94FQgWp8dTn6Dxn oD1nh2cgFJKn5GV5pBlCZjQjtoDIFzGWv5iSutFjf4tkYnJXt0BC9OKnK n6iTuE7KsFOgA3EDiXgGV5Bn2+txKPMWqzTfe+2MyXghLUHkF9ZD2UvA1 dvioxji6AxRt/e5JKGV34RkXetopfBNxELG0Lj+jE5MC5SxYOr2pF60Vo iMToNKVrRy5JgC3hTe99a0L0yrp8CazSQwwP3i3xJbB3UP8E52NNIwEfz dcs3cDL1LOSLa7HlTPLA8D1FHiynwm7Gm3Nw0YkVJWSrEQ0tbURcZao8R g==; IronPort-SDR: ANy9W339HNTNs96y0TtfbQ+ufpYfsmIM67vSFzhomS/A6ND8B4yA9fraVudtLS33ogNIbhyxOx HlJEgOmMI/vr1xWDr2pFabNydBGvG7G0xIL74dBA6RdAeSOaC1kPVifeOTODJFdSWvof5M5H7B 12nbMOmgj1rx3YZJjOXIxYXcwvqaj1+CT2udDXF+dsydO3KaeJfSg2vD5S0SU78YouQS+GaxxF Dfa9rpRSpyqPQM6AokoXd3/uHg3L9GJKSSB1NjUZmJ3WhgS/0QSUz2m86o1+vJV9NXcB7efiu+ Qnc= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="160251031" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:21:36 +0800 IronPort-SDR: qkWw6kjvelhE8xRvRA3qux7RZnO/dOdL9pysK1jfKOOMqzwWvGHwRdpr/8R6p6NV1ybUZIbZmw 5X58/KCUDhquTFV+Jzs+sjo9YHDS+b2CIJOtwZr0Lhv5R8Gxib5mN0whi/2QPkKN3Qh1lVKJ7V b/7kuxoYmyf+itGlOob+RnCwd1gYuYGPxd9EQSen5IXIPycV7HQFU71BZDnizCw/wX+jl69Tb3 glslUc0jEr9GZBVYde9BtQSjz4jspAugUPUbOsrDAInDlqvSlLnvJDadSjzSdyYVXRkJgRI78m SOUTs+MmBqXB7ewGPQOzJH8U Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:03:16 -0800 IronPort-SDR: 9gk3oNBwxrwC1aTxMkidnL19PYe8Nvyqxlvt0goVERQH+fKSSNLuryORUqZRw+FHz0OqqWHhFL Vtb2C5V9BgqVUdro8cDexcUhbJDB/w2GCeuTTmPGbAQ4x0Z4Q2lDHM+y9ZM3sS1BrbpGAw4+Qe f1HkIQ0348RxlfsjsoA2COSu3LWyqaYpjNOtxBsUD1Bw7BKAiKRzm5vDiGyD/yDLT7OB+iBcq6 XYtgRpnaRmgAmRj5oW6PruiPEoC5eY7OEijOHoKpa0ImMZrFzT3mudf+kg2goahUeWtPztLm63 LJ0= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:21:32 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 8/9] scsi: ufshpb: Add support for host control mode Date: Thu, 18 Feb 2021 15:19:31 +0200 Message-Id: <20210218131932.106997-9-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Support devices that report they are using host control mode. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 9a1c525204f7..2c0f63a828ca 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -2523,12 +2523,6 @@ void ufshpb_get_dev_info(struct ufs_hba *hba, u8 *desc_buf) u32 max_hpb_sigle_cmd = 0; hpb_dev_info->control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CONTROL]; - if (hpb_dev_info->control_mode == HPB_HOST_CONTROL) { - dev_err(hba->dev, "%s: host control mode is not supported.\n", - __func__); - hpb_dev_info->hpb_disabled = true; - return; - } version = get_unaligned_be16(desc_buf + DEVICE_DESC_PARAM_HPB_VER); if (version != HPB_SUPPORT_VERSION) { From patchwork Thu Feb 18 13:19:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 385161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72FD0C433E0 for ; Thu, 18 Feb 2021 16:57:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 11F3364EAF for ; Thu, 18 Feb 2021 16:57:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234079AbhBRQ5A (ORCPT ); Thu, 18 Feb 2021 11:57:00 -0500 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:27864 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233453AbhBRNYS (ORCPT ); Thu, 18 Feb 2021 08:24:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1613654657; x=1645190657; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Cg4R6lPvpmjc/V6U/nOVqcAzVVRyIiN6xKAxSzeRWrc=; b=RiMF3JvmnvwgG8fRZbXbd/ZwoNDF0C4bkFSEMfQV/8EL6EQwYOKIH+DU 0jtzcY99ilhdEbZrQZTF87Jgh+9Sd8/fapKSG+i04BEoihmwjfvRxVQhH vQf5ieQFFoqhPxQO0UpYfanUys5jp2nKoqLVQyt98uO7d2Tu2gmMDtX28 LoLarETWCxBdE8B/yvEfmeXlqa5fRy30EVnfJjXjkZ8Sc4m5Ye0jh66fH X/4HjlA+Sju4pKaXqpCl+w2bxNy212itwmiaa0OAli2ul6GHWTUFxFju7 ZAsAtrIJCSBo33ekeoO3wLKjcFS2++egHac5y13DpMvyJtuuWKhusPkLl Q==; IronPort-SDR: sIgLyUjDU6Ak+rGrrkz2+qcG4cMFtNkJiZAyzfjmH0R2MTjAHwr/Ivg2v1L85o8UIpZccy58mb tXtnLk4dQWew0qQrEapP/fGskGWmJ7RZx4Tk4RCL/XHWwr1ORgZTkSI5dcfht7VvVZe3ui5hRb XtsnnY43G/2M4CoKwCJkvX2VUOUyQpxWItHSmkEXR44DiekNm1bQdjNDXYwHVGhi/bZmS5780I Kw8Q3x48m/JXf8Szvc6wPvwx30BqHvIn+SQt/7V6WGeIOHnIFXqU3qXvcO/M4svZ9KAjUT6VGF /Mo= X-IronPort-AV: E=Sophos;i="5.81,187,1610380800"; d="scan'208";a="161433419" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 18 Feb 2021 21:21:48 +0800 IronPort-SDR: xchwWR4HfZvUCyOsewVN2huRALI6dEasuhumnuM5v1fI6z0a6UWawCGNg5YfH7zhn7PooK7m6F iTQDK7zaciGxnsdFB9tqlomOpOBRgOP9pKHbn1K40EUrn0e12e9pExiksGsgOsyIwkXRGtURvq j3p2+k01I0rZF56idnc+72WQ95U4Mp2qX6/gCQhBZhI+o8fOxbn6oWMJ87FMJs1d2hdWvFG8q3 rncO2HJGwPOkCkPWalqBBGJVWlyIOPHb1mArXoWjGi5noRoqON02LH1A/BRNAZ5h6xnRG8J40T jD3eK7OJ+cLh/4ptnVupXK++ Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 05:05:14 -0800 IronPort-SDR: PQ0Bx+BNJ3Rt9ocpY6SeS3zyOoqOqIkqbdvPuBqgfEY4hG/hgg+4anIu1cYfzWEniNydtBhdiH q/rLWRDe8PTYPUYXQVH/a0C0vqjg2JTsJzl6r91TQM/i+iilHJ7i+FqosBz1g+R3YSAauCA+bG w9Jr7HBxtTtUXGqnmi42NUGOwAoP+Yjd5dVGNXkqzdJJgfsR1hjyLLX1x5dYmbm9dZudAOOgMJ TO/mYlfd0Dv5gPrr8ChrcWrpymSiIVW8tW1i2I8PMqppjavgk2wl7FlyJkl15K7AE7k+fQJnhp HiE= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 18 Feb 2021 05:21:43 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v3 9/9] scsi: ufshpb: Make host mode parameters configurable Date: Thu, 18 Feb 2021 15:19:32 +0200 Message-Id: <20210218131932.106997-10-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210218131932.106997-1-avri.altman@wdc.com> References: <20210218131932.106997-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org We can make use of this commit, to elaborate some more of the host control mode logic, explaining what role play each and every variable. While at it, allow those parameters to be configurable. Signed-off-by: Avri Altman --- Documentation/ABI/testing/sysfs-driver-ufs | 68 ++++++ drivers/scsi/ufs/ufshpb.c | 254 +++++++++++++++++++-- drivers/scsi/ufs/ufshpb.h | 18 ++ 3 files changed, 326 insertions(+), 14 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-driver-ufs b/Documentation/ABI/testing/sysfs-driver-ufs index 0017eaf89cbe..1d72201c4953 100644 --- a/Documentation/ABI/testing/sysfs-driver-ufs +++ b/Documentation/ABI/testing/sysfs-driver-ufs @@ -1322,3 +1322,71 @@ Description: This entry shows the maximum HPB data size for using single HPB === ======== The file is read only. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/activation_thld +Date: February 2021 +Contact: Avri Altman +Description: In host control mode, reads are the major source of activation + trials. once this threshold hs met, the region is added to the + "to-be-activated" list. Since we reset the read counter upon + write, this include sending a rb command updating the region + ppn as well. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/normalization_factor +Date: February 2021 +Contact: Avri Altman +Description: In host control mode, We think of the regions as "buckets". + Those buckets are being filled with reads, and emptied on write. + We use entries_per_srgn - the amount of blocks in a subregion as + our bucket size. This applies because HPB1.0 only concern a + single-block reads. Once the bucket size is crossed, we trigger + a normalization work - not only to avoid overflow, but mainly + because we want to keep those counters normalized, as we are + using those reads as a comparative score, to make various decisions. + The normalization is dividing (shift right) the read counter by + the normalization_factor. If during consecutive normalizations + an active region has exhaust its reads - inactivate it. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_enter +Date: February 2021 +Contact: Avri Altman +Description: Region deactivation is often due to the fact that eviction took + place: a region become active on the expense of another. This is + happening when the max-active-regions limit has crossed. + In host mode, eviction is considered an extreme measure. We + want to verify that the entering region has enough reads, and + the exiting region has much less reads. eviction_thld_enter is + the min reads that a region must have in order to be considered + as a candidate to evict other region. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_exit +Date: February 2021 +Contact: Avri Altman +Description: same as above for the exiting region. A region is consider to + be a candidate to be evicted, only if it has less reads than + eviction_thld_exit. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_ms +Date: February 2021 +Contact: Avri Altman +Description: In order not to hang on to “cold” regions, we shall inactivate + a region that has no READ access for a predefined amount of + time - read_timeout_ms. If read_timeout_ms has expired, and the + region is dirty - it is less likely that we can make any use of + HPB-READing it. So we inactivate it. Still, deactivation has + its overhead, and we may still benefit from HPB-READing this + region if it is clean - see read_timeout_expiries. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_expiries +Date: February 2021 +Contact: Avri Altman +Description: if the region read timeout has expired, but the region is clean, + just re-wind its timer for another spin. Do that as long as it + is clean and did not exhaust its read_timeout_expiries threshold. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/timeout_polling_interval_ms +Date: February 2021 +Contact: Avri Altman +Description: the frequency in which the delayed worker that checks the + read_timeouts is awaken. +>>>>>>> ef6bf08e666d... scsi: ufshpb: Make host mode parameters configurable diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 2c0f63a828ca..44069974f627 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -17,7 +17,6 @@ #include "../sd.h" #define ACTIVATION_THRESHOLD 4 /* 4 IOs */ -#define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 6) /* 256 IOs */ #define READ_TO_MS 1000 #define READ_TO_EXPIRIES 100 #define POLLING_INTERVAL_MS 200 @@ -658,7 +657,7 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) */ spin_lock_irqsave(&rgn->rgn_lock, flags); rgn->reads++; - if (rgn->reads == ACTIVATION_THRESHOLD) + if (rgn->reads == hpb->params.activation_thld) activate = true; spin_unlock_irqrestore(&rgn->rgn_lock, flags); if (activate || @@ -1039,6 +1038,7 @@ static void ufshpb_read_to_handler(struct work_struct *work) struct victim_select_info *lru_info; struct ufshpb_region *rgn; unsigned long flags; + unsigned int poll; LIST_HEAD(expired_list); hpb = container_of(dwork, struct ufshpb_lu, ufshpb_read_to_work); @@ -1060,7 +1060,7 @@ static void ufshpb_read_to_handler(struct work_struct *work) list_add(&rgn->list_expired_rgn, &expired_list); else rgn->read_timeout = ktime_add_ms(ktime_get(), - READ_TO_MS); + hpb->params.read_timeout_ms); } } @@ -1076,8 +1076,9 @@ static void ufshpb_read_to_handler(struct work_struct *work) clear_bit(TIMEOUT_WORK_PENDING, &hpb->work_data_bits); + poll = hpb->params.timeout_polling_interval_ms; schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + msecs_to_jiffies(poll)); } static void ufshpb_add_lru_info(struct victim_select_info *lru_info, @@ -1087,8 +1088,11 @@ static void ufshpb_add_lru_info(struct victim_select_info *lru_info, list_add_tail(&rgn->list_lru_rgn, &lru_info->lh_lru_rgn); atomic_inc(&lru_info->active_cnt); if (rgn->hpb->is_hcm) { - rgn->read_timeout = ktime_add_ms(ktime_get(), READ_TO_MS); - rgn->read_timeout_expiries = READ_TO_EXPIRIES; + rgn->read_timeout = + ktime_add_ms(ktime_get(), + rgn->hpb->params.read_timeout_ms); + rgn->read_timeout_expiries = + rgn->hpb->params.read_timeout_expiries; } } @@ -1112,7 +1116,8 @@ static struct ufshpb_region *ufshpb_victim_lru_info(struct ufshpb_lu *hpb) * in host control mode, verify that the exiting region * has less reads */ - if (hpb->is_hcm && rgn->reads > (EVICTION_THRESHOLD >> 1)) + if (hpb->is_hcm && + rgn->reads > hpb->params.eviction_thld_exit) continue; victim_rgn = rgn; @@ -1326,7 +1331,8 @@ static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) * in host control mode, verify that the entering * region has enough reads */ - if (hpb->is_hcm && rgn->reads < EVICTION_THRESHOLD) { + if (hpb->is_hcm && + rgn->reads < hpb->params.eviction_thld_enter) { ret = -EACCES; goto out; } @@ -1646,8 +1652,10 @@ static void ufshpb_normalization_work_handler(struct work_struct *work) { struct ufshpb_lu *hpb; int rgn_idx; + u8 factor; hpb = container_of(work, struct ufshpb_lu, ufshpb_normalization_work); + factor = hpb->params.normalization_factor; for (rgn_idx = 0; rgn_idx < hpb->rgns_per_lu; rgn_idx++) { struct ufshpb_region *rgn = hpb->rgn_tbl + rgn_idx; @@ -1656,7 +1664,7 @@ static void ufshpb_normalization_work_handler(struct work_struct *work) unsigned long flags; spin_lock_irqsave(&rgn->rgn_lock, flags); - rgn->reads = (rgn->reads >> 1); + rgn->reads = (rgn->reads >> factor); spin_unlock_irqrestore(&rgn->rgn_lock, flags); } @@ -1979,8 +1987,217 @@ requeue_timeout_ms_store(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RW(requeue_timeout_ms); +ufshpb_sysfs_param_show_func(activation_thld); +static ssize_t +activation_thld_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.activation_thld = val; + + return count; +} +static DEVICE_ATTR_RW(activation_thld); + +ufshpb_sysfs_param_show_func(normalization_factor); +static ssize_t +normalization_factor_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0 || val > ilog2(hpb->entries_per_srgn)) + return -EINVAL; + + hpb->params.normalization_factor = val; + + return count; +} +static DEVICE_ATTR_RW(normalization_factor); + +ufshpb_sysfs_param_show_func(eviction_thld_enter); +static ssize_t +eviction_thld_enter_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= hpb->params.eviction_thld_exit) + return -EINVAL; + + hpb->params.eviction_thld_enter = val; + + return count; +} +static DEVICE_ATTR_RW(eviction_thld_enter); + +ufshpb_sysfs_param_show_func(eviction_thld_exit); +static ssize_t +eviction_thld_exit_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= hpb->params.activation_thld) + return -EINVAL; + + hpb->params.eviction_thld_exit = val; + + return count; +} +static DEVICE_ATTR_RW(eviction_thld_exit); + +ufshpb_sysfs_param_show_func(read_timeout_ms); +static ssize_t +read_timeout_ms_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.read_timeout_ms = val; + + return count; +} +static DEVICE_ATTR_RW(read_timeout_ms); + +ufshpb_sysfs_param_show_func(read_timeout_expiries); +static ssize_t +read_timeout_expiries_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.read_timeout_expiries = val; + + return count; +} +static DEVICE_ATTR_RW(read_timeout_expiries); + +ufshpb_sysfs_param_show_func(timeout_polling_interval_ms); +static ssize_t +timeout_polling_interval_ms_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.timeout_polling_interval_ms = val; + + return count; +} +static DEVICE_ATTR_RW(timeout_polling_interval_ms); + +static void ufshpb_hcm_param_init(struct ufshpb_lu *hpb) +{ + hpb->params.activation_thld = ACTIVATION_THRESHOLD; + hpb->params.normalization_factor = 1; + hpb->params.eviction_thld_enter = (ACTIVATION_THRESHOLD << 6); + hpb->params.eviction_thld_exit = (ACTIVATION_THRESHOLD << 5); + hpb->params.read_timeout_ms = READ_TO_MS; + hpb->params.read_timeout_expiries = READ_TO_EXPIRIES; + hpb->params.timeout_polling_interval_ms = POLLING_INTERVAL_MS; +} + static struct attribute *hpb_dev_param_attrs[] = { &dev_attr_requeue_timeout_ms.attr, + &dev_attr_activation_thld.attr, + &dev_attr_normalization_factor.attr, + &dev_attr_eviction_thld_enter.attr, + &dev_attr_eviction_thld_exit.attr, + &dev_attr_read_timeout_ms.attr, + &dev_attr_read_timeout_expiries.attr, + &dev_attr_timeout_polling_interval_ms.attr, + NULL, }; struct attribute_group ufs_sysfs_hpb_param_group = { @@ -2054,6 +2271,8 @@ static void ufshpb_stat_init(struct ufshpb_lu *hpb) static void ufshpb_param_init(struct ufshpb_lu *hpb) { hpb->params.requeue_timeout_ms = HPB_REQUEUE_TIME_MS; + if (hpb->is_hcm) + ufshpb_hcm_param_init(hpb); } static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) @@ -2110,9 +2329,13 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) ufshpb_stat_init(hpb); ufshpb_param_init(hpb); - if (hpb->is_hcm) + if (hpb->is_hcm) { + unsigned int poll; + + poll = hpb->params.timeout_polling_interval_ms; schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + msecs_to_jiffies(poll)); + } return 0; @@ -2292,10 +2515,13 @@ void ufshpb_resume(struct ufs_hba *hba) continue; ufshpb_set_state(hpb, HPB_PRESENT); ufshpb_kick_map_work(hpb); - if (hpb->is_hcm) - schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + if (hpb->is_hcm) { + unsigned int poll = + hpb->params.timeout_polling_interval_ms; + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(poll)); + } } } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 75093271d12b..bfb5eff9a66a 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -178,8 +178,26 @@ struct victim_select_info { atomic_t active_cnt; }; +/** + * ufshpb_params - ufs hpb parameters + * @requeue_timeout_ms - requeue threshold of wb command (0x2) + * @activation_thld - min reads [IOs] to activate/update a region + * @normalization_factor - shift right the region's reads + * @eviction_thld_enter - min reads [IOs] for the entering region in eviction + * @eviction_thld_exit - max reads [IOs] for the exiting region in eviction + * @read_timeout_ms - timeout [ms] from the last read IO to the region + * @read_timeout_expiries - amount of allowable timeout expireis + * @timeout_polling_interval_ms - frequency in which timeouts are checked + */ struct ufshpb_params { unsigned int requeue_timeout_ms; + unsigned int activation_thld; + unsigned int normalization_factor; + unsigned int eviction_thld_enter; + unsigned int eviction_thld_exit; + unsigned int read_timeout_ms; + unsigned int read_timeout_expiries; + unsigned int timeout_polling_interval_ms; }; struct ufshpb_stats {