From patchwork Wed Jul 26 11:25:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ranjan Kumar X-Patchwork-Id: 707462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B026AC001DE for ; Wed, 26 Jul 2023 11:27:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232729AbjGZL1k (ORCPT ); Wed, 26 Jul 2023 07:27:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233068AbjGZL1Q (ORCPT ); Wed, 26 Jul 2023 07:27:16 -0400 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 747C330C1 for ; Wed, 26 Jul 2023 04:26:46 -0700 (PDT) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-666e6ecb52dso3908658b3a.2 for ; Wed, 26 Jul 2023 04:26:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1690370800; x=1690975600; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=6DkMWM+1fuc1fb1f3dIb5ss0SAW3N8Lxx7A+nyUDR4I=; b=ca9/yzzwvswYkr/ZrKlJEE48ZspPSzGq5VklVqZaBEiN6OC3j5hzegv4jzjdcFyHD5 KxS6gZJpmrm+YuHLGS/v4t2Hic2Ant+58ZPNXQUmCAhUFzpxJC+bdUTjpHTzjMTti6FH YE6xp/DC8m1kLYGWEKf/LfBJlG2w3398keoOk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690370800; x=1690975600; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6DkMWM+1fuc1fb1f3dIb5ss0SAW3N8Lxx7A+nyUDR4I=; b=HAEWPrtNtrf6xOtBvRqbmDwRno4/41gdg97ZkN8F/hGYlXYVlA7hQ3se0e4ytNAE5S Ci/8yBFhyeLBWf3Qvs76kPolzUD4J/jFEzfQAvxNEvLOCylfhH7M2fm5IAYjd43LAuU8 vnQBMt+Wd9oeVCi4RFdyGlefLEp4ieNri+qdt0Tcjxcz/bWVa0FQ6dW/EIqp0YFHoHGJ bDmDfQwesfaqBVcUjwL+iuynLtKsd6/DvCyqJMQXxK6HWZ+3DFfGLFawnYolvS2tuAE4 OJ/J4uvQpH+uSGOHSr8mWyF2JzgqI6fyxchNGdfGvOS5BhggrjKy/JmFqSJdH+w3ls1O DdgQ== X-Gm-Message-State: ABy/qLYB+6qXMyjcHaaefiCRgj9Lr5wfnVKgCCg+hNaA1gERhAxEyv5u H+HMzNRW4PA1Z9p7+05e7Amn55Upm6AWk1+yk+iT30unJ3n160VPeGg1LhIFr9aOFTUBr55MBez yI3f5Gs+1v0pD1R25HZkq/qmL6PNB1wfZ/Tqjc7J9MJsoyKGqWlRwHIS1Hjleh0GRuFdt+33Xym f0robCI58LPA== X-Google-Smtp-Source: APBJJlECkI8eeIqYjbkk+v3hos8hfE1U5KBe9i9cIH6z/keVUnlokY6nByHRpbm5ayIsAgj9byIJ/A== X-Received: by 2002:a05:6a21:6d99:b0:131:e0a:3cf4 with SMTP id wl25-20020a056a216d9900b001310e0a3cf4mr1523021pzb.57.1690370799912; Wed, 26 Jul 2023 04:26:39 -0700 (PDT) Received: from localhost.localdomain ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id iz11-20020a170902ef8b00b001bbbc655ca1sm3443231plb.219.2023.07.26.04.26.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 04:26:39 -0700 (PDT) From: Ranjan Kumar To: linux-scsi@vger.kernel.org, martin.petersen@oracle.com Cc: sathya.prakash@broadcom.com, sreekanth.reddy@broadcom.com, Ranjan Kumar , stable@vger.kernel.org Subject: [PATCH v3 1/2] Perform additional retries if Doorbell read returns 0 Date: Wed, 26 Jul 2023 16:55:26 +0530 Message-Id: <20230726112527.14987-2-ranjan.kumar@broadcom.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230726112527.14987-1-ranjan.kumar@broadcom.com> References: <20230726112527.14987-1-ranjan.kumar@broadcom.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The driver retries certain register reads for 3 times if the returned value is 0, this was done based on hardware specification where the controller could possibly return 0 for certain registers when there is a parallel access to some of other registers from the firmware due to BMC out of band interactions.Recently it is observed that in certain systems with increased BMC interactions, the values returned are 0 even for 3 retries (the proper value is returned between 4 to 6 retries).Hence this patch changes the retry count 3 to 30, which is a revised recommendation, to avoid the out of band conflict. Fixes: b899202901a8 ("mpt3sas: Add separate function for aero doorbell reads") Cc: stable@vger.kernel.org Signed-off-by: Ranjan Kumar --- drivers/scsi/mpt3sas/mpt3sas_base.c | 53 ++++++++++++++++------------- drivers/scsi/mpt3sas/mpt3sas_base.h | 11 +++++- 2 files changed, 39 insertions(+), 25 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 53f5492579cb..efc9bc48db6f 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -196,25 +196,27 @@ module_param_call(mpt3sas_fwfault_debug, _scsih_set_fwfault_debug, /** * _base_readl_aero - retry readl for max three times. * @addr: MPT Fusion system interface register address - * - * Retry the readl() for max three times if it gets zero value + *@retry_count: max no of retry + + * Retry the readl() for max thirty times if it gets zero value * while reading the system interface register. */ + static inline u32 -_base_readl_aero(const volatile void __iomem *addr) +_base_readl_aero(const volatile void __iomem *addr, u8 retry_count) { u32 i = 0, ret_val; do { ret_val = readl(addr); i++; - } while (ret_val == 0 && i < 3); + } while (ret_val == 0 && i < retry_count); return ret_val; } static inline u32 -_base_readl(const volatile void __iomem *addr) +_base_readl(const volatile void __iomem *addr, u8 retry_count) { return readl(addr); } @@ -940,7 +942,7 @@ mpt3sas_halt_firmware(struct MPT3SAS_ADAPTER *ioc) dump_stack(); - doorbell = ioc->base_readl(&ioc->chip->Doorbell); + doorbell = ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT); if ((doorbell & MPI2_IOC_STATE_MASK) == MPI2_IOC_STATE_FAULT) { mpt3sas_print_fault_code(ioc, doorbell & MPI2_DOORBELL_DATA_MASK); @@ -1617,10 +1619,10 @@ mpt3sas_base_mask_interrupts(struct MPT3SAS_ADAPTER *ioc) u32 him_register; ioc->mask_interrupts = 1; - him_register = ioc->base_readl(&ioc->chip->HostInterruptMask); + him_register = ioc->base_readl(&ioc->chip->HostInterruptMask, READL_RETRY_SHORT_COUNT); him_register |= MPI2_HIM_DIM + MPI2_HIM_RIM + MPI2_HIM_RESET_IRQ_MASK; writel(him_register, &ioc->chip->HostInterruptMask); - ioc->base_readl(&ioc->chip->HostInterruptMask); + ioc->base_readl(&ioc->chip->HostInterruptMask, READL_RETRY_SHORT_COUNT); } /** @@ -1634,7 +1636,7 @@ mpt3sas_base_unmask_interrupts(struct MPT3SAS_ADAPTER *ioc) { u32 him_register; - him_register = ioc->base_readl(&ioc->chip->HostInterruptMask); + him_register = ioc->base_readl(&ioc->chip->HostInterruptMask, READL_RETRY_SHORT_COUNT); him_register &= ~MPI2_HIM_RIM; writel(him_register, &ioc->chip->HostInterruptMask); ioc->mask_interrupts = 0; @@ -6686,7 +6688,7 @@ mpt3sas_base_get_iocstate(struct MPT3SAS_ADAPTER *ioc, int cooked) { u32 s, sc; - s = ioc->base_readl(&ioc->chip->Doorbell); + s = ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT); sc = s & MPI2_IOC_STATE_MASK; return cooked ? sc : s; } @@ -6760,7 +6762,8 @@ _base_wait_for_doorbell_int(struct MPT3SAS_ADAPTER *ioc, int timeout) count = 0; cntdn = 1000 * timeout; do { - int_status = ioc->base_readl(&ioc->chip->HostInterruptStatus); + int_status = ioc->base_readl(&ioc->chip->HostInterruptStatus, + READL_RETRY_SHORT_COUNT); if (int_status & MPI2_HIS_IOC2SYS_DB_STATUS) { dhsprintk(ioc, ioc_info(ioc, "%s: successful count(%d), timeout(%d)\n", @@ -6786,7 +6789,8 @@ _base_spin_on_doorbell_int(struct MPT3SAS_ADAPTER *ioc, int timeout) count = 0; cntdn = 2000 * timeout; do { - int_status = ioc->base_readl(&ioc->chip->HostInterruptStatus); + int_status = ioc->base_readl(&ioc->chip->HostInterruptStatus, + READL_RETRY_SHORT_COUNT); if (int_status & MPI2_HIS_IOC2SYS_DB_STATUS) { dhsprintk(ioc, ioc_info(ioc, "%s: successful count(%d), timeout(%d)\n", @@ -6824,14 +6828,15 @@ _base_wait_for_doorbell_ack(struct MPT3SAS_ADAPTER *ioc, int timeout) count = 0; cntdn = 1000 * timeout; do { - int_status = ioc->base_readl(&ioc->chip->HostInterruptStatus); + int_status = ioc->base_readl(&ioc->chip->HostInterruptStatus, + READL_RETRY_SHORT_COUNT); if (!(int_status & MPI2_HIS_SYS2IOC_DB_STATUS)) { dhsprintk(ioc, ioc_info(ioc, "%s: successful count(%d), timeout(%d)\n", __func__, count, timeout)); return 0; } else if (int_status & MPI2_HIS_IOC2SYS_DB_STATUS) { - doorbell = ioc->base_readl(&ioc->chip->Doorbell); + doorbell = ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT); if ((doorbell & MPI2_IOC_STATE_MASK) == MPI2_IOC_STATE_FAULT) { mpt3sas_print_fault_code(ioc, doorbell); @@ -6871,7 +6876,7 @@ _base_wait_for_doorbell_not_used(struct MPT3SAS_ADAPTER *ioc, int timeout) count = 0; cntdn = 1000 * timeout; do { - doorbell_reg = ioc->base_readl(&ioc->chip->Doorbell); + doorbell_reg = ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT); if (!(doorbell_reg & MPI2_DOORBELL_USED)) { dhsprintk(ioc, ioc_info(ioc, "%s: successful count(%d), timeout(%d)\n", @@ -7019,13 +7024,13 @@ _base_handshake_req_reply_wait(struct MPT3SAS_ADAPTER *ioc, int request_bytes, __le32 *mfp; /* make sure doorbell is not in use */ - if ((ioc->base_readl(&ioc->chip->Doorbell) & MPI2_DOORBELL_USED)) { + if ((ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT) & MPI2_DOORBELL_USED)) { ioc_err(ioc, "doorbell is in use (line=%d)\n", __LINE__); return -EFAULT; } /* clear pending doorbell interrupts from previous state changes */ - if (ioc->base_readl(&ioc->chip->HostInterruptStatus) & + if (ioc->base_readl(&ioc->chip->HostInterruptStatus, READL_RETRY_SHORT_COUNT) & MPI2_HIS_IOC2SYS_DB_STATUS) writel(0, &ioc->chip->HostInterruptStatus); @@ -7068,7 +7073,7 @@ _base_handshake_req_reply_wait(struct MPT3SAS_ADAPTER *ioc, int request_bytes, } /* read the first two 16-bits, it gives the total length of the reply */ - reply[0] = le16_to_cpu(ioc->base_readl(&ioc->chip->Doorbell) + reply[0] = le16_to_cpu(ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT) & MPI2_DOORBELL_DATA_MASK); writel(0, &ioc->chip->HostInterruptStatus); if ((_base_wait_for_doorbell_int(ioc, 5))) { @@ -7076,7 +7081,7 @@ _base_handshake_req_reply_wait(struct MPT3SAS_ADAPTER *ioc, int request_bytes, __LINE__); return -EFAULT; } - reply[1] = le16_to_cpu(ioc->base_readl(&ioc->chip->Doorbell) + reply[1] = le16_to_cpu(ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT) & MPI2_DOORBELL_DATA_MASK); writel(0, &ioc->chip->HostInterruptStatus); @@ -7087,10 +7092,10 @@ _base_handshake_req_reply_wait(struct MPT3SAS_ADAPTER *ioc, int request_bytes, return -EFAULT; } if (i >= reply_bytes/2) /* overflow case */ - ioc->base_readl(&ioc->chip->Doorbell); + ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT); else reply[i] = le16_to_cpu( - ioc->base_readl(&ioc->chip->Doorbell) + ioc->base_readl(&ioc->chip->Doorbell, READL_RETRY_COUNT) & MPI2_DOORBELL_DATA_MASK); writel(0, &ioc->chip->HostInterruptStatus); } @@ -7949,14 +7954,14 @@ _base_diag_reset(struct MPT3SAS_ADAPTER *ioc) goto out; } - host_diagnostic = ioc->base_readl(&ioc->chip->HostDiagnostic); + host_diagnostic = ioc->base_readl(&ioc->chip->HostDiagnostic, READL_RETRY_COUNT); drsprintk(ioc, ioc_info(ioc, "wrote magic sequence: count(%d), host_diagnostic(0x%08x)\n", count, host_diagnostic)); } while ((host_diagnostic & MPI2_DIAG_DIAG_WRITE_ENABLE) == 0); - hcb_size = ioc->base_readl(&ioc->chip->HCBSize); + hcb_size = ioc->base_readl(&ioc->chip->HCBSize, READL_RETRY_SHORT_COUNT); drsprintk(ioc, ioc_info(ioc, "diag reset: issued\n")); writel(host_diagnostic | MPI2_DIAG_RESET_ADAPTER, @@ -7969,7 +7974,7 @@ _base_diag_reset(struct MPT3SAS_ADAPTER *ioc) for (count = 0; count < (300000000 / MPI2_HARD_RESET_PCIE_SECOND_READ_DELAY_MICRO_SEC); count++) { - host_diagnostic = ioc->base_readl(&ioc->chip->HostDiagnostic); + host_diagnostic = ioc->base_readl(&ioc->chip->HostDiagnostic, READL_RETRY_COUNT); if (host_diagnostic == 0xFFFFFFFF) { ioc_info(ioc, diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index 05364aa15ecd..b4e57b89915d 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -160,6 +160,15 @@ #define IOC_OPERATIONAL_WAIT_COUNT 10 +/* + *Due to hardware specific behaviour + *retry count is increased from 3 to 30. + *Delay is not needed as retry count fulfill + *the desired requirement + */ +#define READL_RETRY_COUNT 30 +#define READL_RETRY_SHORT_COUNT 3 + /* * NVMe defines */ @@ -994,7 +1003,7 @@ typedef void (*NVME_BUILD_PRP)(struct MPT3SAS_ADAPTER *ioc, u16 smid, typedef void (*PUT_SMID_IO_FP_HIP) (struct MPT3SAS_ADAPTER *ioc, u16 smid, u16 funcdep); typedef void (*PUT_SMID_DEFAULT) (struct MPT3SAS_ADAPTER *ioc, u16 smid); -typedef u32 (*BASE_READ_REG) (const volatile void __iomem *addr); +typedef u32 (*BASE_READ_REG) (const volatile void __iomem *addr, u8 retry_count); /* * To get high iops reply queue's msix index when high iops mode is enabled * else get the msix index of general reply queues.