From patchwork Thu May 13 08:36:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kashyap Desai X-Patchwork-Id: 437609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MIME_HEADER_CTYPE_ONLY, SPF_HELO_NONE, SPF_PASS, T_TVD_MIME_NO_HEADERS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54857C43460 for ; Thu, 13 May 2021 08:34:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 242E261287 for ; Thu, 13 May 2021 08:34:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231987AbhEMIf0 (ORCPT ); Thu, 13 May 2021 04:35:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232002AbhEMIe7 (ORCPT ); Thu, 13 May 2021 04:34:59 -0400 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C674C06138A for ; Thu, 13 May 2021 01:33:33 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id b15so7807684plh.10 for ; Thu, 13 May 2021 01:33:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MF1NlMWo1pP1S52KvI4FqjjHxjrCJRAit506n3ZPWFg=; b=OGK2ytr1PBYZQRLhLXjWRrcE6tM0oeiBNhOGHOD4BmMHjhVvEj/tE2VpTc3F18GnnF 5Y/AqUNGPkSL95wJty5Y1lebHLHkKDcglxw6v4995BO/T2WF6SRiqarB+hBA2OIru7EW HXuxzff7KW9FkeiW4SAOGIxUBCGLkXfjVt4GA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MF1NlMWo1pP1S52KvI4FqjjHxjrCJRAit506n3ZPWFg=; b=qbqKzzZQE5w/lCw53FIYkJfmruEGTu1EjXEclFNgJ/02D3L6iZ3ddHt1UWQ95YnLWV RB7TB1A2tDkzwDnmyeRorklcUmKX0l2WmuBdtvepqhlebQYPVQ7hR1EmkTTrU+zTKgwF 5PX+AV8Q4pHql/1XkN1rXuAjfNlTNtT3sP3tRD4eUGB769lAWwpI5DW2/HXb57I4yPuZ SUOaouGn55RobRJIWMnenvq3/lnwH/waodK5qr9KrcL/cOxc5LuFyqR652SRL6RrNRZN JOdku+NNqtv7bMIkXix31rcvEfzFOJ5VifKUOhawkvOuGj/tHN330n121Cpzmd3qFi3z SCFw== X-Gm-Message-State: AOAM531RYml70TeFsyiZUyQpE5QX9eFXkAWytzYYQ0pwBKM/YbWss6zl DY9960x9xseLnuSvdxsBZZuvcmoWAJCwxcZHQd/EZYxAumSd888HDEUJrn+ECITHFLiugxQ8dL7 zeHBG7QwuWw3P7gvF5DYPBeddUivnv+BGDPLwaY/IOsModATb8YOILghVliEQOuUN2dUMWIRU4a KD1wxO3Q== X-Google-Smtp-Source: ABdhPJyxBGjwFMCFiwjSFEGtaNWDepJhLW61BDKmKzJOQ2kODb55B5oZr6Q0IdHkJTnCTgxW6Rwh+w== X-Received: by 2002:a17:902:e745:b029:ee:9dd5:f289 with SMTP id p5-20020a170902e745b02900ee9dd5f289mr39623001plf.64.1620894812334; Thu, 13 May 2021 01:33:32 -0700 (PDT) Received: from drv-bst-rhel8.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id i123sm1632468pfc.53.2021.05.13.01.33.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 May 2021 01:33:31 -0700 (PDT) From: Kashyap Desai To: linux-scsi@vger.kernel.org Cc: jejb@linux.ibm.com, martin.petersen@oracle.com, steve.hagan@broadcom.com, peter.rivera@broadcom.com, mpi3mr-linuxdrv.pdl@broadcom.com, Kashyap Desai , sathya.prakash@broadcom.com Subject: [PATCH v5 17/24] mpi3mr: add support of threaded isr Date: Thu, 13 May 2021 14:06:01 +0530 Message-Id: <20210513083608.2243297-18-kashyap.desai@broadcom.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20210513083608.2243297-1-kashyap.desai@broadcom.com> References: <20210513083608.2243297-1-kashyap.desai@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Register driver for threaded interrupt. By default, driver will attempt io completion from interrupt context (primary handler). Since driver tracks per reply queue outstanding ios, it will schedule threaded ISR if there are any outstanding IOs expected on that particular reply queue. Threaded ISR (secondary handler) will loop for IO completion as long as there are outstanding IOs (speculative method using same per reply queue outstanding counter) or it has completed some X amount of commands (something like budget). Signed-off-by: Kashyap Desai Reviewed-by: Hannes Reinecke Reviewed-by: Tomas Henzl Reviewed-by: Himanshu Madhani Cc: sathya.prakash@broadcom.com --- drivers/scsi/mpi3mr/mpi3mr.h | 12 +++++ drivers/scsi/mpi3mr/mpi3mr_fw.c | 79 +++++++++++++++++++++++++++++++-- 2 files changed, 88 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/mpi3mr/mpi3mr.h b/drivers/scsi/mpi3mr/mpi3mr.h index 960943452a7a..9f16e80af06f 100644 --- a/drivers/scsi/mpi3mr/mpi3mr.h +++ b/drivers/scsi/mpi3mr/mpi3mr.h @@ -144,6 +144,10 @@ extern struct list_head mrioc_list; /* Default target device queue depth */ #define MPI3MR_DEFAULT_SDEV_QD 32 +/* Definitions for Threaded IRQ poll*/ +#define MPI3MR_IRQ_POLL_SLEEP 2 +#define MPI3MR_IRQ_POLL_TRIGGER_IOCOUNT 8 + /* SGE Flag definition */ #define MPI3MR_SGEFLAGS_SYSTEM_SIMPLE_END_OF_LIST \ (MPI3_SGE_FLAGS_ELEMENT_TYPE_SIMPLE | MPI3_SGE_FLAGS_DLAS_SYSTEM | \ @@ -295,6 +299,9 @@ struct op_req_qinfo { * @q_segment_list: Segment list base virtual address * @q_segment_list_dma: Segment list base DMA address * @ephase: Expected phased identifier for the reply queue + * @pend_ios: Number of IOs pending in HW for this queue + * @enable_irq_poll: Flag to indicate polling is enabled + * @in_use: Queue is handled by poll/ISR */ struct op_reply_qinfo { u16 ci; @@ -306,6 +313,9 @@ struct op_reply_qinfo { void *q_segment_list; dma_addr_t q_segment_list_dma; u8 ephase; + atomic_t pend_ios; + bool enable_irq_poll; + atomic_t in_use; }; /** @@ -557,6 +567,7 @@ struct scmd_priv { * @shost: Scsi_Host pointer * @id: Controller ID * @cpu_count: Number of online CPUs + * @irqpoll_sleep: usleep unit used in threaded isr irqpoll * @name: Controller ASCII name * @driver_name: Driver ASCII name * @sysif_regs: System interface registers virtual address @@ -658,6 +669,7 @@ struct mpi3mr_ioc { u8 id; int cpu_count; bool enable_segqueue; + u32 irqpoll_sleep; char name[MPI3MR_NAME_LENGTH]; char driver_name[MPI3MR_NAME_LENGTH]; diff --git a/drivers/scsi/mpi3mr/mpi3mr_fw.c b/drivers/scsi/mpi3mr/mpi3mr_fw.c index b733a008f81e..176e8a2c658d 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_fw.c +++ b/drivers/scsi/mpi3mr/mpi3mr_fw.c @@ -345,12 +345,16 @@ static int mpi3mr_process_op_reply_q(struct mpi3mr_ioc *mrioc, reply_qidx = op_reply_q->qid - 1; + if (!atomic_add_unless(&op_reply_q->in_use, 1, 1)) + return 0; + exp_phase = op_reply_q->ephase; reply_ci = op_reply_q->ci; reply_desc = mpi3mr_get_reply_desc(op_reply_q, reply_ci); if ((le16_to_cpu(reply_desc->reply_flags) & MPI3_REPLY_DESCRIPT_FLAGS_PHASE_MASK) != exp_phase) { + atomic_dec(&op_reply_q->in_use); return 0; } @@ -361,6 +365,7 @@ static int mpi3mr_process_op_reply_q(struct mpi3mr_ioc *mrioc, WRITE_ONCE(op_req_q->ci, le16_to_cpu(reply_desc->request_queue_ci)); mpi3mr_process_op_reply_desc(mrioc, reply_desc, &reply_dma, reply_qidx); + atomic_dec(&op_reply_q->pend_ios); if (reply_dma) mpi3mr_repost_reply_buf(mrioc, reply_dma); num_op_reply++; @@ -375,6 +380,14 @@ static int mpi3mr_process_op_reply_q(struct mpi3mr_ioc *mrioc, if ((le16_to_cpu(reply_desc->reply_flags) & MPI3_REPLY_DESCRIPT_FLAGS_PHASE_MASK) != exp_phase) break; + /* + * Exit completion loop to avoid CPU lockup + * Ensure remaining completion happens from threaded ISR. + */ + if (num_op_reply > mrioc->max_host_ios) { + intr_info->op_reply_q->enable_irq_poll = true; + break; + } } while (1); @@ -383,6 +396,7 @@ static int mpi3mr_process_op_reply_q(struct mpi3mr_ioc *mrioc, op_reply_q->ci = reply_ci; op_reply_q->ephase = exp_phase; + atomic_dec(&op_reply_q->in_use); return num_op_reply; } @@ -391,7 +405,7 @@ static irqreturn_t mpi3mr_isr_primary(int irq, void *privdata) struct mpi3mr_intr_info *intr_info = privdata; struct mpi3mr_ioc *mrioc; u16 midx; - u32 num_admin_replies = 0; + u32 num_admin_replies = 0, num_op_reply = 0; if (!intr_info) return IRQ_NONE; @@ -405,8 +419,10 @@ static irqreturn_t mpi3mr_isr_primary(int irq, void *privdata) if (!midx) num_admin_replies = mpi3mr_process_admin_reply_q(mrioc); + if (intr_info->op_reply_q) + num_op_reply = mpi3mr_process_op_reply_q(mrioc, intr_info); - if (num_admin_replies) + if (num_admin_replies || num_op_reply) return IRQ_HANDLED; else return IRQ_NONE; @@ -415,15 +431,32 @@ static irqreturn_t mpi3mr_isr_primary(int irq, void *privdata) static irqreturn_t mpi3mr_isr(int irq, void *privdata) { struct mpi3mr_intr_info *intr_info = privdata; + struct mpi3mr_ioc *mrioc; + u16 midx; int ret; if (!intr_info) return IRQ_NONE; + mrioc = intr_info->mrioc; + midx = intr_info->msix_index; /* Call primary ISR routine */ ret = mpi3mr_isr_primary(irq, privdata); - return ret; + /* + * If more IOs are expected, schedule IRQ polling thread. + * Otherwise exit from ISR. + */ + if (!intr_info->op_reply_q) + return ret; + + if (!intr_info->op_reply_q->enable_irq_poll || + !atomic_read(&intr_info->op_reply_q->pend_ios)) + return ret; + + disable_irq_nosync(pci_irq_vector(mrioc->pdev, midx)); + + return IRQ_WAKE_THREAD; } /** @@ -438,6 +471,36 @@ static irqreturn_t mpi3mr_isr(int irq, void *privdata) */ static irqreturn_t mpi3mr_isr_poll(int irq, void *privdata) { + struct mpi3mr_intr_info *intr_info = privdata; + struct mpi3mr_ioc *mrioc; + u16 midx; + u32 num_op_reply = 0; + + if (!intr_info || !intr_info->op_reply_q) + return IRQ_NONE; + + mrioc = intr_info->mrioc; + midx = intr_info->msix_index; + + /* Poll for pending IOs completions */ + do { + if (!mrioc->intr_enabled) + break; + + if (!midx) + mpi3mr_process_admin_reply_q(mrioc); + if (intr_info->op_reply_q) + num_op_reply += + mpi3mr_process_op_reply_q(mrioc, intr_info); + + usleep_range(mrioc->irqpoll_sleep, 10 * mrioc->irqpoll_sleep); + + } while (atomic_read(&intr_info->op_reply_q->pend_ios) && + (num_op_reply < mrioc->max_host_ios)); + + intr_info->op_reply_q->enable_irq_poll = false; + enable_irq(pci_irq_vector(mrioc->pdev, midx)); + return IRQ_HANDLED; } @@ -1147,6 +1210,9 @@ static int mpi3mr_create_op_reply_q(struct mpi3mr_ioc *mrioc, u16 qidx) op_reply_q->num_replies = MPI3MR_OP_REP_Q_QD; op_reply_q->ci = 0; op_reply_q->ephase = 1; + atomic_set(&op_reply_q->pend_ios, 0); + atomic_set(&op_reply_q->in_use, 0); + op_reply_q->enable_irq_poll = false; if (!op_reply_q->q_segments) { retval = mpi3mr_alloc_op_reply_q_segments(mrioc, qidx); @@ -1465,6 +1531,10 @@ int mpi3mr_op_request_post(struct mpi3mr_ioc *mrioc, pi = 0; op_req_q->pi = pi; + if (atomic_inc_return(&mrioc->op_reply_qinfo[reply_qidx].pend_ios) + > MPI3MR_IRQ_POLL_TRIGGER_IOCOUNT) + mrioc->op_reply_qinfo[reply_qidx].enable_irq_poll = true; + writel(op_req_q->pi, &mrioc->sysif_regs->oper_queue_indexes[reply_qidx].producer_index); @@ -2795,6 +2865,7 @@ int mpi3mr_init_ioc(struct mpi3mr_ioc *mrioc, u8 re_init) u32 ioc_status, ioc_config, i; struct _mpi3_ioc_facts_data facts_data; + mrioc->irqpoll_sleep = MPI3MR_IRQ_POLL_SLEEP; mrioc->change_count = 0; if (!re_init) { mrioc->cpu_count = num_online_cpus(); @@ -3081,6 +3152,8 @@ static void mpi3mr_memset_buffers(struct mpi3mr_ioc *mrioc) mrioc->op_reply_qinfo[i].ci = 0; mrioc->op_reply_qinfo[i].num_replies = 0; mrioc->op_reply_qinfo[i].ephase = 0; + atomic_set(&mrioc->op_reply_qinfo[i].pend_ios, 0); + atomic_set(&mrioc->op_reply_qinfo[i].in_use, 0); mpi3mr_memset_op_reply_q_buffers(mrioc, i); mrioc->req_qinfo[i].ci = 0;