From patchwork Wed Jul 26 09:40:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 706883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E1ADC0015E for ; Wed, 26 Jul 2023 09:41:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231347AbjGZJlc (ORCPT ); Wed, 26 Jul 2023 05:41:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233496AbjGZJl2 (ORCPT ); Wed, 26 Jul 2023 05:41:28 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6601DB for ; Wed, 26 Jul 2023 02:40:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364446; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QvKrmms9fUHhxbhcGxDz7QDKl8B/hBW7QeIiIj8phvY=; b=COaS5kKNUHqm/rRYnWqwd9ur4+S/hxxWmcO01AbI9k+7CBWKTuRQicGOYmGtXNjliizSuy v9RHR3y80Tv2MO99AhFQfrSJtnaNoxNc2sBqtZtXluGkpL1olrlluXKxQYpzvorIrAHzrd kFpjBuwYx4z5b8xKUHPdO3gE4HxHIfI= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-113-vtSBVKQAMs6GQr_RlH85qQ-1; Wed, 26 Jul 2023 05:40:40 -0400 X-MC-Unique: vtSBVKQAMs6GQr_RlH85qQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 773753C01BA1; Wed, 26 Jul 2023 09:40:39 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB50740C2063; Wed, 26 Jul 2023 09:40:38 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei Subject: [PATCH V2 1/9] blk-mq: add blk_mq_max_nr_hw_queues() Date: Wed, 26 Jul 2023 17:40:19 +0800 Message-Id: <20230726094027.535126-2-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org blk_mq_alloc_tag_set() may override set->nr_hw_queues as 1 in case of kdump kernel. This way causes trouble for driver, because blk-mq and driver see different queue mapping. Especially the only online CPU may not be 1 for kdump kernel, in which 'maxcpus=1' is passed from kernel command line, then driver may map hctx0 into one inactive real hw queue which cpu affinity is 0(offline). The issue exists on all drivers which use managed irq and support multiple hw queue. Prepare for fixing this kind of issue by applying the added helper, so driver can take blk-mq max nr_hw_queues knowledge into account when calculating io queues. Signed-off-by: Ming Lei --- block/blk-mq.c | 16 ++++++++++++++++ include/linux/blk-mq.h | 1 + 2 files changed, 17 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index b04ff6f56926..617d6f849a7b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -140,6 +140,22 @@ void blk_mq_freeze_queue_wait(struct request_queue *q) } EXPORT_SYMBOL_GPL(blk_mq_freeze_queue_wait); +/* + * Return the max supported nr_hw_queues for each hw queue type + * + * blk_mq_alloc_tag_set() may change nr_hw_queues for kdump kernel, so + * driver has to take blk-mq max supported nr_hw_queues into account + * when figuring out nr_hw_queues from hardware info, for avoiding + * inconsistency between driver and blk-mq. + */ +unsigned int blk_mq_max_nr_hw_queues(void) +{ + if (is_kdump_kernel()) + return 1; + return nr_cpu_ids; +} +EXPORT_SYMBOL_GPL(blk_mq_max_nr_hw_queues); + int blk_mq_freeze_queue_wait_timeout(struct request_queue *q, unsigned long timeout) { diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 495ca198775f..4c0cfd1f9e52 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -711,6 +711,7 @@ int blk_mq_alloc_sq_tag_set(struct blk_mq_tag_set *set, const struct blk_mq_ops *ops, unsigned int queue_depth, unsigned int set_flags); void blk_mq_free_tag_set(struct blk_mq_tag_set *set); +unsigned int blk_mq_max_nr_hw_queues(void); void blk_mq_free_request(struct request *rq); int blk_rq_poll(struct request *rq, struct io_comp_batch *iob, From patchwork Wed Jul 26 09:40:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 707468 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1729BC41513 for ; Wed, 26 Jul 2023 09:41:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229502AbjGZJla (ORCPT ); Wed, 26 Jul 2023 05:41:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233557AbjGZJl2 (ORCPT ); Wed, 26 Jul 2023 05:41:28 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C531ADD for ; Wed, 26 Jul 2023 02:40:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k/XiItvuETzuFaV0q2/0Ml9SkHL0l2Lbxykjfon98Ww=; b=I+71LlkX1PeTN5NCxjhMTZ3OEBN5VKxOngUb8bTghTMqVC5PymCMPNpoj3EEx+Ei4MufYr 6rroml47er/zPtHQDvoMJCtThVoeWheQC+wBNw6XGxAoS+n1jJlU26ntLKBrHwadT31Fpt ktzOBDLbB0Ix2U31DSoyT2TFAkwH0LY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-351-Ad3sbx7uMbeI3W7dnB5SwQ-1; Wed, 26 Jul 2023 05:40:43 -0400 X-MC-Unique: Ad3sbx7uMbeI3W7dnB5SwQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3B3F4803FDF; Wed, 26 Jul 2023 09:40:43 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C6F3F7835; Wed, 26 Jul 2023 09:40:42 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei Subject: [PATCH V2 2/9] nvme-pci: use blk_mq_max_nr_hw_queues() to calculate io queues Date: Wed, 26 Jul 2023 17:40:20 +0800 Message-Id: <20230726094027.535126-3-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Reported-by: Wen Xiong Signed-off-by: Ming Lei --- drivers/nvme/host/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index baf69af7ea78..a1227ae7eb39 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2251,7 +2251,7 @@ static unsigned int nvme_max_io_queues(struct nvme_dev *dev) */ if (dev->ctrl.quirks & NVME_QUIRK_SHARED_TAGS) return 1; - return num_possible_cpus() + dev->nr_write_queues + dev->nr_poll_queues; + return blk_mq_max_nr_hw_queues() + dev->nr_write_queues + dev->nr_poll_queues; } static int nvme_setup_io_queues(struct nvme_dev *dev) From patchwork Wed Jul 26 09:40:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 706882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A2AAC0015E for ; Wed, 26 Jul 2023 09:41:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233376AbjGZJlx (ORCPT ); Wed, 26 Jul 2023 05:41:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233057AbjGZJlm (ORCPT ); Wed, 26 Jul 2023 05:41:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA75097 for ; Wed, 26 Jul 2023 02:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364451; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XUAbwUholcdP/E78pjIWjWfXWvuCT6ODN3CqmEOBnlY=; b=EEFORplBiEhopXrfaGAEcYSeCrghMnuWVbZDPuQM1r1uHw4bG6isPlizHgQc0idjj5wt9X w0teKbF2IfugL9AsWoLfvsD/QBBx6Wvv3pmInpXfDgkdiUsFhRCPqLD2ASZ3s1uF7BV+sE 11CarqhGOkFi15Ko1BAqAi9Hc6wOEgo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-294-oQFhGE4cMQWWMo61YsirzA-1; Wed, 26 Jul 2023 05:40:46 -0400 X-MC-Unique: oQFhGE4cMQWWMo61YsirzA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4EB1888D583; Wed, 26 Jul 2023 09:40:46 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7CF031401C2E; Wed, 26 Jul 2023 09:40:45 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei Subject: [PATCH V2 3/9] scsi: core: add helper of scsi_max_nr_hw_queues() Date: Wed, 26 Jul 2023 17:40:21 +0800 Message-Id: <20230726094027.535126-4-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org blk_mq_alloc_tag_set() may change nr_hw_queues for kdump kernel, so driver has to take blk-mq max supported nr_hw_queues into account when figuring out nr_hw_queues from hardware info, for avoiding inconsistency between scsi driver and blk-mq. Add helper of scsi_max_nr_hw_queues() for avoiding nr_hw_queues inconsistency between scsi driver and blk-mq. Signed-off-by: Ming Lei --- include/scsi/scsi_host.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 70b7475dcf56..2273f855940f 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -803,6 +803,11 @@ extern int scsi_host_unblock(struct Scsi_Host *shost, int new_state); void scsi_host_busy_iter(struct Scsi_Host *, bool (*fn)(struct scsi_cmnd *, void *), void *priv); +static inline unsigned int scsi_max_nr_hw_queues(void) +{ + return blk_mq_max_nr_hw_queues(); +} + struct class_container; /* From patchwork Wed Jul 26 09:40:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 707466 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2DE2C41513 for ; Wed, 26 Jul 2023 09:41:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233633AbjGZJly (ORCPT ); Wed, 26 Jul 2023 05:41:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233419AbjGZJlm (ORCPT ); Wed, 26 Jul 2023 05:41:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51E0DBC for ; Wed, 26 Jul 2023 02:40:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364453; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fJmMz6GY1NqtDQ+LEnNxukeD0cgVej3lcgCePHgOvCg=; b=C9ZP5Eh6ldWatcWc26VHq3ISsY3tZNsZOeLDDNVcDHmApmP7KKCtKamatph5HnwDR4CPEJ dDR8tSBRG1qGt20Q8z2fZr9EGoQSXNM6BgE3w//UGWZGSLX/q6Fd3k/ifvxrqbyLNUzNID LUAs01pVUgIwHRq8ddnt1cp7jsglYf8= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-135-bqkQtEHjOTuEnIH8CcLJ7g-1; Wed, 26 Jul 2023 05:40:50 -0400 X-MC-Unique: bqkQtEHjOTuEnIH8CcLJ7g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BADB129AB3E6; Wed, 26 Jul 2023 09:40:49 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0041A40C2063; Wed, 26 Jul 2023 09:40:48 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Justin Tee , James Smart Subject: [PATCH V2 4/9] scsi: lpfc: use blk_mq_max_nr_hw_queues() to calculate io vectors Date: Wed, 26 Jul 2023 17:40:22 +0800 Message-Id: <20230726094027.535126-5-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Justin Tee Cc: James Smart Signed-off-by: Ming Lei Reviewed-by: Justin Tee --- drivers/scsi/lpfc/lpfc_init.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 3221a934066b..c546e5275108 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -13022,6 +13022,8 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba) cpu = cpumask_first(aff_mask); cpu_select = lpfc_next_online_cpu(aff_mask, cpu); } else { + vectors = min_t(unsigned int, vectors, + scsi_max_nr_hw_queues()); flags |= PCI_IRQ_AFFINITY; } From patchwork Wed Jul 26 09:40:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 706881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F34B5C04A94 for ; Wed, 26 Jul 2023 09:41:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233665AbjGZJl5 (ORCPT ); Wed, 26 Jul 2023 05:41:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232469AbjGZJlr (ORCPT ); Wed, 26 Jul 2023 05:41:47 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED53111F for ; Wed, 26 Jul 2023 02:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364457; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tpdmXVrSFhq/BTLlYN4vy8lpJH7u5Vwwe/yeAm4djqM=; b=Upwl+OcNHHghyTNwr03FR+rgAr+5Oipc1V0Bt9VInVlRl1tbaf4XIO4hGSVmBQacRLxC+U OCEWeqm6yn0augRRKvqKTey4qXT3FqUtibxxXy8ux6mUSlEIJDDpAYDkBRtyDHWQIFanQA C706BRsxBjqMzxqUgRAtgAqc+Nk06+Y= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-481-0nPrqDWGMH-CodtW9dOuMg-1; Wed, 26 Jul 2023 05:40:53 -0400 X-MC-Unique: 0nPrqDWGMH-CodtW9dOuMg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 36142104458B; Wed, 26 Jul 2023 09:40:53 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 21C23200B41D; Wed, 26 Jul 2023 09:40:51 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Xiang Chen Subject: [PATCH V2 5/9] scsi: hisi: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:23 +0800 Message-Id: <20230726094027.535126-6-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Xiang Chen Signed-off-by: Ming Lei --- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c index 20e1607c6282..60d2301e7f9d 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c @@ -2550,6 +2550,9 @@ static int interrupt_preinit_v3_hw(struct hisi_hba *hisi_hba) hisi_hba->cq_nvecs = vectors - BASE_VECTORS_V3_HW - hisi_hba->iopoll_q_cnt; + if (hisi_hba->cq_nvecs > scsi_max_nr_hw_queues()) + hisi_hba->cq_nvecs = scsi_max_nr_hw_queues(); + shost->nr_hw_queues = hisi_hba->cq_nvecs + hisi_hba->iopoll_q_cnt; return devm_add_action(&pdev->dev, hisi_sas_v3_free_vectors, pdev); From patchwork Wed Jul 26 09:40:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 707465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0010C0015E for ; Wed, 26 Jul 2023 09:42:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233734AbjGZJmD (ORCPT ); Wed, 26 Jul 2023 05:42:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233695AbjGZJlv (ORCPT ); Wed, 26 Jul 2023 05:41:51 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46789121 for ; Wed, 26 Jul 2023 02:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364464; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2DxJS8RxHnnXfV+djc15xnTM9KjPIA9NavQgadLTDR8=; b=bUFoL8jbpbbMVT0/u+JH6RIB1JtDxgp02H33z/ImVjjBMRhs/hRwLfgKVFV+CL6Icfor4V FMwFUmOwMzCwG8rQ2YTu9dUBoXJKjTQNcCdVkjEAHNWZR1Dos27o9+u55m9xqyD8OLB9U3 OFRSpnG+HcKo1nE517dsO/T7Af1MQco= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-315-w_AlwBYUO3Sf2SeTYlwxOw-1; Wed, 26 Jul 2023 05:40:57 -0400 X-MC-Unique: w_AlwBYUO3Sf2SeTYlwxOw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 04DE285A58A; Wed, 26 Jul 2023 09:40:57 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 356B0F782E; Wed, 26 Jul 2023 09:40:55 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Sreekanth Reddy , Sathya Prakash Veerichetty , Kashyap Desai , Sumit Saxena Subject: [PATCH V2 6/9] scsi: mpi3mr: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:24 +0800 Message-Id: <20230726094027.535126-7-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Sreekanth Reddy Cc: Sathya Prakash Veerichetty Cc: Kashyap Desai Cc: Sumit Saxena Signed-off-by: Ming Lei --- drivers/scsi/mpi3mr/mpi3mr_fw.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/scsi/mpi3mr/mpi3mr_fw.c b/drivers/scsi/mpi3mr/mpi3mr_fw.c index 5fa07d6ee5b8..8fe14e728af6 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_fw.c +++ b/drivers/scsi/mpi3mr/mpi3mr_fw.c @@ -815,6 +815,9 @@ static int mpi3mr_setup_isr(struct mpi3mr_ioc *mrioc, u8 setup_one) desc.post_vectors = mrioc->requested_poll_qcount; min_vec = desc.pre_vectors + desc.post_vectors; + if (max_vectors - min_vec > scsi_max_nr_hw_queues()) + max_vectors = min_vec + scsi_max_nr_hw_queues(); + irq_flags |= PCI_IRQ_AFFINITY | PCI_IRQ_ALL_TYPES; retval = pci_alloc_irq_vectors_affinity(mrioc->pdev, From patchwork Wed Jul 26 09:40:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 706880 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBF29C001E0 for ; Wed, 26 Jul 2023 09:42:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233721AbjGZJmP (ORCPT ); Wed, 26 Jul 2023 05:42:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233717AbjGZJlz (ORCPT ); Wed, 26 Jul 2023 05:41:55 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7ECFA122 for ; Wed, 26 Jul 2023 02:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364464; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ketWVMOak+Jss00Ni7DMSRSVvfoPSxigoCA6y+QxH1E=; b=SAHrv4+DnJH0WOX2BNXO1JwAF9C2tPPJgD5TTUizGwnCpRSQ9edG1g4R3H6As71fyvxkTw +CCaSFZ8MZaW/lJS4cb2jvhUudQZJb0PTNiX+jZOi6K+ALcw4YzyzjCsouheGD0aPmLJVr SsHAvf9gxrHxM9RqwulhJRs/fHZXnbs= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-88-QbZBECAnMwK9prNNVi7zDQ-1; Wed, 26 Jul 2023 05:41:01 -0400 X-MC-Unique: QbZBECAnMwK9prNNVi7zDQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1E55D29AB3E3; Wed, 26 Jul 2023 09:41:00 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B13A200B41D; Wed, 26 Jul 2023 09:40:58 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Sreekanth Reddy , Sathya Prakash Veerichetty , Kashyap Desai , Sumit Saxena Subject: [PATCH V2 7/9] scsi: megaraid: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:25 +0800 Message-Id: <20230726094027.535126-8-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Sreekanth Reddy Cc: Sathya Prakash Veerichetty Cc: Kashyap Desai Cc: Sumit Saxena Signed-off-by: Ming Lei --- drivers/scsi/megaraid/megaraid_sas_base.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c index 050eed8e2684..587ccbc1ec92 100644 --- a/drivers/scsi/megaraid/megaraid_sas_base.c +++ b/drivers/scsi/megaraid/megaraid_sas_base.c @@ -5921,6 +5921,10 @@ __megasas_alloc_irq_vectors(struct megasas_instance *instance) int i, irq_flags; struct irq_affinity desc = { .pre_vectors = instance->low_latency_index_start }; struct irq_affinity *descp = &desc; + unsigned max_vecs = instance->msix_vectors - instance->iopoll_q_count; + + if (max_vecs > scsi_max_nr_hw_queues()) + max_vecs = scsi_max_nr_hw_queues(); irq_flags = PCI_IRQ_MSIX; @@ -5934,7 +5938,7 @@ __megasas_alloc_irq_vectors(struct megasas_instance *instance) */ i = pci_alloc_irq_vectors_affinity(instance->pdev, instance->low_latency_index_start, - instance->msix_vectors - instance->iopoll_q_count, irq_flags, descp); + max_vecs, irq_flags, descp); return i; } From patchwork Wed Jul 26 09:40:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 706879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D110C0015E for ; Wed, 26 Jul 2023 09:42:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233695AbjGZJmW (ORCPT ); Wed, 26 Jul 2023 05:42:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58160 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233699AbjGZJmC (ORCPT ); Wed, 26 Jul 2023 05:42:02 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E51513D for ; Wed, 26 Jul 2023 02:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364470; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jmP+yqN1CMl4seuzyOdC4zCzUZWNFx4ZDmn6s0/7WWk=; b=Ddndm1PQlpf+ivSPX+S9cidNCrEglgxp2wcmRzxVjbhj9pD3TbkuYtT8IoZ5QH8/mi44TR wDdWucJZKpylj0yUSjA7mKCEjXhpIypEWW9FvWBaqSvulATcdPHSikx7pQYa4FavYbIUBL Q6sfGOsJF2F7/Gqfud0E97W8qbk+N+A= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-492-8sZ-sCqpOy-Gs1sNG80Ajw-1; Wed, 26 Jul 2023 05:41:04 -0400 X-MC-Unique: 8sZ-sCqpOy-Gs1sNG80Ajw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 80B303815F65; Wed, 26 Jul 2023 09:41:03 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF7D9200B41D; Wed, 26 Jul 2023 09:41:02 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Sathya Prakash , Sreekanth Reddy , Suganath Prabu Subramani Subject: [PATCH V2 8/9] scsi: mpt3sas: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:26 +0800 Message-Id: <20230726094027.535126-9-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Cc: Sathya Prakash Cc: Sreekanth Reddy Cc: Suganath Prabu Subramani Signed-off-by: Ming Lei --- drivers/scsi/mpt3sas/mpt3sas_base.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 53f5492579cb..d238e0275edd 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -3332,8 +3332,8 @@ _base_alloc_irq_vectors(struct MPT3SAS_ADAPTER *ioc) * Don't allocate msix vectors for poll_queues. * msix_vectors is always within a range of FW supported reply queue. */ - int nr_msix_vectors = ioc->iopoll_q_start_index; - + int nr_msix_vectors = min_t(unsigned int, ioc->iopoll_q_start_index, + scsi_max_nr_hw_queues()); if (ioc->smp_affinity_enable) irq_flags |= PCI_IRQ_AFFINITY | PCI_IRQ_ALL_TYPES; From patchwork Wed Jul 26 09:40:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 707464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A32DC001DE for ; Wed, 26 Jul 2023 09:42:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233776AbjGZJmZ (ORCPT ); Wed, 26 Jul 2023 05:42:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233735AbjGZJmD (ORCPT ); Wed, 26 Jul 2023 05:42:03 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D914113E for ; Wed, 26 Jul 2023 02:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690364471; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=991prPOr3CC0OYGragucunFfWE1ASbB8T8+eCTuUx4g=; b=GESk7ezAkcSkYgwui/QwYZ92WO56zoNdwLfCG4bpa4yROBZGZVJSCk4XZ4PzdqpUvy7x4D YPP29mDkbL1ddJD6x82Rk5+meQrZR1tRa7rcW3Bbz0ihRkNL6laW9JvBDNQOVLjK0Yx1qX WcJ6dH7E6tWyS4+TcyP0HSf7d8rZYI0= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-438-06SQxYjyMtOj-FMRXjmHLA-1; Wed, 26 Jul 2023 05:41:07 -0400 X-MC-Unique: 06SQxYjyMtOj-FMRXjmHLA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F1EF53815F60; Wed, 26 Jul 2023 09:41:06 +0000 (UTC) Received: from localhost (unknown [10.72.120.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2ED79492C13; Wed, 26 Jul 2023 09:41:05 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: linux-block@vger.kernel.org, Wen Xiong , Keith Busch , Ming Lei , Jack Wang Subject: [PATCH V2 9/9] scsi: pm8001: take blk_mq_max_nr_hw_queues() into account for calculating io vectors Date: Wed, 26 Jul 2023 17:40:27 +0800 Message-Id: <20230726094027.535126-10-ming.lei@redhat.com> In-Reply-To: <20230726094027.535126-1-ming.lei@redhat.com> References: <20230726094027.535126-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Take blk-mq's knowledge into account for calculating io queues. Fix wrong queue mapping in case of kdump kernel. On arm and ppc64, 'maxcpus=1' is passed to kdump kernel command line, see `Documentation/admin-guide/kdump/kdump.rst`, so num_possible_cpus() still returns all CPUs because 'maxcpus=1' just bring up one single cpu core during booting. blk-mq sees single queue in kdump kernel, and in driver's viewpoint there are still multiple queues, this inconsistency causes driver to apply wrong queue mapping for handling IO, and IO timeout is triggered. Meantime, single queue makes much less resource utilization, and reduce risk of kernel failure. Acked-by: Jack Wang Signed-off-by: Ming Lei --- drivers/scsi/pm8001/pm8001_init.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c index 2e886c1d867d..3e769f4fe26d 100644 --- a/drivers/scsi/pm8001/pm8001_init.c +++ b/drivers/scsi/pm8001/pm8001_init.c @@ -965,6 +965,8 @@ static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) rc = pci_alloc_irq_vectors(pm8001_ha->pdev, 1, 1, PCI_IRQ_MSIX); } else { + unsigned int max_vecs = min_t(unsigned int, PM8001_MAX_MSIX_VEC, + scsi_max_nr_hw_queues() + 1); /* * Queue index #0 is used always for housekeeping, so don't * include in the affinity spreading. @@ -973,7 +975,7 @@ static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) .pre_vectors = 1, }; rc = pci_alloc_irq_vectors_affinity( - pm8001_ha->pdev, 2, PM8001_MAX_MSIX_VEC, + pm8001_ha->pdev, 2, max_vecs, PCI_IRQ_MSIX | PCI_IRQ_AFFINITY, &desc); }