From patchwork Tue Nov 26 09:14:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hannes Reinecke X-Patchwork-Id: 180180 Delivered-To: patch@linaro.org Received: by 2002:a92:38d5:0:0:0:0:0 with SMTP id g82csp4361629ilf; Tue, 26 Nov 2019 01:14:28 -0800 (PST) X-Google-Smtp-Source: APXvYqxLhdu6tkVRkpOXuDU0WpJxr6Im887WPQqYKl3NKXWkIlfMw6oZKrbAR8HrdbdRHR/yRFv+ X-Received: by 2002:a50:a52e:: with SMTP id y43mr22208609edb.147.1574759668023; Tue, 26 Nov 2019 01:14:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574759668; cv=none; d=google.com; s=arc-20160816; b=QLhkvxwWQ00XEwOgMx999n8nUpR6VBndO5vVGk6xst7p4S8svQD7NTo8OjJcE9Or9u Z5wd6C2EqgxLow4U0qGcluUXKhPBj7crNNj8hV2aS71gufh3sr2LzqRBHGOrFOUZFBc2 COOCkpT+/MjG3E0t3QtWgcKrYzskhJWP66w0dNSLmc2INBtS78dyAhCt707dH0Qw0AvV booLpfaO/zR7i4ywD70kcy+05DYvndJW0H6NePBZdQfUW7liV/i9ZDKzJObiqlr2Wju/ iR1q5+EpWvLF6MoCXjfEY+7MHvZGcpM4ZbXSYjW8f4fWn5PbHKPHML9WIIkaDRHijqEK Vp2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=GuF9wVJBczF/3gcvf3nPttjMBNRN9Nv5PjfWIC5egxs=; b=ylHdyLyaAPHwOyOMb5FergzNC3V4hBrzNasMJPuxWOUzLSHCSX1v+dciPgee2W4HcT VU8DfaoeUbm9sz2Ld7nx04+2dmdReSNzMU3FReECDSKz9gy7BAPNJmDQnIpNvRV0BW85 A7QekJxqhAMRHJ+d8H/x3ofA6PbFRiz+vCNdzpLZeDELB+EPAAzHQNVIS61kl+53F4yt cCVuJv/v/F4GVapWyB7pssDfaxyMFJiz5nH7Edktb06+VaSdQo4XRAXLF+YYe1dvcH2U fnvMO04ZL2n3r+lWxBXGaGeOcs+yxwvRaW7Pgpz/y15pTibaGj+4g67T311Fgx3E72n+ Uyyg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d7si7580803eda.429.2019.11.26.01.14.27 for ; Tue, 26 Nov 2019 01:14:28 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727518AbfKZJO0 (ORCPT ); Tue, 26 Nov 2019 04:14:26 -0500 Received: from mx2.suse.de ([195.135.220.15]:41596 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727305AbfKZJO0 (ORCPT ); Tue, 26 Nov 2019 04:14:26 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 8F086ACA4; Tue, 26 Nov 2019 09:14:23 +0000 (UTC) From: Hannes Reinecke To: "Martin K. Petersen" Cc: Christoph Hellwig , James Bottomley , Ming Lei , Bart van Assche , John Garry , linux-scsi@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH 1/8] blk-mq: Remove some unused function arguments Date: Tue, 26 Nov 2019 10:14:09 +0100 Message-Id: <20191126091416.20052-2-hare@suse.de> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20191126091416.20052-1-hare@suse.de> References: <20191126091416.20052-1-hare@suse.de> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: John Garry The struct blk_mq_hw_ctx * argument in blk_mq_put_tag(), blk_mq_poll_nsecs(), and blk_mq_poll_hybrid_sleep() is unused, so remove it. Reviewed-by: Hannes Reinecke Signed-off-by: John Garry --- block/blk-mq-tag.c | 4 ++-- block/blk-mq-tag.h | 4 ++-- block/blk-mq.c | 10 ++++------ block/blk-mq.h | 2 +- 4 files changed, 9 insertions(+), 11 deletions(-) -- 2.16.4 diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index 008388e82b5c..53b4a9414fbd 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -191,8 +191,8 @@ unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data) return tag + tag_offset; } -void blk_mq_put_tag(struct blk_mq_hw_ctx *hctx, struct blk_mq_tags *tags, - struct blk_mq_ctx *ctx, unsigned int tag) +void blk_mq_put_tag(struct blk_mq_tags *tags, struct blk_mq_ctx *ctx, + unsigned int tag) { if (!blk_mq_tag_is_reserved(tags, tag)) { const int real_tag = tag - tags->nr_reserved_tags; diff --git a/block/blk-mq-tag.h b/block/blk-mq-tag.h index 61deab0b5a5a..66d04dea0bdb 100644 --- a/block/blk-mq-tag.h +++ b/block/blk-mq-tag.h @@ -26,8 +26,8 @@ extern struct blk_mq_tags *blk_mq_init_tags(unsigned int nr_tags, unsigned int r extern void blk_mq_free_tags(struct blk_mq_tags *tags); extern unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data); -extern void blk_mq_put_tag(struct blk_mq_hw_ctx *hctx, struct blk_mq_tags *tags, - struct blk_mq_ctx *ctx, unsigned int tag); +extern void blk_mq_put_tag(struct blk_mq_tags *tags, struct blk_mq_ctx *ctx, + unsigned int tag); extern bool blk_mq_has_free_tags(struct blk_mq_tags *tags); extern int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, struct blk_mq_tags **tags, diff --git a/block/blk-mq.c b/block/blk-mq.c index 6e3b15f70cd7..16aa20d23b67 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -499,9 +499,9 @@ static void __blk_mq_free_request(struct request *rq) blk_pm_mark_last_busy(rq); rq->mq_hctx = NULL; if (rq->tag != -1) - blk_mq_put_tag(hctx, hctx->tags, ctx, rq->tag); + blk_mq_put_tag(hctx->tags, ctx, rq->tag); if (sched_tag != -1) - blk_mq_put_tag(hctx, hctx->sched_tags, ctx, sched_tag); + blk_mq_put_tag(hctx->sched_tags, ctx, sched_tag); blk_mq_sched_restart(hctx); blk_queue_exit(q); } @@ -3354,7 +3354,6 @@ static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb) } static unsigned long blk_mq_poll_nsecs(struct request_queue *q, - struct blk_mq_hw_ctx *hctx, struct request *rq) { unsigned long ret = 0; @@ -3387,7 +3386,6 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q, } static bool blk_mq_poll_hybrid_sleep(struct request_queue *q, - struct blk_mq_hw_ctx *hctx, struct request *rq) { struct hrtimer_sleeper hs; @@ -3407,7 +3405,7 @@ static bool blk_mq_poll_hybrid_sleep(struct request_queue *q, if (q->poll_nsec > 0) nsecs = q->poll_nsec; else - nsecs = blk_mq_poll_nsecs(q, hctx, rq); + nsecs = blk_mq_poll_nsecs(q, rq); if (!nsecs) return false; @@ -3462,7 +3460,7 @@ static bool blk_mq_poll_hybrid(struct request_queue *q, return false; } - return blk_mq_poll_hybrid_sleep(q, hctx, rq); + return blk_mq_poll_hybrid_sleep(q, rq); } /** diff --git a/block/blk-mq.h b/block/blk-mq.h index 32c62c64e6c2..78d38b5f2793 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -208,7 +208,7 @@ static inline bool blk_mq_get_dispatch_budget(struct blk_mq_hw_ctx *hctx) static inline void __blk_mq_put_driver_tag(struct blk_mq_hw_ctx *hctx, struct request *rq) { - blk_mq_put_tag(hctx, hctx->tags, rq->mq_ctx, rq->tag); + blk_mq_put_tag(hctx->tags, rq->mq_ctx, rq->tag); rq->tag = -1; if (rq->rq_flags & RQF_MQ_INFLIGHT) { From patchwork Tue Nov 26 09:14:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hannes Reinecke X-Patchwork-Id: 180181 Delivered-To: patch@linaro.org Received: by 2002:a92:38d5:0:0:0:0:0 with SMTP id g82csp4361673ilf; Tue, 26 Nov 2019 01:14:30 -0800 (PST) X-Google-Smtp-Source: APXvYqznwYiAg3SVhpkHgeM2c6v1Oa5GiHV+Ak2y+NYz4S97ZoEEgM/roo1ZrDGRQfAYr/zQTvsE X-Received: by 2002:a17:906:7e04:: with SMTP id e4mr23174230ejr.113.1574759670374; Tue, 26 Nov 2019 01:14:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574759670; cv=none; d=google.com; s=arc-20160816; b=JhEZRtipPRfAdXaq3ryWNsejUn/QYirJKU5e9AD1uAuXo7QxrbORogVczMnJF30vYs D3jCOKEMvQjLsP7rHr7MnTflynULCyUbFIETccKnqm6471Otmhpfc4ZGk5Q2ZAyP81E6 LqJxSr1QwECpFMC5nHtxgIjb7SuQTqmVqbv1bKWQkc0121Rl3wvlh9RsJqg2SyiuQkGs qfJPnNLlDS+HrM9XJN8G93j2aiKSgNyJPSvToFoBvHbcZWb8xpsQV7pXxNRtG1NM/kpb Rk/vSm1X/M0bhBAaverxh8+8VWnJL30A+MDvQUvji4SWtBLPSW0p7g9xnQZgR5l1ujgi GlUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=pSc/x/48J31xDbgPns3Yj0ZhkB+hRFb+mMP71ucw1eU=; b=JyLh4bxB6pkXPs4x+Z3w5iy79lbM46s5ukLYnFvgC4aTxkoa1kfKjnDF2xT0w+rmEU PMZI2ChKNAuYum6cUS+rFfHywDYv2yML7/F0lMECqcuJ0NZIxieCC1WTALwqXFUsq5Bt IEcX25O64hHs2ZiVoj2QQajfrqFL/eBd4qJA7AH3g2v9FPJcFb4VKgjS0+8ZGy9cP9aR Lmwla0SlDb9E/M+mHnG4pos0s1oWH2JgZ3nftJgTQLBsx3PqWl/vxN7m8xjxN72sVOfN lUhvofYgQPBiRSMw0RiicDEj14K2ESE2NfyJyvMRLp29pEwf091jWFpOULlNcHvhzYTI WfvQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d7si7580803eda.429.2019.11.26.01.14.30 for ; Tue, 26 Nov 2019 01:14:30 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727538AbfKZJO3 (ORCPT ); Tue, 26 Nov 2019 04:14:29 -0500 Received: from mx2.suse.de ([195.135.220.15]:41592 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727452AbfKZJO1 (ORCPT ); Tue, 26 Nov 2019 04:14:27 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 8F555BA0B; Tue, 26 Nov 2019 09:14:23 +0000 (UTC) From: Hannes Reinecke To: "Martin K. Petersen" Cc: Christoph Hellwig , James Bottomley , Ming Lei , Bart van Assche , John Garry , linux-scsi@vger.kernel.org, linux-block@vger.kernel.org, Hannes Reinecke Subject: [PATCH 4/8] blk-mq: Facilitate a shared sbitmap per tagset Date: Tue, 26 Nov 2019 10:14:12 +0100 Message-Id: <20191126091416.20052-5-hare@suse.de> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20191126091416.20052-1-hare@suse.de> References: <20191126091416.20052-1-hare@suse.de> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: John Garry Some SCSI HBAs (such as HPSA, megaraid, mpt3sas, hisi_sas_v3 ..) support multiple reply queues with single hostwide tags. In addition, these drivers want to use interrupt assignment in pci_alloc_irq_vectors(PCI_IRQ_AFFINITY). However, as discussed in [0], CPU hotplug may cause in-flight IO completion to not be serviced when an interrupt is shutdown. To solve that problem, Ming's patchset to drain hctx's should ensure no IOs are missed in-flight [1]. However, to take advantage of that patchset, we need to map the HBA HW queues to blk mq hctx's; to do that, we need to expose the HBA HW queues. In making that transition, the per-SCSI command request tags are no longer unique per Scsi host - they are just unique per hctx. As such, the HBA LLDD would have to generate this tag internally, which has a certain performance overhead. However another problem is that blk mq assumes the host may accept (Scsi_host.can_queue * #hw queue) commands. In [2], we removed the Scsi host busy counter, which would stop the LLDD being sent more than .can_queue commands; however, we should still ensure that the block layer does not issue more than .can_queue commands to the Scsi host. To solve this problem, introduce a shared sbitmap per blk_mq_tag_set, which may be requested at init time. New flag BLK_MQ_F_TAG_HCTX_SHARED should be set when requesting the tagset to indicate whether the shared sbitmap should be used. Even when BLK_MQ_F_TAG_HCTX_SHARED is set, we still allocate a full set of tags and requests per hctx; the reason for this is that if we only allocate tags and requests for a single hctx - like hctx0 - we may break block drivers which expect a request be associated with a specific hctx, i.e. not hctx0. This is based on work originally from Ming Lei in [3] and from Bart's suggestion in [4]. [0] https://lore.kernel.org/linux-block/alpine.DEB.2.21.1904051331270.1802@nanos.tec.linutronix.de/ [1] https://lore.kernel.org/linux-block/20191014015043.25029-1-ming.lei@redhat.com/ [2] https://lore.kernel.org/linux-scsi/20191025065855.6309-1-ming.lei@redhat.com/ [3] https://lore.kernel.org/linux-block/20190531022801.10003-1-ming.lei@redhat.com/ [4] https://lore.kernel.org/linux-block/ff77beff-5fd9-9f05-12b6-826922bace1f@huawei.com/T/#m3db0a602f095cbcbff27e9c884d6b4ae826144be Signed-off-by: John Garry Signed-off-by: Hannes Reinecke --- block/blk-mq-sched.c | 8 ++++++-- block/blk-mq-tag.c | 36 +++++++++++++++++++++++++++++------- block/blk-mq-tag.h | 6 +++++- block/blk-mq.c | 45 ++++++++++++++++++++++++++++++++++++--------- block/blk-mq.h | 7 ++++++- include/linux/blk-mq.h | 7 +++++++ 6 files changed, 89 insertions(+), 20 deletions(-) -- 2.16.4 diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index ca22afd47b3d..f3589f42b96d 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -452,7 +452,7 @@ static void blk_mq_sched_free_tags(struct blk_mq_tag_set *set, { if (hctx->sched_tags) { blk_mq_free_rqs(set, hctx->sched_tags, hctx_idx); - blk_mq_free_rq_map(hctx->sched_tags); + blk_mq_free_rq_map(hctx->sched_tags, false); hctx->sched_tags = NULL; } } @@ -462,10 +462,14 @@ static int blk_mq_sched_alloc_tags(struct request_queue *q, unsigned int hctx_idx) { struct blk_mq_tag_set *set = q->tag_set; + int flags = set->flags; int ret; + /* Scheduler tags are never shared */ + set->flags &= ~BLK_MQ_F_TAG_HCTX_SHARED; hctx->sched_tags = blk_mq_alloc_rq_map(set, hctx_idx, q->nr_requests, set->reserved_tags); + set->flags = flags; if (!hctx->sched_tags) return -ENOMEM; @@ -484,7 +488,7 @@ static void blk_mq_sched_tags_teardown(struct request_queue *q) queue_for_each_hw_ctx(q, hctx, i) { if (hctx->sched_tags) { - blk_mq_free_rq_map(hctx->sched_tags); + blk_mq_free_rq_map(hctx->sched_tags, false); hctx->sched_tags = NULL; } } diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index 332d71cd3976..53b4c4c53c6a 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -228,7 +228,7 @@ static bool bt_iter(struct sbitmap *bitmap, unsigned int bitnr, void *data) * We can hit rq == NULL here, because the tagging functions * test and set the bit before assigning ->rqs[]. */ - if (rq && rq->q == hctx->queue) + if (rq && rq->q == hctx->queue && rq->mq_hctx == hctx) return iter_data->fn(hctx, rq, iter_data->data, reserved); return true; } @@ -470,7 +470,27 @@ static int blk_mq_init_bitmap_tags(struct blk_mq_tags *tags, return -ENOMEM; } -struct blk_mq_tags *blk_mq_init_tags(unsigned int total_tags, +bool blk_mq_init_shared_sbitmap(struct blk_mq_tag_set *tag_set) +{ + unsigned int depth = tag_set->queue_depth -tag_set->reserved_tags; + int alloc_policy = BLK_MQ_FLAG_TO_ALLOC_POLICY(tag_set->flags); + bool round_robin = alloc_policy == BLK_TAG_ALLOC_RR; + int node = tag_set->numa_node; + + if (bt_alloc(&tag_set->shared_bitmap_tags, depth, round_robin, node)) + return false; + if (bt_alloc(&tag_set->shared_breserved_tags, tag_set->reserved_tags, round_robin, + node)) + goto free_bitmap_tags; + + return true; +free_bitmap_tags: + sbitmap_queue_free(&tag_set->shared_bitmap_tags); + return false; +} + +struct blk_mq_tags *blk_mq_init_tags(struct blk_mq_tag_set *set, + unsigned int total_tags, unsigned int reserved_tags, int node, int alloc_policy) { @@ -488,9 +508,11 @@ struct blk_mq_tags *blk_mq_init_tags(unsigned int total_tags, tags->nr_tags = total_tags; tags->nr_reserved_tags = reserved_tags; - if (blk_mq_init_bitmap_tags(tags, node, alloc_policy) < 0) { - kfree(tags); - tags = NULL; + if (!blk_mq_is_sbitmap_shared(set)) { + if (blk_mq_init_bitmap_tags(tags, node, alloc_policy) < 0) { + kfree(tags); + tags = NULL; + } } return tags; } @@ -538,12 +560,12 @@ int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, return -ENOMEM; ret = blk_mq_alloc_rqs(set, new, hctx->queue_num, tdepth); if (ret) { - blk_mq_free_rq_map(new); + blk_mq_free_rq_map(new, blk_mq_is_sbitmap_shared(set)); return -ENOMEM; } blk_mq_free_rqs(set, *tagsptr, hctx->queue_num); - blk_mq_free_rq_map(*tagsptr); + blk_mq_free_rq_map(*tagsptr, blk_mq_is_sbitmap_shared(set)); *tagsptr = new; } else { /* diff --git a/block/blk-mq-tag.h b/block/blk-mq-tag.h index 10b66fd4664a..279a861c7e58 100644 --- a/block/blk-mq-tag.h +++ b/block/blk-mq-tag.h @@ -22,7 +22,11 @@ struct blk_mq_tags { }; -extern struct blk_mq_tags *blk_mq_init_tags(unsigned int nr_tags, unsigned int reserved_tags, int node, int alloc_policy); +extern bool blk_mq_init_shared_sbitmap(struct blk_mq_tag_set *tag_set); +extern struct blk_mq_tags *blk_mq_init_tags(struct blk_mq_tag_set *tag_set, + unsigned int nr_tags, + unsigned int reserved_tags, + int node, int alloc_policy); extern void blk_mq_free_tags(struct blk_mq_tags *tags); extern unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data); diff --git a/block/blk-mq.c b/block/blk-mq.c index f0c40fcbd8ae..db87b0f57dbe 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2075,13 +2075,14 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags, } } -void blk_mq_free_rq_map(struct blk_mq_tags *tags) +void blk_mq_free_rq_map(struct blk_mq_tags *tags, bool shared) { kfree(tags->rqs); tags->rqs = NULL; kfree(tags->static_rqs); tags->static_rqs = NULL; - blk_mq_free_tags(tags); + if (!shared) + blk_mq_free_tags(tags); } struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, @@ -2096,7 +2097,7 @@ struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, if (node == NUMA_NO_NODE) node = set->numa_node; - tags = blk_mq_init_tags(nr_tags, reserved_tags, node, + tags = blk_mq_init_tags(set, nr_tags, reserved_tags, node, BLK_MQ_FLAG_TO_ALLOC_POLICY(set->flags)); if (!tags) return NULL; @@ -2105,7 +2106,8 @@ struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, node); if (!tags->rqs) { - blk_mq_free_tags(tags); + if (!blk_mq_is_sbitmap_shared(set)) + blk_mq_free_tags(tags); return NULL; } @@ -2114,7 +2116,8 @@ struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, node); if (!tags->static_rqs) { kfree(tags->rqs); - blk_mq_free_tags(tags); + if (!blk_mq_is_sbitmap_shared(set)) + blk_mq_free_tags(tags); return NULL; } @@ -2446,7 +2449,7 @@ static bool __blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, int hctx_idx) if (!ret) return true; - blk_mq_free_rq_map(set->tags[hctx_idx]); + blk_mq_free_rq_map(set->tags[hctx_idx], blk_mq_is_sbitmap_shared(set)); set->tags[hctx_idx] = NULL; return false; } @@ -2456,7 +2459,8 @@ static void blk_mq_free_map_and_requests(struct blk_mq_tag_set *set, { if (set->tags && set->tags[hctx_idx]) { blk_mq_free_rqs(set, set->tags[hctx_idx], hctx_idx); - blk_mq_free_rq_map(set->tags[hctx_idx]); + blk_mq_free_rq_map(set->tags[hctx_idx], + blk_mq_is_sbitmap_shared(set)); set->tags[hctx_idx] = NULL; } } @@ -2954,7 +2958,7 @@ static int __blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set) out_unwind: while (--i >= 0) - blk_mq_free_rq_map(set->tags[i]); + blk_mq_free_rq_map(set->tags[i], blk_mq_is_sbitmap_shared(set)); return -ENOMEM; } @@ -3099,6 +3103,20 @@ int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) if (ret) goto out_free_mq_map; + if (blk_mq_is_sbitmap_shared(set)) { + if (!blk_mq_init_shared_sbitmap(set)) { + ret = -ENOMEM; + goto out_free_mq_map; + } + + for (i = 0; i < set->nr_hw_queues; i++) { + struct blk_mq_tags *tags = set->tags[i]; + + tags->bitmap_tags = &set->shared_bitmap_tags; + tags->breserved_tags = &set->shared_breserved_tags; + } + } + mutex_init(&set->tag_list_lock); INIT_LIST_HEAD(&set->tag_list); @@ -3168,8 +3186,17 @@ int blk_mq_update_nr_requests(struct request_queue *q, unsigned int nr) q->elevator->type->ops.depth_updated(hctx); } - if (!ret) + if (!ret) { + if (blk_mq_is_sbitmap_shared(set)) { + sbitmap_queue_resize(&set->shared_bitmap_tags, nr); + sbitmap_queue_resize(&set->shared_breserved_tags, nr); + } q->nr_requests = nr; + } + /* + * if ret != 0, q->nr_requests would not be updated, yet the depth + * for some hctx may have changed - is that right? + */ blk_mq_unquiesce_queue(q); blk_mq_unfreeze_queue(q); diff --git a/block/blk-mq.h b/block/blk-mq.h index 78d38b5f2793..c4b8213dfdfc 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -53,7 +53,7 @@ struct request *blk_mq_dequeue_from_ctx(struct blk_mq_hw_ctx *hctx, */ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags, unsigned int hctx_idx); -void blk_mq_free_rq_map(struct blk_mq_tags *tags); +void blk_mq_free_rq_map(struct blk_mq_tags *tags, bool shared); struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, unsigned int hctx_idx, unsigned int nr_tags, @@ -166,6 +166,11 @@ struct blk_mq_alloc_data { struct blk_mq_hw_ctx *hctx; }; +static inline bool blk_mq_is_sbitmap_shared(struct blk_mq_tag_set *tag_set) +{ + return !!(tag_set->flags & BLK_MQ_F_TAG_HCTX_SHARED); +} + static inline struct blk_mq_tags *blk_mq_tags_from_data(struct blk_mq_alloc_data *data) { if (data->flags & BLK_MQ_REQ_INTERNAL) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 147185394a25..670e9a949d32 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -109,6 +109,12 @@ struct blk_mq_tag_set { unsigned int flags; /* BLK_MQ_F_* */ void *driver_data; + struct sbitmap_queue shared_bitmap_tags; + struct sbitmap_queue shared_breserved_tags; + + struct sbitmap_queue sched_shared_bitmap_tags; + struct sbitmap_queue sched_shared_breserved_tags; + struct blk_mq_tags **tags; struct mutex tag_list_lock; @@ -226,6 +232,7 @@ struct blk_mq_ops { enum { BLK_MQ_F_SHOULD_MERGE = 1 << 0, BLK_MQ_F_TAG_QUEUE_SHARED = 1 << 1, + BLK_MQ_F_TAG_HCTX_SHARED = 1 << 2, BLK_MQ_F_BLOCKING = 1 << 5, BLK_MQ_F_NO_SCHED = 1 << 6, BLK_MQ_F_ALLOC_POLICY_START_BIT = 8, From patchwork Tue Nov 26 09:14:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hannes Reinecke X-Patchwork-Id: 180182 Delivered-To: patch@linaro.org Received: by 2002:a92:38d5:0:0:0:0:0 with SMTP id g82csp4361757ilf; Tue, 26 Nov 2019 01:14:34 -0800 (PST) X-Google-Smtp-Source: APXvYqxDlJwSJh0o1ASiek0ZabSRmQA4jRrJFCIrqf+66XMgmM+xAehrQVnEVQcMGuY+gE8IQRgN X-Received: by 2002:a17:906:3107:: with SMTP id 7mr22876551ejx.125.1574759674698; Tue, 26 Nov 2019 01:14:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574759674; cv=none; d=google.com; s=arc-20160816; b=J6q36ctRpPA+9VYzR/P74MaswHvfPKxHiduauITi6Hzwz0OJI8pTEPTRpgy1sLXJGh jmftDv+A/uMi/HDJRh7Pc0Aic9t0y/nE8Ueob21fDePUS9wosWxo8gfyWohqDMt70CYL xWyFS66evL2WF8wDmskN8oh3JIjrowvEUfLbhLHdUMVjQgtJBY8+kIJPPzkKtHSzFmpC rp3kd9L9VS4Rt8bhoTTUagsJ5wFDjvsT02FUMw0KNNdHYfN3PcgNg6KL0GxqAsjFl5kW 3SYq5srhalwoIhPhMEyVFShCpYX4crmL+1HcpoDKyA867ATFdorohtsxPYsLy+zD8nd9 VyTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=2QriT5WgwfLU4n0/Rv6uE0jur1Odi2IFMPsJteSIS6o=; b=OhvUjEIm3VmJtZNqvd+2Bvbvr/Fid6VUQ6RfHiEyqwnYRpJKNaEhb05dH93moeH6N+ dZm1g2d6dGDQ3cpdds2XBrR3XDVBkVXBE0D0R73PilC32Z6DDynqFNtCPV0ddcTesgjt 7nbznhaqG8ZCUOl/3ssNIncNR1My9ApPtACe+jjSOU0Bkt22+Zzu1s8PO1h8CSyRnmNF Z4hHPy9jytC7YxpkwAG87M0w4P0g6w0WgcqZTqjH5mbRGEHxnG2D9UCabsdZBASQnGg5 +LhIn/9aneIZOiMV3bgVj7AG39Om93BhBU0OF+En4IyhWSjV6C+CMwjav7O687mwLKjn sXmg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d7si7580803eda.429.2019.11.26.01.14.34 for ; Tue, 26 Nov 2019 01:14:34 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-scsi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727524AbfKZJO3 (ORCPT ); Tue, 26 Nov 2019 04:14:29 -0500 Received: from mx2.suse.de ([195.135.220.15]:41738 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727495AbfKZJO2 (ORCPT ); Tue, 26 Nov 2019 04:14:28 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id E8A05BA54; Tue, 26 Nov 2019 09:14:23 +0000 (UTC) From: Hannes Reinecke To: "Martin K. Petersen" Cc: Christoph Hellwig , James Bottomley , Ming Lei , Bart van Assche , John Garry , linux-scsi@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH 6/8] scsi: hisi_sas: Switch v3 hw to MQ Date: Tue, 26 Nov 2019 10:14:14 +0100 Message-Id: <20191126091416.20052-7-hare@suse.de> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20191126091416.20052-1-hare@suse.de> References: <20191126091416.20052-1-hare@suse.de> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: John Garry Now that the block layer provides a shared tag, we can switch the driver to expose all HW queues. Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas.h | 3 +- drivers/scsi/hisi_sas/hisi_sas_main.c | 36 +++++++------- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 86 ++++++++++++++-------------------- 3 files changed, 56 insertions(+), 69 deletions(-) -- 2.16.4 diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h index 233c73e01246..0405602df2a4 100644 --- a/drivers/scsi/hisi_sas/hisi_sas.h +++ b/drivers/scsi/hisi_sas/hisi_sas.h @@ -8,6 +8,8 @@ #define _HISI_SAS_H_ #include +#include +#include #include #include #include @@ -431,7 +433,6 @@ struct hisi_hba { u32 intr_coal_count; /* Interrupt count to coalesce */ int cq_nvecs; - unsigned int *reply_map; /* bist */ enum sas_linkrate debugfs_bist_linkrate; diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index 03588ec3c394..e185935c3399 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -421,6 +421,7 @@ static int hisi_sas_task_prep(struct sas_task *task, struct device *dev = hisi_hba->dev; int dlvry_queue_slot, dlvry_queue, rc, slot_idx; int n_elem = 0, n_elem_dif = 0, n_elem_req = 0; + struct scsi_cmnd *scmd = NULL; struct hisi_sas_dq *dq; unsigned long flags; int wr_q_index; @@ -436,10 +437,23 @@ static int hisi_sas_task_prep(struct sas_task *task, return -ECOMM; } - if (hisi_hba->reply_map) { - int cpu = raw_smp_processor_id(); - unsigned int dq_index = hisi_hba->reply_map[cpu]; + if (task->uldd_task) { + struct ata_queued_cmd *qc; + if (dev_is_sata(device)) { + qc = task->uldd_task; + scmd = qc->scsicmd; + } else { + scmd = task->uldd_task; + } + } + + if (scmd) { + unsigned int dq_index; + u32 blk_tag; + + blk_tag = blk_mq_unique_tag(scmd->request); + dq_index = blk_mq_unique_tag_to_hwq(blk_tag); *dq_pointer = dq = &hisi_hba->dq[dq_index]; } else { *dq_pointer = dq = sas_dev->dq; @@ -468,21 +482,9 @@ static int hisi_sas_task_prep(struct sas_task *task, if (hisi_hba->hw->slot_index_alloc) rc = hisi_hba->hw->slot_index_alloc(hisi_hba, device); - else { - struct scsi_cmnd *scsi_cmnd = NULL; - - if (task->uldd_task) { - struct ata_queued_cmd *qc; + else + rc = hisi_sas_slot_index_alloc(hisi_hba, scmd); - if (dev_is_sata(device)) { - qc = task->uldd_task; - scsi_cmnd = qc->scsicmd; - } else { - scsi_cmnd = task->uldd_task; - } - } - rc = hisi_sas_slot_index_alloc(hisi_hba, scsi_cmnd); - } if (rc < 0) goto err_out_dif_dma_unmap; diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c index bf5d5f138437..e7b015d88968 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c @@ -2353,66 +2353,35 @@ static irqreturn_t cq_interrupt_v3_hw(int irq_no, void *p) return IRQ_HANDLED; } -static void setup_reply_map_v3_hw(struct hisi_hba *hisi_hba, int nvecs) +static int interrupt_preinit_v3_hw(struct hisi_hba *hisi_hba) { - const struct cpumask *mask; - int queue, cpu; - - for (queue = 0; queue < nvecs; queue++) { - struct hisi_sas_cq *cq = &hisi_hba->cq[queue]; + int vectors; + int max_msi = HISI_SAS_MSI_COUNT_V3_HW, min_msi; + struct Scsi_Host *shost = hisi_hba->shost; + struct irq_affinity desc = { + .pre_vectors = BASE_VECTORS_V3_HW, + }; + + min_msi = MIN_AFFINE_VECTORS_V3_HW; + vectors = pci_alloc_irq_vectors_affinity(hisi_hba->pci_dev, + min_msi, max_msi, + PCI_IRQ_MSI | + PCI_IRQ_AFFINITY, + &desc); + if (vectors < 0) + return -ENOENT; - mask = pci_irq_get_affinity(hisi_hba->pci_dev, queue + - BASE_VECTORS_V3_HW); - if (!mask) - goto fallback; - cq->pci_irq_mask = mask; - for_each_cpu(cpu, mask) - hisi_hba->reply_map[cpu] = queue; - } - return; + hisi_hba->cq_nvecs = vectors - BASE_VECTORS_V3_HW; + shost->nr_hw_queues = hisi_hba->cq_nvecs; -fallback: - for_each_possible_cpu(cpu) - hisi_hba->reply_map[cpu] = cpu % hisi_hba->queue_count; - /* Don't clean all CQ masks */ + return 0; } static int interrupt_init_v3_hw(struct hisi_hba *hisi_hba) { struct device *dev = hisi_hba->dev; struct pci_dev *pdev = hisi_hba->pci_dev; - int vectors, rc, i; - int max_msi = HISI_SAS_MSI_COUNT_V3_HW, min_msi; - - if (auto_affine_msi_experimental) { - struct irq_affinity desc = { - .pre_vectors = BASE_VECTORS_V3_HW, - }; - - min_msi = MIN_AFFINE_VECTORS_V3_HW; - - hisi_hba->reply_map = devm_kcalloc(dev, nr_cpu_ids, - sizeof(unsigned int), - GFP_KERNEL); - if (!hisi_hba->reply_map) - return -ENOMEM; - vectors = pci_alloc_irq_vectors_affinity(hisi_hba->pci_dev, - min_msi, max_msi, - PCI_IRQ_MSI | - PCI_IRQ_AFFINITY, - &desc); - if (vectors < 0) - return -ENOENT; - setup_reply_map_v3_hw(hisi_hba, vectors - BASE_VECTORS_V3_HW); - } else { - min_msi = max_msi; - vectors = pci_alloc_irq_vectors(hisi_hba->pci_dev, min_msi, - max_msi, PCI_IRQ_MSI); - if (vectors < 0) - return vectors; - } - - hisi_hba->cq_nvecs = vectors - BASE_VECTORS_V3_HW; + int rc, i; rc = devm_request_irq(dev, pci_irq_vector(pdev, 1), int_phy_up_down_bcast_v3_hw, 0, @@ -3057,6 +3026,15 @@ static int debugfs_set_bist_v3_hw(struct hisi_hba *hisi_hba, bool enable) return 0; } +static int hisi_sas_map_queues(struct Scsi_Host *shost) +{ + struct hisi_hba *hisi_hba = shost_priv(shost); + struct blk_mq_queue_map *qmap = &shost->tag_set.map[HCTX_TYPE_DEFAULT]; + + return blk_mq_pci_map_queues(qmap, hisi_hba->pci_dev, + BASE_VECTORS_V3_HW); +} + static struct scsi_host_template sht_v3_hw = { .name = DRV_NAME, .module = THIS_MODULE, @@ -3065,6 +3043,7 @@ static struct scsi_host_template sht_v3_hw = { .slave_configure = hisi_sas_slave_configure, .scan_finished = hisi_sas_scan_finished, .scan_start = hisi_sas_scan_start, + .map_queues = hisi_sas_map_queues, .change_queue_depth = sas_change_queue_depth, .bios_param = sas_bios_param, .this_id = -1, @@ -3078,6 +3057,7 @@ static struct scsi_host_template sht_v3_hw = { .shost_attrs = host_attrs_v3_hw, .tag_alloc_policy = BLK_TAG_ALLOC_RR, .host_reset = hisi_sas_host_reset, + .host_tagset = 1, }; static const struct hisi_sas_hw hisi_sas_v3_hw = { @@ -3249,6 +3229,10 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (hisi_sas_debugfs_enable) hisi_sas_debugfs_init(hisi_hba); + rc = interrupt_preinit_v3_hw(hisi_hba); + if (rc) + goto err_out_ha; + dev_err(dev, "%d hw qeues\n", shost->nr_hw_queues); rc = scsi_add_host(shost, dev); if (rc) goto err_out_ha;