From patchwork Thu Jan 14 20:31:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0CBFC433DB for ; Thu, 14 Jan 2021 20:34:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7670223A6C for ; Thu, 14 Jan 2021 20:34:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727346AbhANUcm (ORCPT ); Thu, 14 Jan 2021 15:32:42 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:59024 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726066AbhANUcl (ORCPT ); Thu, 14 Jan 2021 15:32:41 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKVe2b081769; Thu, 14 Jan 2021 15:31:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Ml8kfxsPhwNXqasmcOZfmEsQBI6122H+O2Q5MbHy7Hk=; b=V39VVUp6TPglLtq0ZAznOvkH4qqbUHcR+fYEDZQtq7u7Rz77rmhboBl1FdLzsER1N/2j D4DmGxglpggJk3XsOl/sZgUICgf4FXFI0EudxxujHMfmXbzNuuumsrH0qjQd1ugw8fxO JmFuWcBDfk3Z1r5rehUlMK+2i7HF67LsIVopooh9a+s7jY12rnF2/uVpbVOGxlIS2Njl yCDQhdDDfhNDXN0Y2X257YE8tg0e3G42syVFx340WAdKUGu5ljSYF1ULeUWdXlRfqCHn Fw7ds+TAAlx2X5gP1otUNsP1E9tEYQ/VTBuXNMqWVScvQ1ydE0LWklkN5eXVntMbmyvG LA== Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0b-001b2d01.pphosted.com with ESMTP id 362vcvh1fv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:53 -0500 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKSkTN012299; Thu, 14 Jan 2021 20:31:52 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma04dal.us.ibm.com with ESMTP id 35y449ss3p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:52 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVoMq33227176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:50 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B7A6A6E04E; Thu, 14 Jan 2021 20:31:50 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 651796E04C; Thu, 14 Jan 2021 20:31:50 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:50 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 01/21] ibmvfc: add vhost fields and defaults for MQ enablement Date: Thu, 14 Jan 2021 14:31:28 -0600 Message-Id: <20210114203148.246656-2-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 bulkscore=0 lowpriorityscore=0 clxscore=1015 phishscore=0 adultscore=0 mlxlogscore=999 spamscore=0 impostorscore=0 mlxscore=0 priorityscore=1501 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce several new vhost fields for managing MQ state of the adapter as well as initial defaults for MQ enablement. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 8 ++++++++ drivers/scsi/ibmvscsi/ibmvfc.h | 9 +++++++++ 2 files changed, 17 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index ba95438a8912..9200fe49c57e 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3302,6 +3302,7 @@ static struct scsi_host_template driver_template = { .max_sectors = IBMVFC_MAX_SECTORS, .shost_attrs = ibmvfc_attrs, .track_queue_depth = 1, + .host_tagset = 1, }; /** @@ -5290,6 +5291,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) shost->max_sectors = IBMVFC_MAX_SECTORS; shost->max_cmd_len = IBMVFC_MAX_CDB_LEN; shost->unique_id = shost->host_no; + shost->nr_hw_queues = IBMVFC_MQ ? IBMVFC_SCSI_HW_QUEUES : 1; vhost = shost_priv(shost); INIT_LIST_HEAD(&vhost->targets); @@ -5300,6 +5302,12 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->partition_number = -1; vhost->log_level = log_level; vhost->task_set = 1; + + vhost->mq_enabled = IBMVFC_MQ; + vhost->client_scsi_channels = IBMVFC_SCSI_CHANNELS; + vhost->using_channels = 0; + vhost->do_enquiry = 1; + strcpy(vhost->partition_name, "UNKNOWN"); init_waitqueue_head(&vhost->work_wait_q); init_waitqueue_head(&vhost->init_wait_q); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 632e977449c5..dd6d89292867 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -41,6 +41,11 @@ #define IBMVFC_DEFAULT_LOG_LEVEL 2 #define IBMVFC_MAX_CDB_LEN 16 #define IBMVFC_CLS3_ERROR 0 +#define IBMVFC_MQ 0 +#define IBMVFC_SCSI_CHANNELS 0 +#define IBMVFC_SCSI_HW_QUEUES 1 +#define IBMVFC_MIG_NO_SUB_TO_CRQ 0 +#define IBMVFC_MIG_NO_N_TO_M 0 /* * Ensure we have resources for ERP and initialization: @@ -840,6 +845,10 @@ struct ibmvfc_host { int delay_init; int scan_complete; int logged_in; + int mq_enabled; + int using_channels; + int do_enquiry; + int client_scsi_channels; int aborting_passthru; int events_to_log; #define IBMVFC_AE_LINKUP 0x0001 From patchwork Thu Jan 14 20:31:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8791BC4321A for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D9EF23B03 for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727192AbhANUcl (ORCPT ); Thu, 14 Jan 2021 15:32:41 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:31104 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725918AbhANUck (ORCPT ); Thu, 14 Jan 2021 15:32:40 -0500 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK3WpO133088; Thu, 14 Jan 2021 15:31:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=w9GOEHUIwadmmQ6/WimiQJ9BqdJfITO+Oan6ZTv5mO4=; b=eWg2vY6Rpq1pKQ9E+c/N4BRnUP81IpJSiX4tmCZ1qYPq7GAQ2r9dnrRSuBG3dNggOzpi Q5saCITWT7H8DEYFeJWWBz6AtAuDORvQkzx6f2K7aIm8Xf4b3L66BiixFAksMPAeCUKN oBf4fhzneIkC+N++QKzaNcTqYSObJNEr9jZ9wZV9qUTkJfeSOv94leXKiakeU6SeueoH Pk2fNjCmfeuenvLZCdTBf2wZKxMQu7VM4MjFht3K+TWc3CPT8OMlqKfn9CQZ0zg5Ixq4 c4SnOmKlaJdFDmSaeFutQ0Fpf4/gcSpizxLA6O7n/tgy7ZnatMsdQ0oYbbBs1DBbj1Yi HQ== Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vnm8pe0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:53 -0500 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKRP40032079; Thu, 14 Jan 2021 20:31:52 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma04wdc.us.ibm.com with ESMTP id 35y449fsjs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:52 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVpHX27066818 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:51 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 31CF16E052; Thu, 14 Jan 2021 20:31:51 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D2F906E04C; Thu, 14 Jan 2021 20:31:50 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:50 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 02/21] ibmvfc: move event pool init/free routines Date: Thu, 14 Jan 2021 14:31:29 -0600 Message-Id: <20210114203148.246656-3-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=959 phishscore=0 mlxscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 impostorscore=0 malwarescore=0 adultscore=0 spamscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The next patch in this series reworks the event pool allocation calls to happen within the individual queue allocation routines instead of as independent calls. Move the init/free routines earlier in ibmvfc.c to prevent undefined reference errors when calling these functions from the queue allocation code. No functional change. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 151 +++++++++++++++++---------------- 1 file changed, 76 insertions(+), 75 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 9200fe49c57e..cd9273a5fadb 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -716,6 +716,82 @@ static int ibmvfc_send_crq_init_complete(struct ibmvfc_host *vhost) return ibmvfc_send_crq(vhost, 0xC002000000000000LL, 0); } +/** + * ibmvfc_init_event_pool - Allocates and initializes the event pool for a host + * @vhost: ibmvfc host who owns the event pool + * + * Returns zero on success. + **/ +static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, + struct ibmvfc_queue *queue) +{ + int i; + struct ibmvfc_event_pool *pool = &queue->evt_pool; + + ENTER; + pool->size = max_requests + IBMVFC_NUM_INTERNAL_REQ; + pool->events = kcalloc(pool->size, sizeof(*pool->events), GFP_KERNEL); + if (!pool->events) + return -ENOMEM; + + pool->iu_storage = dma_alloc_coherent(vhost->dev, + pool->size * sizeof(*pool->iu_storage), + &pool->iu_token, 0); + + if (!pool->iu_storage) { + kfree(pool->events); + return -ENOMEM; + } + + INIT_LIST_HEAD(&queue->sent); + INIT_LIST_HEAD(&queue->free); + spin_lock_init(&queue->l_lock); + + for (i = 0; i < pool->size; ++i) { + struct ibmvfc_event *evt = &pool->events[i]; + + atomic_set(&evt->free, 1); + evt->crq.valid = 0x80; + evt->crq.ioba = cpu_to_be64(pool->iu_token + (sizeof(*evt->xfer_iu) * i)); + evt->xfer_iu = pool->iu_storage + i; + evt->vhost = vhost; + evt->queue = queue; + evt->ext_list = NULL; + list_add_tail(&evt->queue_list, &queue->free); + } + + LEAVE; + return 0; +} + +/** + * ibmvfc_free_event_pool - Frees memory of the event pool of a host + * @vhost: ibmvfc host who owns the event pool + * + **/ +static void ibmvfc_free_event_pool(struct ibmvfc_host *vhost, + struct ibmvfc_queue *queue) +{ + int i; + struct ibmvfc_event_pool *pool = &queue->evt_pool; + + ENTER; + for (i = 0; i < pool->size; ++i) { + list_del(&pool->events[i].queue_list); + BUG_ON(atomic_read(&pool->events[i].free) != 1); + if (pool->events[i].ext_list) + dma_pool_free(vhost->sg_pool, + pool->events[i].ext_list, + pool->events[i].ext_list_token); + } + + kfree(pool->events); + dma_free_coherent(vhost->dev, + pool->size * sizeof(*pool->iu_storage), + pool->iu_storage, pool->iu_token); + LEAVE; +} + /** * ibmvfc_free_queue - Deallocate queue * @vhost: ibmvfc host struct @@ -1312,81 +1388,6 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) strncpy(login_info->drc_name, location, IBMVFC_MAX_NAME); } -/** - * ibmvfc_init_event_pool - Allocates and initializes the event pool for a host - * @vhost: ibmvfc host who owns the event pool - * - * Returns zero on success. - **/ -static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, - struct ibmvfc_queue *queue) -{ - int i; - struct ibmvfc_event_pool *pool = &queue->evt_pool; - - ENTER; - pool->size = max_requests + IBMVFC_NUM_INTERNAL_REQ; - pool->events = kcalloc(pool->size, sizeof(*pool->events), GFP_KERNEL); - if (!pool->events) - return -ENOMEM; - - pool->iu_storage = dma_alloc_coherent(vhost->dev, - pool->size * sizeof(*pool->iu_storage), - &pool->iu_token, 0); - - if (!pool->iu_storage) { - kfree(pool->events); - return -ENOMEM; - } - - INIT_LIST_HEAD(&queue->sent); - INIT_LIST_HEAD(&queue->free); - spin_lock_init(&queue->l_lock); - - for (i = 0; i < pool->size; ++i) { - struct ibmvfc_event *evt = &pool->events[i]; - atomic_set(&evt->free, 1); - evt->crq.valid = 0x80; - evt->crq.ioba = cpu_to_be64(pool->iu_token + (sizeof(*evt->xfer_iu) * i)); - evt->xfer_iu = pool->iu_storage + i; - evt->vhost = vhost; - evt->queue = queue; - evt->ext_list = NULL; - list_add_tail(&evt->queue_list, &queue->free); - } - - LEAVE; - return 0; -} - -/** - * ibmvfc_free_event_pool - Frees memory of the event pool of a host - * @vhost: ibmvfc host who owns the event pool - * - **/ -static void ibmvfc_free_event_pool(struct ibmvfc_host *vhost, - struct ibmvfc_queue *queue) -{ - int i; - struct ibmvfc_event_pool *pool = &queue->evt_pool; - - ENTER; - for (i = 0; i < pool->size; ++i) { - list_del(&pool->events[i].queue_list); - BUG_ON(atomic_read(&pool->events[i].free) != 1); - if (pool->events[i].ext_list) - dma_pool_free(vhost->sg_pool, - pool->events[i].ext_list, - pool->events[i].ext_list_token); - } - - kfree(pool->events); - dma_free_coherent(vhost->dev, - pool->size * sizeof(*pool->iu_storage), - pool->iu_storage, pool->iu_token); - LEAVE; -} - /** * ibmvfc_get_event - Gets the next free event in pool * @vhost: ibmvfc host struct From patchwork Thu Jan 14 20:31:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12952C43381 for ; Thu, 14 Jan 2021 20:34:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CE65323A6C for ; Thu, 14 Jan 2021 20:34:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728963AbhANUdk (ORCPT ); Thu, 14 Jan 2021 15:33:40 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:18626 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726007AbhANUcn (ORCPT ); Thu, 14 Jan 2021 15:32:43 -0500 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKKntT186081; Thu, 14 Jan 2021 15:31:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=KiW4KtoxoUb3zWD6erav8E6aKJ0GSkiplUjOZMLfJNM=; b=N8C96SLdakxFMh+jnGZwVWB05AB38GmJGuA5gZYb613Vj13p7Iy3WQSEralRuuzwL0mO SzrNZnu4Vk7d+D4O/K4url4WeTYoRXtdO2ZWKSgmiEJAHixlHL/cdSsywFGDao5I17vr IP9VXyDQkwFqqiimRM9GaTtXtrCZUjek5K7n/K/yf//8uD7ZPuteBf8DhYO19T4cIPqD s9TH0hNvZkUfYfOUd+sRxLv8MIEx0T9jZYkmC/SZmVWjfxjehFrtPJsJy4Ule5VWdjSJ yJ+3HJHXTlG+ksi81FVGn5HHLRC4RojC8tKSOp93HHJUWKzuR2gAft8ElcNkkDyqYM1H Qg== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vwr07ba-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:56 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKHRYC004746; Thu, 14 Jan 2021 20:31:52 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma02wdc.us.ibm.com with ESMTP id 35y449fry0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:52 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVpUn27066826 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:51 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 932D66E04E; Thu, 14 Jan 2021 20:31:51 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 40AF06E056; Thu, 14 Jan 2021 20:31:51 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:51 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 03/21] ibmvfc: init/free event pool during queue allocation/free Date: Thu, 14 Jan 2021 14:31:30 -0600 Message-Id: <20210114203148.246656-4-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 phishscore=0 lowpriorityscore=0 clxscore=1015 malwarescore=0 mlxlogscore=999 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The event pool and CRQ used to be separate entities of the adapter host structure and as such were allocated and freed independently of each other. Recent work as defined a generic queue structure with an event pool specific to each queue. As such the event pool for each queue shouldn't be allocated/freed independently, but instead performed as part of the queue allocation/free routines. Move the calls to ibmvfc_event_pool_{init|free} into ibmvfc_{alloc|free}_queue respectively. The only functional change here is that the CRQ cannot be released in ibmvfc_remove until after the event pool has been successfully purged since releasing the queue will also free the event pool. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index cd9273a5fadb..9330f5a65a7e 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -807,6 +807,8 @@ static void ibmvfc_free_queue(struct ibmvfc_host *vhost, dma_unmap_single(dev, queue->msg_token, PAGE_SIZE, DMA_BIDIRECTIONAL); free_page((unsigned long)queue->msgs.handle); queue->msgs.handle = NULL; + + ibmvfc_free_event_pool(vhost, queue); } /** @@ -5019,6 +5021,10 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, switch (fmt) { case IBMVFC_CRQ_FMT: fmt_size = sizeof(*queue->msgs.crq); + if (ibmvfc_init_event_pool(vhost, queue)) { + dev_err(dev, "Couldn't initialize event pool.\n"); + return -ENOMEM; + } break; case IBMVFC_ASYNC_FMT: fmt_size = sizeof(*queue->msgs.async); @@ -5333,13 +5339,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) goto kill_kthread; } - if ((rc = ibmvfc_init_event_pool(vhost, &vhost->crq))) { - dev_err(dev, "Couldn't initialize event pool. rc=%d\n", rc); - goto release_crq; - } - if ((rc = scsi_add_host(shost, dev))) - goto release_event_pool; + goto release_crq; fc_host_dev_loss_tmo(shost) = IBMVFC_DEV_LOSS_TMO; @@ -5362,8 +5363,6 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) remove_shost: scsi_remove_host(shost); -release_event_pool: - ibmvfc_free_event_pool(vhost, &vhost->crq); release_crq: ibmvfc_release_crq_queue(vhost); kill_kthread: @@ -5398,7 +5397,6 @@ static int ibmvfc_remove(struct vio_dev *vdev) spin_unlock_irqrestore(vhost->host->host_lock, flags); ibmvfc_wait_while_resetting(vhost); - ibmvfc_release_crq_queue(vhost); kthread_stop(vhost->work_thread); fc_remove_host(vhost->host); scsi_remove_host(vhost->host); @@ -5408,7 +5406,7 @@ static int ibmvfc_remove(struct vio_dev *vdev) list_splice_init(&vhost->purge, &purge); spin_unlock_irqrestore(vhost->host->host_lock, flags); ibmvfc_complete_purge(&purge); - ibmvfc_free_event_pool(vhost, &vhost->crq); + ibmvfc_release_crq_queue(vhost); ibmvfc_free_mem(vhost); spin_lock(&ibmvfc_driver_lock); From patchwork Thu Jan 14 20:31:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363246 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1C25C4332D for ; Thu, 14 Jan 2021 20:35:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71BF323A9A for ; Thu, 14 Jan 2021 20:35:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730402AbhANUf1 (ORCPT ); Thu, 14 Jan 2021 15:35:27 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:8792 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726049AbhANUfM (ORCPT ); Thu, 14 Jan 2021 15:35:12 -0500 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK3I3x055568; Thu, 14 Jan 2021 15:31:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=s7uIzYnvjJVpUj7SPxdNdB2CeheK+3/3guDKAgZ1/JU=; b=iyq/aw891Uxv59iJkQ0vcBCwqk82Gye5EQlFR6RKuKymwuasXHK1dCs//CPfNIgR+VoU u3vfT2FbbBWMdx74fQQf+NRVblQws6Ls1ijDi/8/i53OcMnD09AqynH6pbmXh+aLtfqW wBUjTbkV1YZDkWdjRp0z1u1Bt80wvoKYuEib6xVyxNh9NUJJ4g4s+8WrDRljvAs0xH9T XPpGQeqVADgLqf65WKsPATpjuX+yT6T5F8CmCFVDdN8vz9say7BfTsp7x+QDGemeUGcs Yyd2DJr5jW31NFe40ckTIeMjyZO04Orq/8BDZQdZHBqpwtE65KlVs7KtT21WkvtbCP+n fg== Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vca94cb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:54 -0500 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKSHEG007288; Thu, 14 Jan 2021 20:31:53 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma05wdc.us.ibm.com with ESMTP id 362cwmdsqf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:53 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVqk927394382 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:52 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 009836E053; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A27DC6E052; Thu, 14 Jan 2021 20:31:51 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:51 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 04/21] ibmvfc: add size parameter to ibmvfc_init_event_pool Date: Thu, 14 Jan 2021 14:31:31 -0600 Message-Id: <20210114203148.246656-5-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 spamscore=0 clxscore=1015 malwarescore=0 priorityscore=1501 impostorscore=0 mlxscore=0 phishscore=0 adultscore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org With the upcoming addition of Sub-CRQs the event pool size may vary per-queue. Add a size parameter to ibmvfc_init_event_pool such that different size event pools can be requested by ibmvfc_alloc_queue. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 9330f5a65a7e..524e81164d70 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -723,19 +723,23 @@ static int ibmvfc_send_crq_init_complete(struct ibmvfc_host *vhost) * Returns zero on success. **/ static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, - struct ibmvfc_queue *queue) + struct ibmvfc_queue *queue, + unsigned int size) { int i; struct ibmvfc_event_pool *pool = &queue->evt_pool; ENTER; - pool->size = max_requests + IBMVFC_NUM_INTERNAL_REQ; - pool->events = kcalloc(pool->size, sizeof(*pool->events), GFP_KERNEL); + if (!size) + return 0; + + pool->size = size; + pool->events = kcalloc(size, sizeof(*pool->events), GFP_KERNEL); if (!pool->events) return -ENOMEM; pool->iu_storage = dma_alloc_coherent(vhost->dev, - pool->size * sizeof(*pool->iu_storage), + size * sizeof(*pool->iu_storage), &pool->iu_token, 0); if (!pool->iu_storage) { @@ -747,7 +751,7 @@ static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, INIT_LIST_HEAD(&queue->free); spin_lock_init(&queue->l_lock); - for (i = 0; i < pool->size; ++i) { + for (i = 0; i < size; ++i) { struct ibmvfc_event *evt = &pool->events[i]; atomic_set(&evt->free, 1); @@ -5013,6 +5017,7 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, { struct device *dev = vhost->dev; size_t fmt_size; + unsigned int pool_size = 0; ENTER; spin_lock_init(&queue->_lock); @@ -5021,10 +5026,7 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, switch (fmt) { case IBMVFC_CRQ_FMT: fmt_size = sizeof(*queue->msgs.crq); - if (ibmvfc_init_event_pool(vhost, queue)) { - dev_err(dev, "Couldn't initialize event pool.\n"); - return -ENOMEM; - } + pool_size = max_requests + IBMVFC_NUM_INTERNAL_REQ; break; case IBMVFC_ASYNC_FMT: fmt_size = sizeof(*queue->msgs.async); @@ -5034,6 +5036,11 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, return -EINVAL; } + if (ibmvfc_init_event_pool(vhost, queue, pool_size)) { + dev_err(dev, "Couldn't initialize event pool.\n"); + return -ENOMEM; + } + queue->msgs.handle = (void *)get_zeroed_page(GFP_KERNEL); if (!queue->msgs.handle) return -ENOMEM; From patchwork Thu Jan 14 20:31:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A8FBC43217 for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBF3E23A5E for ; Thu, 14 Jan 2021 20:34:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727300AbhANUdx (ORCPT ); Thu, 14 Jan 2021 15:33:53 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:61900 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727092AbhANUcm (ORCPT ); Thu, 14 Jan 2021 15:32:42 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKVjRX036399; Thu, 14 Jan 2021 15:31:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=nKk1Um+B2/FZbZZiiN+/A8qPSztGvj/zmOxIlgLrIkE=; b=DTeRt6Kxay4pdWDRTekxaER5ytzAeX1TmVEG3xxLwh6VgVfd6cKf3Yau6yCBQJTQO9pz nzbgGO57FrALBqkCEVmQtbB8xeZ8o86MrKwOFl4rUtlsrp7e6sEYgOZ5Mi0Qgc4kvX8B JgYc0gZmkb9+ru7BDSjAAmS9yW5BZE7KW0jlF8eCZUaAASUaycExx7xcDRHn58a6XUNB BcyzCzKuokxKnMOA5yNHTVbttLP5zlim2Jg8Tym8xefpZPTpd/5sYzSmwgz3Cqt/dZkR fp4+M6HR5mUji0U9fpKfeKCdlIxUgu1Nfx1VAJaj3ESLgicZ5Q9FDTxPuVqwVAW0aBfC LQ== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vbj94rp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:55 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKS7fS007035; Thu, 14 Jan 2021 20:31:54 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma01dal.us.ibm.com with ESMTP id 35y449sr0f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:54 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVqLK23265688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:52 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 628A16E05F; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0F98D6E052; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:51 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 05/21] ibmvfc: define hcall wrapper for registering a Sub-CRQ Date: Thu, 14 Jan 2021 14:31:32 -0600 Message-Id: <20210114203148.246656-6-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 bulkscore=0 suspectscore=0 spamscore=0 impostorscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Sub-CRQs are registred with firmware via a hypercall. Abstract that interface into a simpler helper function. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 524e81164d70..612c7f3d7bd3 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -138,6 +138,20 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *); static const char *unknown_error = "unknown error"; +static long h_reg_sub_crq(unsigned long unit_address, unsigned long ioba, + unsigned long length, unsigned long *cookie, + unsigned long *irq) +{ + unsigned long retbuf[PLPAR_HCALL_BUFSIZE]; + long rc; + + rc = plpar_hcall(H_REG_SUB_CRQ, retbuf, unit_address, ioba, length); + *cookie = retbuf[0]; + *irq = retbuf[1]; + + return rc; +} + static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags) { u64 host_caps = be64_to_cpu(vhost->login_buf->resp.capabilities); From patchwork Thu Jan 14 20:31:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9648C43603 for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 81EE523A7C for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726897AbhANUcl (ORCPT ); Thu, 14 Jan 2021 15:32:41 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:11574 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726058AbhANUck (ORCPT ); Thu, 14 Jan 2021 15:32:40 -0500 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKSQI9153407; Thu, 14 Jan 2021 15:31:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=TGs63mVFywZBnj3AKF8wn2z5azFR5i+RSJ9O5mdxhAY=; b=RHe7J+pU6cEpHQJ+IlvRr5YRpQk7t5H2fwK93HyK4YMQi4IPrRAE2aXgEKCRCwRX7WYr CUh6OSx0lHgMYJSWzO0e/jyDnSWhYWxK2BJAan/7mzmpF22nAxTobv+Gc9SLztKUhFUe +ivtEyzHv4ihAQXDA2zUgxPS73wUdVuxrXi6WABGbeHqCD6mFUkHlyqsyYW25i2A+GhZ Nmi9wl3Rg6i4vQ5unlpn/lFqngs57aZP/iVApRoJ+WfAAOV9LBe6Ib0ZMvyzjt7CJnIG WntF0FPBkh5uBDo7xP5IgoM6rXv/xtyXJ9z36sVXOhugDob9PdaZ+7fp9YxHj8eiRhWo xg== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 362w19r1yg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:55 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKHF38004680; Thu, 14 Jan 2021 20:31:54 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma02wdc.us.ibm.com with ESMTP id 35y449fry9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:54 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVqD925428354 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:52 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C378B6E04E; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7B1916E052; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 06/21] ibmvfc: add Subordinate CRQ definitions Date: Thu, 14 Jan 2021 14:31:33 -0600 Message-Id: <20210114203148.246656-7-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 clxscore=1015 adultscore=0 phishscore=0 malwarescore=0 bulkscore=0 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140111 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Subordinate Command Response Queues (Sub CRQ) are used in conjunction with the primary CRQ when more than one queue is needed by the virtual IO adapter. Recent phyp firmware versions support Sub CRQ's with ibmvfc adapters. This feature is a prerequisite for supporting multiple hardware backed submission queues in the vfc adapter. The Sub CRQ command element differs from the standard CRQ in that it is 32bytes long as opposed to 16bytes for the latter. Despite this extra 16bytes the ibmvfc protocol will use the original CRQ command element mapped to the first 16bytes of the Sub CRQ element initially. Add definitions for the Sub CRQ command element and queue. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index dd6d89292867..b9eed05c165f 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -650,6 +650,11 @@ struct ibmvfc_crq { volatile __be64 ioba; } __packed __aligned(8); +struct ibmvfc_sub_crq { + struct ibmvfc_crq crq; + __be64 reserved[2]; +} __packed __aligned(8); + enum ibmvfc_ae_link_state { IBMVFC_AE_LS_LINK_UP = 0x01, IBMVFC_AE_LS_LINK_BOUNCED = 0x02, @@ -761,12 +766,14 @@ struct ibmvfc_event_pool { enum ibmvfc_msg_fmt { IBMVFC_CRQ_FMT = 0, IBMVFC_ASYNC_FMT, + IBMVFC_SUB_CRQ_FMT, }; union ibmvfc_msgs { void *handle; struct ibmvfc_crq *crq; struct ibmvfc_async_crq *async; + struct ibmvfc_sub_crq *scrq; }; struct ibmvfc_queue { @@ -781,6 +788,20 @@ struct ibmvfc_queue { struct list_head sent; struct list_head free; spinlock_t l_lock; + + /* Sub-CRQ fields */ + struct ibmvfc_host *vhost; + unsigned long cookie; + unsigned long vios_cookie; + unsigned long hw_irq; + unsigned long irq; + unsigned long hwq_id; + char name[32]; +}; + +struct ibmvfc_scsi_channels { + struct ibmvfc_queue *scrqs; + unsigned int active_queues; }; enum ibmvfc_host_action { From patchwork Thu Jan 14 20:31:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2217C433DB for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A9B0D23A7E for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729481AbhANUeH (ORCPT ); Thu, 14 Jan 2021 15:34:07 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:13374 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726262AbhANUcl (ORCPT ); Thu, 14 Jan 2021 15:32:41 -0500 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK2D8j069176; Thu, 14 Jan 2021 15:31:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=9rCZsdpWTuc9zRargT+2e60gHxzDGTdQNT6kzrtC5v0=; b=ip77GuWC4QLW4ZMEOqizsn6NO30SRfZ0E4hZugLHnSt80dD4XPdDGYGF4B1+VXOagv5r 53AsoWnf8ukberKEV4tk9c+2RGRDsHnEb+qpPU17CfHu4/XD5ZioiTJdcNVyNyb9nj9u 13NXqf+dWfN35lkFUwQoFYlJ4+Fhk//E2zRvX9hgkZ45TIc27tXZZGnLJuvrcvh5aNGW YPR0Z+b9OEgTdbnZC1dTzV+kZFkv1ah9XdDJXVpIzLRqaALpPNHuHrwWO571VhOaE7d8 UDGG/TM5d4V82YnxFNJ35R+P77N7TgrcbUB7tbX7hXhF5I05pSa0wabKGnRgvY4WDKat zQ== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 362ucvjutf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:55 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKS7Yc026129; Thu, 14 Jan 2021 20:31:54 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma03dal.us.ibm.com with ESMTP id 35y449sne3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:54 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVr1423527776 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:53 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3AA0E6E04E; Thu, 14 Jan 2021 20:31:53 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC1DE6E04C; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:52 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 07/21] ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels Date: Thu, 14 Jan 2021 14:31:34 -0600 Message-Id: <20210114203148.246656-8-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 clxscore=1015 lowpriorityscore=0 adultscore=0 phishscore=0 impostorscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140111 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Allocate a set of Sub-CRQs in advance. During channel setup the client and VIOS negotiate the number of queues the VIOS supports and the number that the client desires to request. Its possible that the final channel resources allocated is less than requested, but the client is still responsible for sending handles for every queue it is hoping for. Also, provide deallocation cleanup routines. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 125 +++++++++++++++++++++++++++++++++ drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 126 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 612c7f3d7bd3..a198e118887d 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -895,6 +895,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) unsigned long flags; struct vio_dev *vdev = to_vio_dev(vhost->dev); struct ibmvfc_queue *crq = &vhost->crq; + struct ibmvfc_queue *scrq; + int i; /* Close the CRQ */ do { @@ -912,6 +914,16 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) memset(crq->msgs.crq, 0, PAGE_SIZE); crq->cur = 0; + if (vhost->scsi_scrqs.scrqs) { + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + scrq = &vhost->scsi_scrqs.scrqs[i]; + spin_lock(scrq->q_lock); + memset(scrq->msgs.scrq, 0, PAGE_SIZE); + scrq->cur = 0; + spin_unlock(scrq->q_lock); + } + } + /* And re-open it again */ rc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address, crq->msg_token, PAGE_SIZE); @@ -5045,6 +5057,11 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, case IBMVFC_ASYNC_FMT: fmt_size = sizeof(*queue->msgs.async); break; + case IBMVFC_SUB_CRQ_FMT: + fmt_size = sizeof(*queue->msgs.scrq); + /* We need one extra event for Cancel Commands */ + pool_size = max_requests + 1; + break; default: dev_warn(dev, "Unknown command/response queue message format: %d\n", fmt); return -EINVAL; @@ -5136,6 +5153,107 @@ static int ibmvfc_init_crq(struct ibmvfc_host *vhost) return retrc; } +static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, + int index) +{ + struct device *dev = vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + struct ibmvfc_queue *scrq = &vhost->scsi_scrqs.scrqs[index]; + int rc = -ENOMEM; + + ENTER; + + if (ibmvfc_alloc_queue(vhost, scrq, IBMVFC_SUB_CRQ_FMT)) + return -ENOMEM; + + rc = h_reg_sub_crq(vdev->unit_address, scrq->msg_token, PAGE_SIZE, + &scrq->cookie, &scrq->hw_irq); + + if (rc) { + dev_warn(dev, "Error registering sub-crq: %d\n", rc); + if (rc == H_PARAMETER) + dev_warn_once(dev, "Firmware may not support MQ\n"); + goto reg_failed; + } + + scrq->hwq_id = index; + scrq->vhost = vhost; + + LEAVE; + return 0; + +reg_failed: + ibmvfc_free_queue(vhost, scrq); + LEAVE; + return rc; +} + +static void ibmvfc_deregister_scsi_channel(struct ibmvfc_host *vhost, int index) +{ + struct device *dev = vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + struct ibmvfc_queue *scrq = &vhost->scsi_scrqs.scrqs[index]; + long rc; + + ENTER; + + do { + rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, + scrq->cookie); + } while (rc == H_BUSY || H_IS_LONG_BUSY(rc)); + + if (rc) + dev_err(dev, "Failed to free sub-crq[%d]: rc=%ld\n", index, rc); + + ibmvfc_free_queue(vhost, scrq); + LEAVE; +} + +static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) +{ + int i, j; + + ENTER; + + vhost->scsi_scrqs.scrqs = kcalloc(IBMVFC_SCSI_HW_QUEUES, + sizeof(*vhost->scsi_scrqs.scrqs), + GFP_KERNEL); + if (!vhost->scsi_scrqs.scrqs) + return -1; + + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + if (ibmvfc_register_scsi_channel(vhost, i)) { + for (j = i; j > 0; j--) + ibmvfc_deregister_scsi_channel(vhost, j - 1); + kfree(vhost->scsi_scrqs.scrqs); + vhost->scsi_scrqs.scrqs = NULL; + vhost->scsi_scrqs.active_queues = 0; + LEAVE; + return -1; + } + } + + LEAVE; + return 0; +} + +static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) +{ + int i; + + ENTER; + if (!vhost->scsi_scrqs.scrqs) + return; + + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) + ibmvfc_deregister_scsi_channel(vhost, i); + + kfree(vhost->scsi_scrqs.scrqs); + vhost->scsi_scrqs.scrqs = NULL; + vhost->scsi_scrqs.active_queues = 0; + LEAVE; +} + /** * ibmvfc_free_mem - Free memory for vhost * @vhost: ibmvfc host struct @@ -5371,6 +5489,12 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) goto remove_shost; } + if (vhost->mq_enabled) { + rc = ibmvfc_init_sub_crqs(vhost); + if (rc) + dev_warn(dev, "Failed to allocate Sub-CRQs. rc=%d\n", rc); + } + if (shost_to_fc_host(shost)->rqst_q) blk_queue_max_segments(shost_to_fc_host(shost)->rqst_q, 1); dev_set_drvdata(dev, vhost); @@ -5427,6 +5551,7 @@ static int ibmvfc_remove(struct vio_dev *vdev) list_splice_init(&vhost->purge, &purge); spin_unlock_irqrestore(vhost->host->host_lock, flags); ibmvfc_complete_purge(&purge); + ibmvfc_release_sub_crqs(vhost); ibmvfc_release_crq_queue(vhost); ibmvfc_free_mem(vhost); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index b9eed05c165f..bdafe9956649 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -850,6 +850,7 @@ struct ibmvfc_host { mempool_t *tgt_pool; struct ibmvfc_queue crq; struct ibmvfc_queue async_crq; + struct ibmvfc_scsi_channels scsi_scrqs; struct ibmvfc_npiv_login login_info; union ibmvfc_npiv_login_data *login_buf; dma_addr_t login_buf_dma; From patchwork Thu Jan 14 20:31:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1628C433E0 for ; Thu, 14 Jan 2021 20:34:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9534923A79 for ; Thu, 14 Jan 2021 20:34:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727319AbhANUcm (ORCPT ); Thu, 14 Jan 2021 15:32:42 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:16098 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726266AbhANUcl (ORCPT ); Thu, 14 Jan 2021 15:32:41 -0500 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK3JQB056365; Thu, 14 Jan 2021 15:31:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=xfieu9zl98sOWzs8UXpok0I8NH8GzGhj/l73ksvoGzI=; b=qkQpNQ3+TVsX5DMn4iriNURVRmJ983awnsDf7+7v346qOF5/RK/gMqLkJIxOqhYKVBbU kHs8rCJzpHecjcPIvzLBz3c6dd+38YDRSrXN+B3A8WE5O4WfKuAPXh4J12GNoMO8f98h coe9C1/7wh1fR11GkLgmZ8GfykuPULEniVeLiDx7sIbxR+ydGYbYF0dMwEMEQBO/4R7n QvYKe3xzSUogrVpcQ4t3lCHFDE+MYtQC2Mqu+1YnCm7wtZEd/RakiOQz+w2M7tyR/KGQ JsA8zLhndZoSknWSwau1SvY436kGVl3QL7dMV4Oy++yNWGf9Gx+nBYO6lnUPFUsz/C24 aA== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vca94d8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:56 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKS7UX006995; Thu, 14 Jan 2021 20:31:55 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma01dal.us.ibm.com with ESMTP id 35y449sr0j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:55 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVrrt13697424 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:53 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A52BF6E05D; Thu, 14 Jan 2021 20:31:53 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4940C6E052; Thu, 14 Jan 2021 20:31:53 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:53 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 08/21] ibmvfc: add Sub-CRQ IRQ enable/disable routine Date: Thu, 14 Jan 2021 14:31:35 -0600 Message-Id: <20210114203148.246656-9-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 spamscore=0 clxscore=1015 malwarescore=0 priorityscore=1501 impostorscore=0 mlxscore=0 phishscore=0 adultscore=0 suspectscore=0 bulkscore=0 mlxlogscore=853 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Each Sub-CRQ has its own interrupt. A hypercall is required to toggle the IRQ state. Provide the necessary mechanism via a helper function. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index a198e118887d..5d7ada0ed0d6 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3465,6 +3465,26 @@ static void ibmvfc_tasklet(void *data) } } +static int ibmvfc_toggle_scrq_irq(struct ibmvfc_queue *scrq, int enable) +{ + struct device *dev = scrq->vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + unsigned long rc; + int irq_action = H_ENABLE_VIO_INTERRUPT; + + if (!enable) + irq_action = H_DISABLE_VIO_INTERRUPT; + + rc = plpar_hcall_norets(H_VIOCTL, vdev->unit_address, irq_action, + scrq->hw_irq, 0, 0); + + if (rc) + dev_err(dev, "Couldn't %s sub-crq[%lu] irq. rc=%ld\n", + enable ? "enable" : "disable", scrq->hwq_id, rc); + + return rc; +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Thu Jan 14 20:31:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00935C43333 for ; Thu, 14 Jan 2021 20:34:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AF26423B09 for ; Thu, 14 Jan 2021 20:34:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729220AbhANUdx (ORCPT ); Thu, 14 Jan 2021 15:33:53 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:21854 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727056AbhANUcm (ORCPT ); Thu, 14 Jan 2021 15:32:42 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK2JWD115719; Thu, 14 Jan 2021 15:31:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Tp7Apfjr247x8ESCxwtc0OwFyKGd4qn1wU74cmHH1kg=; b=E8Sc1OR4p8MCWVZSTBWv5UU2+Yb1oJXYmjwfqnlh+KTEqv/CQbt6HVLz70aXlty6tMYq fVKzrxFZpzf6EbwYit69wrQAf+XC+jEjHOPQircdsHnIl8I/dwZzpkZMFFQKc1iqOr8/ UK0rxzxpcFQPokgBkCEmAkcGzOE1A3uWnGozmUXpiE1BJIGIk2W1973+bHLTN7z4HwiS RBgEgb8QcfVagW8eWsNvcBRG5kg0raPHjibHuzqlgUaTcBM7MgcVV7e/X/PEQ+P1SJSA ef0SZMuOGuzHRLiRz8r3IfTMGAM1sV4LL/zDFeF+rYJ5HqfpAFAHXjb9mUWlqS09LZbG sw== Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vf4900x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:56 -0500 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKRP43032079; Thu, 14 Jan 2021 20:31:55 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma04wdc.us.ibm.com with ESMTP id 35y449fsk6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:55 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVsT915466844 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:54 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 13BB66E054; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B48556E04E; Thu, 14 Jan 2021 20:31:53 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:53 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 09/21] ibmvfc: add handlers to drain and complete Sub-CRQ responses Date: Thu, 14 Jan 2021 14:31:36 -0600 Message-Id: <20210114203148.246656-10-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 malwarescore=0 impostorscore=0 mlxlogscore=999 spamscore=0 mlxscore=0 clxscore=1015 bulkscore=0 priorityscore=1501 adultscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140111 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The logic for iterating over the Sub-CRQ responses is similiar to that of the primary CRQ. Add the necessary handlers for processing those responses. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 86 ++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 5d7ada0ed0d6..f3cd092478ee 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3485,6 +3485,92 @@ static int ibmvfc_toggle_scrq_irq(struct ibmvfc_queue *scrq, int enable) return rc; } +static void ibmvfc_handle_scrq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost, + struct list_head *evt_doneq) +{ + struct ibmvfc_event *evt = (struct ibmvfc_event *)be64_to_cpu(crq->ioba); + + switch (crq->valid) { + case IBMVFC_CRQ_CMD_RSP: + break; + case IBMVFC_CRQ_XPORT_EVENT: + return; + default: + dev_err(vhost->dev, "Got and invalid message type 0x%02x\n", crq->valid); + return; + } + + /* The only kind of payload CRQs we should get are responses to + * things we send. Make sure this response is to something we + * actually sent + */ + if (unlikely(!ibmvfc_valid_event(&evt->queue->evt_pool, evt))) { + dev_err(vhost->dev, "Returned correlation_token 0x%08llx is invalid!\n", + crq->ioba); + return; + } + + if (unlikely(atomic_read(&evt->free))) { + dev_err(vhost->dev, "Received duplicate correlation_token 0x%08llx!\n", + crq->ioba); + return; + } + + spin_lock(&evt->queue->l_lock); + list_move_tail(&evt->queue_list, evt_doneq); + spin_unlock(&evt->queue->l_lock); +} + +static struct ibmvfc_crq *ibmvfc_next_scrq(struct ibmvfc_queue *scrq) +{ + struct ibmvfc_crq *crq; + + crq = &scrq->msgs.scrq[scrq->cur].crq; + if (crq->valid & 0x80) { + if (++scrq->cur == scrq->size) + scrq->cur = 0; + rmb(); + } else + crq = NULL; + + return crq; +} + +static void ibmvfc_drain_sub_crq(struct ibmvfc_queue *scrq) +{ + struct ibmvfc_crq *crq; + struct ibmvfc_event *evt, *temp; + unsigned long flags; + int done = 0; + LIST_HEAD(evt_doneq); + + spin_lock_irqsave(scrq->q_lock, flags); + while (!done) { + while ((crq = ibmvfc_next_scrq(scrq)) != NULL) { + ibmvfc_handle_scrq(crq, scrq->vhost, &evt_doneq); + crq->valid = 0; + wmb(); + } + + ibmvfc_toggle_scrq_irq(scrq, 1); + if ((crq = ibmvfc_next_scrq(scrq)) != NULL) { + ibmvfc_toggle_scrq_irq(scrq, 0); + ibmvfc_handle_scrq(crq, scrq->vhost, &evt_doneq); + crq->valid = 0; + wmb(); + } else + done = 1; + } + spin_unlock_irqrestore(scrq->q_lock, flags); + + list_for_each_entry_safe(evt, temp, &evt_doneq, queue_list) { + del_timer(&evt->timer); + list_del(&evt->queue_list); + ibmvfc_trc_end(evt); + evt->done(evt); + } +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Thu Jan 14 20:31:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364246 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8066FC4332E for ; Thu, 14 Jan 2021 20:35:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 59FD823A7E for ; Thu, 14 Jan 2021 20:35:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730331AbhANUfW (ORCPT ); Thu, 14 Jan 2021 15:35:22 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:31014 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730143AbhANUfP (ORCPT ); Thu, 14 Jan 2021 15:35:15 -0500 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKW9S1086990; Thu, 14 Jan 2021 15:32:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=FPqU0gK+vFLag/1Y+iJtqWIyGFSfnPgAKvI4hLlWICc=; b=WM3V+kW5CrtRz+Vmuz95KqQvKY7anXd/e3z8vGyTnmgvFy+Wp1JIyRIV4c6qfkXNN8wx lQ3EybjUEjODahR5N4yB7EjJ2cMUNU/TlDMTNXO60XcjcbdZH3tiOzTDEK/ViWAVmpKs od7DyALSCkwF16V1ZKhwMj4tx+5WKn7KOv2qJ9f8tKu5Y8qSu5HZxc2ggK9JnlVabXfK B43wiwgQqi8s6ilZ9K6Zs6Ux8IYVysrJEsjvl/YaL0Nzkp+gx2ipzqQjMUh16a4RAVxj PD4ye3dlt6qlmB/8GwJaIyua6SFaWACO4Iw92f+0GCzSxscaJYAxqV8tTB4H1m34wvsh KQ== Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0b-001b2d01.pphosted.com with ESMTP id 362v01hsjs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:32:14 -0500 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKSB69015079; Thu, 14 Jan 2021 20:31:56 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma02dal.us.ibm.com with ESMTP id 35y44a1q39-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:56 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVspl23200068 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:54 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7F2316E054; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 229046E04E; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 10/21] ibmvfc: define Sub-CRQ interrupt handler routine Date: Thu, 14 Jan 2021 14:31:37 -0600 Message-Id: <20210114203148.246656-11-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=999 mlxscore=0 priorityscore=1501 adultscore=0 clxscore=1015 bulkscore=0 lowpriorityscore=0 impostorscore=0 malwarescore=0 spamscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Simple handler that calls Sub-CRQ drain routine directly. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index f3cd092478ee..51bcafad9490 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3571,6 +3571,16 @@ static void ibmvfc_drain_sub_crq(struct ibmvfc_queue *scrq) } } +static irqreturn_t ibmvfc_interrupt_scsi(int irq, void *scrq_instance) +{ + struct ibmvfc_queue *scrq = (struct ibmvfc_queue *)scrq_instance; + + ibmvfc_toggle_scrq_irq(scrq, 0); + ibmvfc_drain_sub_crq(scrq); + + return IRQ_HANDLED; +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Thu Jan 14 20:31:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59129C4332D for ; Thu, 14 Jan 2021 20:34:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1CD5323A7C for ; Thu, 14 Jan 2021 20:34:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729084AbhANUdp (ORCPT ); Thu, 14 Jan 2021 15:33:45 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:49384 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727300AbhANUcm (ORCPT ); Thu, 14 Jan 2021 15:32:42 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK2sfj051410; Thu, 14 Jan 2021 15:31:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=C19kqcusT6BaZ+vFfyqwz/QhfC+/ifYdrrhKkXGhhfE=; b=LPIMhqy5fh9oKtV9SLjplYmVZqRghbl2xAg2VzlfevyTGcB33IvrPeieYlCBOw/gz+st BfgEpofgApTTMLh8I85vEhkHBhYBiJ5yzK57+KfARiTvrIaRD+wwV09y0lIp3/2oehDe bvQw59a/FQ3bjfX4jdXokAtcSKsqmPc11IF+vckacyVBCM/anSitrNTjTaJor71xUYG8 V8lUmcvwZjqaE+e1Db8Zt8o1OtF7fbwqU0kmUjnRqPxJ7Hwmzs4pwAOiwsi7/ZxXE2vM qIumGg56r5fJCJLso6Kv120+dKjuNND8zk8CBYVI3eSC+mWz1+dOFTfvF8mEE8g8Agx5 ZQ== Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 362v29sggg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:57 -0500 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKSMXd007313; Thu, 14 Jan 2021 20:31:56 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma05wdc.us.ibm.com with ESMTP id 362cwmdsqt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:56 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVsII20447722 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:55 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E0FE56E04E; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8E93D6E058; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 11/21] ibmvfc: map/request irq and register Sub-CRQ interrupt handler Date: Thu, 14 Jan 2021 14:31:38 -0600 Message-Id: <20210114203148.246656-12-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0 priorityscore=1501 mlxlogscore=999 malwarescore=0 impostorscore=0 spamscore=0 clxscore=1015 bulkscore=0 adultscore=0 phishscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Create an irq mapping for the hw_irq number provided from phyp firmware. Request an irq assigned our Sub-CRQ interrupt handler. Unmap these irqs at Sub-CRQ teardown. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 51bcafad9490..d3d7c6b53d4f 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -5292,12 +5292,34 @@ static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, goto reg_failed; } + scrq->irq = irq_create_mapping(NULL, scrq->hw_irq); + + if (!scrq->irq) { + rc = -EINVAL; + dev_err(dev, "Error mapping sub-crq[%d] irq\n", index); + goto irq_failed; + } + + snprintf(scrq->name, sizeof(scrq->name), "ibmvfc-%x-scsi%d", + vdev->unit_address, index); + rc = request_irq(scrq->irq, ibmvfc_interrupt_scsi, 0, scrq->name, scrq); + + if (rc) { + dev_err(dev, "Couldn't register sub-crq[%d] irq\n", index); + irq_dispose_mapping(scrq->irq); + goto irq_failed; + } + scrq->hwq_id = index; scrq->vhost = vhost; LEAVE; return 0; +irq_failed: + do { + plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, scrq->cookie); + } while (rc == H_BUSY || H_IS_LONG_BUSY(rc)); reg_failed: ibmvfc_free_queue(vhost, scrq); LEAVE; @@ -5313,6 +5335,9 @@ static void ibmvfc_deregister_scsi_channel(struct ibmvfc_host *vhost, int index) ENTER; + free_irq(scrq->irq, scrq); + irq_dispose_mapping(scrq->irq); + do { rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, scrq->cookie); From patchwork Thu Jan 14 20:31:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3AF5C433E0 for ; Thu, 14 Jan 2021 20:33:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9B3E823A5F for ; Thu, 14 Jan 2021 20:33:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728134AbhANUcq (ORCPT ); Thu, 14 Jan 2021 15:32:46 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:48542 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727954AbhANUcp (ORCPT ); Thu, 14 Jan 2021 15:32:45 -0500 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKKnAP186082; Thu, 14 Jan 2021 15:31:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=l431NZQBSk+btUL8OiC0lBjrprIvUfWPsnRrx67uZ3g=; b=HY4RpHhBJN2zRnZ7PE6O0aj4pvEqTQm0RnqzOUUa3EpuJRFMOiKtO53M5vPt28X9aiOt GTNUbdfvK38ZYYT8tcpCteyuJTbzzMbKzg4anyLyZdm9k5bW+0VPbRMPwNqP7mBGA2zK 8ApYLN3f+kKg0MrjNQrC64yfylU1Lq8xp6AIW0rjWgEA9Dt8Dwkj4oBjbAYsrh3xtex+ gDfYwUCoZ+QMwEqittul21WyCXFtNfla608IQfZ3c5nJhexy5XOFqegGC1UIGLwzRgYb TIldi0w9Pz8xruBYh7bK3gIkzhjg4dhckSr6/Kow/19xJpvdppBmtSa3zJwAFTyNSsfu 9Q== Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vwr07db-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:58 -0500 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKRSh4013849; Thu, 14 Jan 2021 20:31:56 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma03wdc.us.ibm.com with ESMTP id 35y449fsv2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:56 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVtLN10158564 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:55 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4FF7D6E04C; Thu, 14 Jan 2021 20:31:55 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F07A16E050; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:54 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 12/21] ibmvfc: implement channel enquiry and setup commands Date: Thu, 14 Jan 2021 14:31:39 -0600 Message-Id: <20210114203148.246656-13-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 phishscore=0 lowpriorityscore=0 clxscore=1015 malwarescore=0 mlxlogscore=999 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org New NPIV_ENQUIRY_CHANNEL and NPIV_SETUP_CHANNEL management datagrams (MADs) were defined in a previous patchset. If the client advertises a desire to use channels and the partner VIOS is channel capable then the client must proceed with channel enquiry to determine the maximum number of channels the VIOS is capable of providing, and registering SubCRQs via channel setup with the VIOS immediately following NPIV Login. This handshaking should not be performed for subsequent NPIV Logins unless the CRQ connection has been reset. Implement these two new MADs and issue them following a successful NPIV login where the VIOS has set the SUPPORT_CHANNELS capability bit in the NPIV Login response. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 135 ++++++++++++++++++++++++++++++++- drivers/scsi/ibmvscsi/ibmvfc.h | 3 + 2 files changed, 136 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index d3d7c6b53d4f..a00f38558613 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -909,6 +909,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) spin_lock(vhost->crq.q_lock); vhost->state = IBMVFC_NO_CRQ; vhost->logged_in = 0; + vhost->do_enquiry = 1; + vhost->using_channels = 0; /* Clean out the queue */ memset(crq->msgs.crq, 0, PAGE_SIZE); @@ -4586,6 +4588,118 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost) ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); } +static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) +{ + struct ibmvfc_host *vhost = evt->vhost; + u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); + int level = IBMVFC_DEFAULT_LOG_LEVEL; + + ibmvfc_free_event(evt); + + switch (mad_status) { + case IBMVFC_MAD_SUCCESS: + ibmvfc_dbg(vhost, "Channel Setup succeded\n"); + vhost->do_enquiry = 0; + break; + case IBMVFC_MAD_FAILED: + level += ibmvfc_retry_host_init(vhost); + ibmvfc_log(vhost, level, "Channel Setup failed\n"); + fallthrough; + case IBMVFC_MAD_DRIVER_FAILED: + return; + default: + dev_err(vhost->dev, "Invalid Channel Setup response: 0x%x\n", + mad_status); + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); + return; + } + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); + wake_up(&vhost->work_wait_q); +} + +static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) +{ + struct ibmvfc_channel_setup_mad *mad; + struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf; + struct ibmvfc_event *evt = ibmvfc_get_event(&vhost->crq); + + memset(setup_buf, 0, sizeof(*setup_buf)); + setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + + ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); + mad = &evt->iu.channel_setup; + memset(mad, 0, sizeof(*mad)); + mad->common.version = cpu_to_be32(1); + mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_SETUP); + mad->common.length = cpu_to_be16(sizeof(*mad)); + mad->buffer.va = cpu_to_be64(vhost->channel_setup_dma); + mad->buffer.len = cpu_to_be32(sizeof(*vhost->channel_setup_buf)); + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); + + if (!ibmvfc_send_event(evt, vhost, default_timeout)) + ibmvfc_dbg(vhost, "Sent channel setup\n"); + else + ibmvfc_link_down(vhost, IBMVFC_LINK_DOWN); +} + +static void ibmvfc_channel_enquiry_done(struct ibmvfc_event *evt) +{ + struct ibmvfc_host *vhost = evt->vhost; + struct ibmvfc_channel_enquiry *rsp = &evt->xfer_iu->channel_enquiry; + u32 mad_status = be16_to_cpu(rsp->common.status); + int level = IBMVFC_DEFAULT_LOG_LEVEL; + + switch (mad_status) { + case IBMVFC_MAD_SUCCESS: + ibmvfc_dbg(vhost, "Channel Enquiry succeeded\n"); + vhost->max_vios_scsi_channels = be32_to_cpu(rsp->num_scsi_subq_channels); + ibmvfc_free_event(evt); + break; + case IBMVFC_MAD_FAILED: + level += ibmvfc_retry_host_init(vhost); + ibmvfc_log(vhost, level, "Channel Enquiry failed\n"); + fallthrough; + case IBMVFC_MAD_DRIVER_FAILED: + ibmvfc_free_event(evt); + return; + default: + dev_err(vhost->dev, "Invalid Channel Enquiry response: 0x%x\n", + mad_status); + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); + ibmvfc_free_event(evt); + return; + } + + ibmvfc_channel_setup(vhost); +} + +static void ibmvfc_channel_enquiry(struct ibmvfc_host *vhost) +{ + struct ibmvfc_channel_enquiry *mad; + struct ibmvfc_event *evt = ibmvfc_get_event(&vhost->crq); + + ibmvfc_init_event(evt, ibmvfc_channel_enquiry_done, IBMVFC_MAD_FORMAT); + mad = &evt->iu.channel_enquiry; + memset(mad, 0, sizeof(*mad)); + mad->common.version = cpu_to_be32(1); + mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_ENQUIRY); + mad->common.length = cpu_to_be16(sizeof(*mad)); + + if (IBMVFC_MIG_NO_SUB_TO_CRQ) + mad->flags |= cpu_to_be32(IBMVFC_NO_CHANNELS_TO_CRQ_SUPPORT); + if (IBMVFC_MIG_NO_N_TO_M) + mad->flags |= cpu_to_be32(IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT); + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); + + if (!ibmvfc_send_event(evt, vhost, default_timeout)) + ibmvfc_dbg(vhost, "Send channel enquiry\n"); + else + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); +} + /** * ibmvfc_npiv_login_done - Completion handler for NPIV Login * @evt: ibmvfc event struct @@ -4667,8 +4781,14 @@ static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt) vhost->host->can_queue = be32_to_cpu(rsp->max_cmds) - IBMVFC_NUM_INTERNAL_REQ; vhost->host->max_sectors = npiv_max_sectors; - ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); - wake_up(&vhost->work_wait_q); + + if (ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPORT_CHANNELS) && vhost->do_enquiry) { + ibmvfc_channel_enquiry(vhost); + } else { + vhost->do_enquiry = 0; + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); + wake_up(&vhost->work_wait_q); + } } /** @@ -5477,9 +5597,20 @@ static int ibmvfc_alloc_mem(struct ibmvfc_host *vhost) goto free_trace; } + vhost->channel_setup_buf = dma_alloc_coherent(dev, sizeof(*vhost->channel_setup_buf), + &vhost->channel_setup_dma, + GFP_KERNEL); + + if (!vhost->channel_setup_buf) { + dev_err(dev, "Couldn't allocate Channel Setup buffer\n"); + goto free_tgt_pool; + } + LEAVE; return 0; +free_tgt_pool: + mempool_destroy(vhost->tgt_pool); free_trace: kfree(vhost->trace); free_disc_buffer: diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index bdafe9956649..3d76cd3c1fd9 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -854,10 +854,13 @@ struct ibmvfc_host { struct ibmvfc_npiv_login login_info; union ibmvfc_npiv_login_data *login_buf; dma_addr_t login_buf_dma; + struct ibmvfc_channel_setup *channel_setup_buf; + dma_addr_t channel_setup_dma; int disc_buf_sz; int log_level; struct ibmvfc_discover_targets_entry *disc_buf; struct mutex passthru_mutex; + int max_vios_scsi_channels; int task_set; int init_retries; int discovery_threads; From patchwork Thu Jan 14 20:31:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FB0EC43332 for ; Thu, 14 Jan 2021 20:33:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 76C7723B18 for ; Thu, 14 Jan 2021 20:33:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728221AbhANUcq (ORCPT ); Thu, 14 Jan 2021 15:32:46 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:36738 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727908AbhANUcp (ORCPT ); Thu, 14 Jan 2021 15:32:45 -0500 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK2oPB185766; Thu, 14 Jan 2021 15:31:58 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=93FWYEBzS1jw1+gJXU9Weiq+Fq0bioDeCgcTWeVwlFw=; b=rzNZeacOO2QFcCuPWh07MX2ASDgQIq80Vw7puFqF4awPTs7wgMmdfZo1PSfT7VKvq8q4 X55sfiurg9HnN9Wy/J4PRBS/xECissBIXDpJGh2+8VX/26W3zKsH5WonvDyxkdmKRZwR xHnvcYs+XAfOvSxvjl0wdGNUoIWAThpzHFtXxMsZpjYtKvPrwPKXCOKIcLwNcw3YIXqX 6Pd3L5h74h9QyIyq3JoNBihTh8ywpDiUKAVmO3R4AUFT5Fz9VxXvip0LwDaopvtoNOSk 7Kht9XdJX7nUQeIVRMsWhQNhbAdmBgM7vBcjxxYUHUukt3Jzr6hg4+ZRA0qu9pd7n49h VA== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vhhgv4h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:57 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKHFxM004677; Thu, 14 Jan 2021 20:31:57 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma02wdc.us.ibm.com with ESMTP id 35y449frye-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:57 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVtcX4850102 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:55 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B1F026E050; Thu, 14 Jan 2021 20:31:55 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5F02A6E056; Thu, 14 Jan 2021 20:31:55 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:55 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 13/21] ibmvfc: advertise client support for using hardware channels Date: Thu, 14 Jan 2021 14:31:40 -0600 Message-Id: <20210114203148.246656-14-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 priorityscore=1501 spamscore=0 bulkscore=0 clxscore=1015 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140111 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Previous patches have plumbed the necessary Sub-CRQ interface and channel negotiation MADs to fully channelize via hardware backed queues. Advertise client support via NPIV Login capability IBMVFC_CAN_USE_CHANNELS when the client bits have MQ enabled via vhost->mq_enabled, or when channels were already in use during a subsequent NPIV Login. The later is required because channel support is only renegotiated after a CRQ pair is broken. Simple NPIV Logout/Logins require the client to continue to advertise the channel capability until the CRQ pair between the client is broken. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index a00f38558613..0653d52d4ea0 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1410,6 +1410,10 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) login_info->max_cmds = cpu_to_be32(max_requests + IBMVFC_NUM_INTERNAL_REQ); login_info->capabilities = cpu_to_be64(IBMVFC_CAN_MIGRATE | IBMVFC_CAN_SEND_VF_WWPN); + + if (vhost->mq_enabled || vhost->using_channels) + login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_CHANNELS); + login_info->async.va = cpu_to_be64(vhost->async_crq.msg_token); login_info->async.len = cpu_to_be32(async_crq->size * sizeof(*async_crq->msgs.async)); From patchwork Thu Jan 14 20:31:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 013BDC433DB for ; Thu, 14 Jan 2021 20:33:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BAB9523A6A for ; Thu, 14 Jan 2021 20:33:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726523AbhANUdC (ORCPT ); Thu, 14 Jan 2021 15:33:02 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:60542 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727290AbhANUcq (ORCPT ); Thu, 14 Jan 2021 15:32:46 -0500 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKKntc186081; Thu, 14 Jan 2021 15:32:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=lqJX5KIw2JgSFcZPOaCXc/4q+kuQCWiYmoDWMGLj0/o=; b=HSd1Ubrq2c0VC6chr/TjbKvhLQFk1Tq97BTVKysMkHj9equhYgpUTZrfC9ZWPFza0C2t wmarXlO387SmuF4M/ui2diqxvOisVDFIwRiejHFsnqrw945JFTSkIB2CEeJn7040I7bq Xayb6mZ0OT/FSYDPLk1vRmZYYrMNJ4lNOOZd5V8YOrMIDEpZ02vkrhrUz3ooFHmcDlRs IHQi90s3vGzZ13DedfW7bUaDDi10Z9/aJmWMa8c2wLMOkX3lRWn/7JyVmJfHQXmRiniv zfE2uDBqVjiIY/JUmKNHG1HOJTQVMw7SDXB9qtbUTkZQbItf+bX6UGt7EXDmGV1N3iRC gw== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vwr07dt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:32:00 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKHGsK004688; Thu, 14 Jan 2021 20:31:57 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma02wdc.us.ibm.com with ESMTP id 35y449fryg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:57 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVuDQ38273384 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:56 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1FB9E6E050; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C0FAA6E052; Thu, 14 Jan 2021 20:31:55 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:55 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 14/21] ibmvfc: set and track hw queue in ibmvfc_event struct Date: Thu, 14 Jan 2021 14:31:41 -0600 Message-Id: <20210114203148.246656-15-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 phishscore=0 lowpriorityscore=0 clxscore=1015 malwarescore=0 mlxlogscore=999 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Extract the hwq id from a SCSI command and store it in the ibmvfc_event structure to identify which Sub-CRQ to send the command down when channels are being utilized. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 5 +++++ drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 0653d52d4ea0..3f3cc37a263f 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1483,6 +1483,7 @@ static void ibmvfc_init_event(struct ibmvfc_event *evt, evt->_done = done; evt->done = ibmvfc_locked_done; } + evt->hwq = 0; } /** @@ -1839,6 +1840,8 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) struct ibmvfc_cmd *vfc_cmd; struct ibmvfc_fcp_cmd_iu *iu; struct ibmvfc_event *evt; + u32 tag_and_hwq = blk_mq_unique_tag(cmnd->request); + u16 hwq = blk_mq_unique_tag_to_hwq(tag_and_hwq); int rc; if (unlikely((rc = fc_remote_port_chkready(rport))) || @@ -1865,6 +1868,8 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) } vfc_cmd->correlation = cpu_to_be64((u64)evt); + if (vhost->using_channels) + evt->hwq = hwq % vhost->scsi_scrqs.active_queues; if (likely(!(rc = ibmvfc_map_sg_data(cmnd, evt, vfc_cmd, vhost->dev)))) return ibmvfc_send_event(evt, vhost, 0); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 3d76cd3c1fd9..2dbce7071409 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -753,6 +753,7 @@ struct ibmvfc_event { struct completion comp; struct completion *eh_comp; struct timer_list timer; + u16 hwq; }; /* a pool of event structs for use */ From patchwork Thu Jan 14 20:31:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DCE0C43381 for ; Thu, 14 Jan 2021 20:35:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2CE9823A7E for ; Thu, 14 Jan 2021 20:35:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729906AbhANUfM (ORCPT ); Thu, 14 Jan 2021 15:35:12 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:4564 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729869AbhANUfL (ORCPT ); Thu, 14 Jan 2021 15:35:11 -0500 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK3ak3133595; Thu, 14 Jan 2021 15:32:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=fNSmaWXEdss3crWyVb0QJEI7ET0P47FXQa0CfGzvW7c=; b=oo8U4k6KkGg+Ftoa0D86/uBjh5bg9EB9r45WNwt1RgVKO12WTH2Cycv/FeU+4pbsI6YR A4XPlFIq+2pQHN6tC/V3Ndfg7YOly6RavVBl/1nMOhuEQGC7dLa2eFHc1zc9hk1YhmIL 6WL41HTqRk0t0O8YLDyxgX3n+qCINW+CGvr/l73hS/tUEmtBuPkvAyT/skloXQOAz5we pvDQee7nDPLnTMcR1t9D4zplRGF0qLpOT+lUfMpEM8UKl2Dy9aAN/sM6AMm6PuS05nop tMut0adSEm2TJkoMMR0ptFmk4Gpf4KxHZH4xbU4yO75f3+fQT8Ow/Ufbn/FiVBrMX2s5 sw== Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vnm8pgj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:59 -0500 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKSHEI007288; Thu, 14 Jan 2021 20:31:57 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma05wdc.us.ibm.com with ESMTP id 362cwmdsqw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:57 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVuDE36176204 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:56 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 819F26E04E; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2E5056E04C; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 15/21] ibmvfc: send commands down HW Sub-CRQ when channelized Date: Thu, 14 Jan 2021 14:31:42 -0600 Message-Id: <20210114203148.246656-16-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 phishscore=0 mlxscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 impostorscore=0 malwarescore=0 adultscore=0 spamscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org When the client has negotiated the use of channels all vfcFrames are required to go down a Sub-CRQ channel or it is a protocoal violation. If the adapter state is channelized submit vfcFrames to the appropriate Sub-CRQ via the h_send_sub_crq() helper. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 39 ++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 3f3cc37a263f..865b87881d86 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -704,6 +704,15 @@ static int ibmvfc_send_crq(struct ibmvfc_host *vhost, u64 word1, u64 word2) return plpar_hcall_norets(H_SEND_CRQ, vdev->unit_address, word1, word2); } +static int ibmvfc_send_sub_crq(struct ibmvfc_host *vhost, u64 cookie, u64 word1, + u64 word2, u64 word3, u64 word4) +{ + struct vio_dev *vdev = to_vio_dev(vhost->dev); + + return plpar_hcall_norets(H_SEND_SUB_CRQ, vdev->unit_address, cookie, + word1, word2, word3, word4); +} + /** * ibmvfc_send_crq_init - Send a CRQ init message * @vhost: ibmvfc host struct @@ -1623,8 +1632,17 @@ static int ibmvfc_send_event(struct ibmvfc_event *evt, mb(); - if ((rc = ibmvfc_send_crq(vhost, be64_to_cpu(crq_as_u64[0]), - be64_to_cpu(crq_as_u64[1])))) { + if (evt->queue->fmt == IBMVFC_SUB_CRQ_FMT) + rc = ibmvfc_send_sub_crq(vhost, + evt->queue->vios_cookie, + be64_to_cpu(crq_as_u64[0]), + be64_to_cpu(crq_as_u64[1]), + 0, 0); + else + rc = ibmvfc_send_crq(vhost, be64_to_cpu(crq_as_u64[0]), + be64_to_cpu(crq_as_u64[1])); + + if (rc) { list_del(&evt->queue_list); spin_unlock_irqrestore(&evt->queue->l_lock, flags); del_timer(&evt->timer); @@ -1842,6 +1860,7 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) struct ibmvfc_event *evt; u32 tag_and_hwq = blk_mq_unique_tag(cmnd->request); u16 hwq = blk_mq_unique_tag_to_hwq(tag_and_hwq); + u16 scsi_channel; int rc; if (unlikely((rc = fc_remote_port_chkready(rport))) || @@ -1852,7 +1871,13 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) } cmnd->result = (DID_OK << 16); - evt = ibmvfc_get_event(&vhost->crq); + if (vhost->using_channels) { + scsi_channel = hwq % vhost->scsi_scrqs.active_queues; + evt = ibmvfc_get_event(&vhost->scsi_scrqs.scrqs[scsi_channel]); + evt->hwq = hwq % vhost->scsi_scrqs.active_queues; + } else + evt = ibmvfc_get_event(&vhost->crq); + ibmvfc_init_event(evt, ibmvfc_scsi_done, IBMVFC_CMD_FORMAT); evt->cmnd = cmnd; @@ -1868,8 +1893,6 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) } vfc_cmd->correlation = cpu_to_be64((u64)evt); - if (vhost->using_channels) - evt->hwq = hwq % vhost->scsi_scrqs.active_queues; if (likely(!(rc = ibmvfc_map_sg_data(cmnd, evt, vfc_cmd, vhost->dev)))) return ibmvfc_send_event(evt, vhost, 0); @@ -2200,7 +2223,11 @@ static int ibmvfc_reset_device(struct scsi_device *sdev, int type, char *desc) spin_lock_irqsave(vhost->host->host_lock, flags); if (vhost->state == IBMVFC_ACTIVE) { - evt = ibmvfc_get_event(&vhost->crq); + if (vhost->using_channels) + evt = ibmvfc_get_event(&vhost->scsi_scrqs.scrqs[0]); + else + evt = ibmvfc_get_event(&vhost->crq); + ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_CMD_FORMAT); tmf = ibmvfc_init_vfc_cmd(evt, sdev); iu = ibmvfc_get_fcp_iu(vhost, tmf); From patchwork Thu Jan 14 20:31:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87D91C4321A for ; Thu, 14 Jan 2021 20:33:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 516E223A56 for ; Thu, 14 Jan 2021 20:33:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727273AbhANUda (ORCPT ); Thu, 14 Jan 2021 15:33:30 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:64916 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727703AbhANUcn (ORCPT ); Thu, 14 Jan 2021 15:32:43 -0500 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK3JEx056320; Thu, 14 Jan 2021 15:31:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=OVJmnGCGIYi28nfmWuMS2jM2uI1tF69N45Tl2rSaKQ4=; b=jBVzLL5SKWynYgYRVMSwy53M2PwbOrzJ9D76mZosjRfGZ1Ko81nL0xAoX8mwK8rvr9VN Z16yYUELMx4JxisdLZJQtUU8no8V9z73z638/6KAfOWyOy70fPJyw13nKcX0W6Rewtgs pPku1t6FZ3Dg/cKjC3TOT27dppEXFtfdj/wX2M8t4Bn7/vFM0DpBPH+3I263b9IfqWdJ p7J3H/44ZXZxDaexkJmniU3uwCpM0uegHJY2qZNDJdMQiYvs8HgWDlCpgzFidXSk7L00 KrBmDjD8oTAwM0aSOWoPGfX5A6Q/HF4jSlCmKgRDP56Yx8CLUxxmuCdwmElVfLcGuWy+ sg== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vca94eh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:31:59 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKS8W2026137; Thu, 14 Jan 2021 20:31:58 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma03dal.us.ibm.com with ESMTP id 35y449sneh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:58 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVu7E26739060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:57 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E36C66E04C; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 908CB6E052; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 16/21] ibmvfc: register Sub-CRQ handles with VIOS during channel setup Date: Thu, 14 Jan 2021 14:31:43 -0600 Message-Id: <20210114203148.246656-17-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 spamscore=0 clxscore=1015 malwarescore=0 priorityscore=1501 impostorscore=0 mlxscore=0 phishscore=0 adultscore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org If the ibmvfc client adapter requests channels it must submit a number of Sub-CRQ handles matching the number of channels being requested. The VIOS in its response will overwrite the actual number of channel resources allocated which may be less than what was requested. The client then must store the VIOS Sub-CRQ handle for each queue. This VIOS handle is needed as a parameter with h_send_sub_crq(). Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 865b87881d86..578e27180f10 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -4627,15 +4627,35 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost) static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) { struct ibmvfc_host *vhost = evt->vhost; + struct ibmvfc_channel_setup *setup = vhost->channel_setup_buf; + struct ibmvfc_scsi_channels *scrqs = &vhost->scsi_scrqs; u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); int level = IBMVFC_DEFAULT_LOG_LEVEL; + int flags, active_queues, i; ibmvfc_free_event(evt); switch (mad_status) { case IBMVFC_MAD_SUCCESS: ibmvfc_dbg(vhost, "Channel Setup succeded\n"); + flags = be32_to_cpu(setup->flags); vhost->do_enquiry = 0; + active_queues = be32_to_cpu(setup->num_scsi_subq_channels); + scrqs->active_queues = active_queues; + + if (flags & IBMVFC_CHANNELS_CANCELED) { + ibmvfc_dbg(vhost, "Channels Canceled\n"); + vhost->using_channels = 0; + } else { + if (active_queues) + vhost->using_channels = 1; + for (i = 0; i < active_queues; i++) + scrqs->scrqs[i].vios_cookie = + be64_to_cpu(setup->channel_handles[i]); + + ibmvfc_dbg(vhost, "Using %u channels\n", + vhost->scsi_scrqs.active_queues); + } break; case IBMVFC_MAD_FAILED: level += ibmvfc_retry_host_init(vhost); @@ -4659,9 +4679,19 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) struct ibmvfc_channel_setup_mad *mad; struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf; struct ibmvfc_event *evt = ibmvfc_get_event(&vhost->crq); + struct ibmvfc_scsi_channels *scrqs = &vhost->scsi_scrqs; + unsigned int num_channels = + min(vhost->client_scsi_channels, vhost->max_vios_scsi_channels); + int i; memset(setup_buf, 0, sizeof(*setup_buf)); - setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + if (num_channels == 0) + setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + else { + setup_buf->num_scsi_subq_channels = cpu_to_be32(num_channels); + for (i = 0; i < num_channels; i++) + setup_buf->channel_handles[i] = cpu_to_be64(scrqs->scrqs[i].cookie); + } ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); mad = &evt->iu.channel_setup; From patchwork Thu Jan 14 20:31:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BEADC433DB for ; Thu, 14 Jan 2021 20:35:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DA2C623A6C for ; Thu, 14 Jan 2021 20:35:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727244AbhANUez (ORCPT ); Thu, 14 Jan 2021 15:34:55 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:18056 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726459AbhANUey (ORCPT ); Thu, 14 Jan 2021 15:34:54 -0500 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKY0JU161229; Thu, 14 Jan 2021 15:34:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=pF1kTWzRDvOqDI/b1vn9yhlfp+aMDWf1vIB0dT8+eMA=; b=agpmYs3VPjX1Ad+N5CNvVMC8g7YPaKwNOpMq1UMNZL/3ibSGwqctN2czRIEZJCyS1Uej Jre6KmHADjcdHZ585ksSfBU0mr2raKcYrnoM1jnQan63zdq2341xsaKt8We5+jca8oCc hxAr1I9JV9ePN5IVOtxeLjYHXw8Gmey3+VVu41CA0rgbdOW8cIzoEGalpmCBpAzp83t9 t6o+6ywyJw4Q0pBcC36LzE3foZKAUH7l1kDjSzePiPAFSgQFuJ8CN7CJR8TAOEa66Qvt soKJmM2AbNJUPRYWHqY11GwVfncdsUnFm2eOgnnnRlnVg4JhRJMsYgi0sG+t64M9BqDX iw== Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0b-001b2d01.pphosted.com with ESMTP id 362vtb8c59-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:34:04 -0500 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKRQ1L013843; Thu, 14 Jan 2021 20:31:58 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma03wdc.us.ibm.com with ESMTP id 35y449fsv8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:58 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVviO25756030 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:57 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 512B86E056; Thu, 14 Jan 2021 20:31:57 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F28F86E052; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:56 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 17/21] ibmvfc: add cancel mad initialization helper Date: Thu, 14 Jan 2021 14:31:44 -0600 Message-Id: <20210114203148.246656-18-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 spamscore=0 impostorscore=0 phishscore=0 adultscore=0 clxscore=1015 priorityscore=1501 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add a helper routine for initializing a Cancel MAD. This will be useful for a channelized client that needs to send Cancel commands down every channel commands were sent for a particular LUN. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 68 ++++++++++++++++++++-------------- 1 file changed, 40 insertions(+), 28 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 578e27180f10..b0b0212344f3 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -2379,6 +2379,45 @@ static int ibmvfc_wait_for_ops(struct ibmvfc_host *vhost, void *device, return SUCCESS; } +static struct ibmvfc_event *ibmvfc_init_tmf(struct ibmvfc_queue *queue, + struct scsi_device *sdev, + int type) +{ + struct ibmvfc_host *vhost = shost_priv(sdev->host); + struct scsi_target *starget = scsi_target(sdev); + struct fc_rport *rport = starget_to_rport(starget); + struct ibmvfc_event *evt; + struct ibmvfc_tmf *tmf; + + evt = ibmvfc_get_event(queue); + ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_MAD_FORMAT); + + tmf = &evt->iu.tmf; + memset(tmf, 0, sizeof(*tmf)); + if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN)) { + tmf->common.version = cpu_to_be32(2); + tmf->target_wwpn = cpu_to_be64(rport->port_name); + } else { + tmf->common.version = cpu_to_be32(1); + } + tmf->common.opcode = cpu_to_be32(IBMVFC_TMF_MAD); + tmf->common.length = cpu_to_be16(sizeof(*tmf)); + tmf->scsi_id = cpu_to_be64(rport->port_id); + int_to_scsilun(sdev->lun, &tmf->lun); + if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS)) + type &= ~IBMVFC_TMF_SUPPRESS_ABTS; + if (vhost->state == IBMVFC_ACTIVE) + tmf->flags = cpu_to_be32((type | IBMVFC_TMF_LUA_VALID)); + else + tmf->flags = cpu_to_be32(((type & IBMVFC_TMF_SUPPRESS_ABTS) | IBMVFC_TMF_LUA_VALID)); + tmf->cancel_key = cpu_to_be32((unsigned long)sdev->hostdata); + tmf->my_cancel_key = cpu_to_be32((unsigned long)starget->hostdata); + + init_completion(&evt->comp); + + return evt; +} + /** * ibmvfc_cancel_all - Cancel all outstanding commands to the device * @sdev: scsi device to cancel commands @@ -2393,9 +2432,6 @@ static int ibmvfc_wait_for_ops(struct ibmvfc_host *vhost, void *device, static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) { struct ibmvfc_host *vhost = shost_priv(sdev->host); - struct scsi_target *starget = scsi_target(sdev); - struct fc_rport *rport = starget_to_rport(starget); - struct ibmvfc_tmf *tmf; struct ibmvfc_event *evt, *found_evt; union ibmvfc_iu rsp; int rsp_rc = -EBUSY; @@ -2422,32 +2458,8 @@ static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) } if (vhost->logged_in) { - evt = ibmvfc_get_event(&vhost->crq); - ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_MAD_FORMAT); - - tmf = &evt->iu.tmf; - memset(tmf, 0, sizeof(*tmf)); - if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN)) { - tmf->common.version = cpu_to_be32(2); - tmf->target_wwpn = cpu_to_be64(rport->port_name); - } else { - tmf->common.version = cpu_to_be32(1); - } - tmf->common.opcode = cpu_to_be32(IBMVFC_TMF_MAD); - tmf->common.length = cpu_to_be16(sizeof(*tmf)); - tmf->scsi_id = cpu_to_be64(rport->port_id); - int_to_scsilun(sdev->lun, &tmf->lun); - if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS)) - type &= ~IBMVFC_TMF_SUPPRESS_ABTS; - if (vhost->state == IBMVFC_ACTIVE) - tmf->flags = cpu_to_be32((type | IBMVFC_TMF_LUA_VALID)); - else - tmf->flags = cpu_to_be32(((type & IBMVFC_TMF_SUPPRESS_ABTS) | IBMVFC_TMF_LUA_VALID)); - tmf->cancel_key = cpu_to_be32((unsigned long)sdev->hostdata); - tmf->my_cancel_key = cpu_to_be32((unsigned long)starget->hostdata); - + evt = ibmvfc_init_tmf(&vhost->crq, sdev, type); evt->sync_iu = &rsp; - init_completion(&evt->comp); rsp_rc = ibmvfc_send_event(evt, vhost, default_timeout); } From patchwork Thu Jan 14 20:31:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5FDAC433E9 for ; Thu, 14 Jan 2021 20:33:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7109223A56 for ; Thu, 14 Jan 2021 20:33:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728364AbhANUct (ORCPT ); Thu, 14 Jan 2021 15:32:49 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:37236 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728099AbhANUcr (ORCPT ); Thu, 14 Jan 2021 15:32:47 -0500 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKVm5A108803; Thu, 14 Jan 2021 15:32:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=isxwZPqQSQ5/Q/W06H/NV4P1fDoItP4mLe1+8ylQYm8=; b=JsZtVrfkDPfVQ9tjroKcCSKskQx7GIgCz2e9/eHXwAnCclvrtaz3u6HpwVQ+YHt812EL Hgq0uASKXN79vl1AcxU3Kpdo5jZSA8XMhfc6iTF+JLcRsOwSBxcZ2bPnff0NvimTDiCF s7QnH3ZELi0SbEDI/WtKf9Ij6JM82Ge0FEheRPZieseZIZwslNjg3+QvXeHtP4OWlCTJ Q9wdvElA2MmRYdIryKqTFKyM1YPSMluRdMEhGsw2oGP6svUstGe9PPj10J4gWUWv1L+S OKgI1te0fpuvdrsebL+JmmsWB7ZKvsz8JMXAX50pERPk/twjO+IdLSOL4TuWa3FXJDJl qQ== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0b-001b2d01.pphosted.com with ESMTP id 362veg9148-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:32:00 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKSBGG007089; Thu, 14 Jan 2021 20:31:59 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma01dal.us.ibm.com with ESMTP id 35y449sr12-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:59 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVvDE25887034 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:57 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB48E6E050; Thu, 14 Jan 2021 20:31:57 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 600EA6E052; Thu, 14 Jan 2021 20:31:57 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:57 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v5 18/21] ibmvfc: send Cancel MAD down each hw scsi channel Date: Thu, 14 Jan 2021 14:31:45 -0600 Message-Id: <20210114203148.246656-19-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 impostorscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 clxscore=1015 mlxscore=0 adultscore=0 priorityscore=1501 spamscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In general the client needs to send Cancel MADs and task management commands down the same channel as the command(s) intended to cancel or abort. The client assigns cancel keys per LUN and thus must send a Cancel down each channel commands were submitted for that LUN. Further, the client then must wait for those cancel completions prior to submitting a LUN RESET or ABORT TASK SET. Add a cancel rsp iu syncronization field to the ibmvfc_queue struct such that the cancel routine can sync the cancel response to each queue that requires a cancel command. Build a list of each cancel event sent and wait for the completion of each submitted cancel. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 109 +++++++++++++++++++++++++++++---- drivers/scsi/ibmvscsi/ibmvfc.h | 3 + 2 files changed, 100 insertions(+), 12 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index b0b0212344f3..5ca8fcafd1d5 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -2418,18 +2418,82 @@ static struct ibmvfc_event *ibmvfc_init_tmf(struct ibmvfc_queue *queue, return evt; } -/** - * ibmvfc_cancel_all - Cancel all outstanding commands to the device - * @sdev: scsi device to cancel commands - * @type: type of error recovery being performed - * - * This sends a cancel to the VIOS for the specified device. This does - * NOT send any abort to the actual device. That must be done separately. - * - * Returns: - * 0 on success / other on failure - **/ -static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) +static int ibmvfc_cancel_all_mq(struct scsi_device *sdev, int type) +{ + struct ibmvfc_host *vhost = shost_priv(sdev->host); + struct ibmvfc_event *evt, *found_evt, *temp; + struct ibmvfc_queue *queues = vhost->scsi_scrqs.scrqs; + unsigned long flags; + int num_hwq, i; + int fail = 0; + LIST_HEAD(cancelq); + u16 status; + + ENTER; + spin_lock_irqsave(vhost->host->host_lock, flags); + num_hwq = vhost->scsi_scrqs.active_queues; + for (i = 0; i < num_hwq; i++) { + spin_lock(queues[i].q_lock); + spin_lock(&queues[i].l_lock); + found_evt = NULL; + list_for_each_entry(evt, &queues[i].sent, queue_list) { + if (evt->cmnd && evt->cmnd->device == sdev) { + found_evt = evt; + break; + } + } + spin_unlock(&queues[i].l_lock); + + if (found_evt && vhost->logged_in) { + evt = ibmvfc_init_tmf(&queues[i], sdev, type); + evt->sync_iu = &queues[i].cancel_rsp; + ibmvfc_send_event(evt, vhost, default_timeout); + list_add_tail(&evt->cancel, &cancelq); + } + + spin_unlock(queues[i].q_lock); + } + spin_unlock_irqrestore(vhost->host->host_lock, flags); + + if (list_empty(&cancelq)) { + if (vhost->log_level > IBMVFC_DEFAULT_LOG_LEVEL) + sdev_printk(KERN_INFO, sdev, "No events found to cancel\n"); + return 0; + } + + sdev_printk(KERN_INFO, sdev, "Cancelling outstanding commands.\n"); + + list_for_each_entry_safe(evt, temp, &cancelq, cancel) { + wait_for_completion(&evt->comp); + status = be16_to_cpu(evt->queue->cancel_rsp.mad_common.status); + list_del(&evt->cancel); + ibmvfc_free_event(evt); + + if (status != IBMVFC_MAD_SUCCESS) { + sdev_printk(KERN_WARNING, sdev, "Cancel failed with rc=%x\n", status); + switch (status) { + case IBMVFC_MAD_DRIVER_FAILED: + case IBMVFC_MAD_CRQ_ERROR: + /* Host adapter most likely going through reset, return success to + * the caller will wait for the command being cancelled to get returned + */ + break; + default: + fail = 1; + break; + } + } + } + + if (fail) + return -EIO; + + sdev_printk(KERN_INFO, sdev, "Successfully cancelled outstanding commands\n"); + LEAVE; + return 0; +} + +static int ibmvfc_cancel_all_sq(struct scsi_device *sdev, int type) { struct ibmvfc_host *vhost = shost_priv(sdev->host); struct ibmvfc_event *evt, *found_evt; @@ -2498,6 +2562,27 @@ static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) return 0; } +/** + * ibmvfc_cancel_all - Cancel all outstanding commands to the device + * @sdev: scsi device to cancel commands + * @type: type of error recovery being performed + * + * This sends a cancel to the VIOS for the specified device. This does + * NOT send any abort to the actual device. That must be done separately. + * + * Returns: + * 0 on success / other on failure + **/ +static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) +{ + struct ibmvfc_host *vhost = shost_priv(sdev->host); + + if (vhost->mq_enabled && vhost->using_channels) + return ibmvfc_cancel_all_mq(sdev, type); + else + return ibmvfc_cancel_all_sq(sdev, type); +} + /** * ibmvfc_match_key - Match function for specified cancel key * @evt: ibmvfc event struct diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 2dbce7071409..c3bb83c9d8a6 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -737,6 +737,7 @@ struct ibmvfc_target { /* a unit of work for the hosting partition */ struct ibmvfc_event { struct list_head queue_list; + struct list_head cancel; struct ibmvfc_host *vhost; struct ibmvfc_queue *queue; struct ibmvfc_target *tgt; @@ -790,6 +791,8 @@ struct ibmvfc_queue { struct list_head free; spinlock_t l_lock; + union ibmvfc_iu cancel_rsp; + /* Sub-CRQ fields */ struct ibmvfc_host *vhost; unsigned long cookie; From patchwork Thu Jan 14 20:31:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B522EC4332D for ; Thu, 14 Jan 2021 20:33:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8EFC823A56 for ; Thu, 14 Jan 2021 20:33:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728623AbhANUdL (ORCPT ); Thu, 14 Jan 2021 15:33:11 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:56062 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728060AbhANUcq (ORCPT ); Thu, 14 Jan 2021 15:32:46 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EKVl5B036570; Thu, 14 Jan 2021 15:32:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=z3nI/cWv0TReJ7MoPJifthz7mo+0RrF8eBMrSt6XeU0=; b=ffreLlDgKO2EhYoW8D4HFwBnytpqwN0yuaRI5ETvLPDafQ0rb6CgIoY9Xt5/oAhNhLnw ycIyWn8xc8Nw8zWozfv4R4CYsp1NTQ1b/Z4KDbIqWHLX23TQaNuRyi0HBjHHduQ3FyzE uiVepPWoUDMaYCRMjxzJ0PznxVkqWHWcI+S/zcmukKruLEb7SsEvc0vrY9UWu+lNjOBv D0YKtF5+eVKBN9KffQC4RAqkBWf3wYwm5gbLah9lDbHvCscQgbeRyE3GDRcL0+goSZEb nKVOp+fGZ39nfJ0pNdGMxzX8xH4ZD/d0RWC68u1fQQfaQzqMW0hJ5IbO0jehCEm0fTdk cw== Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 362vbj94ur-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:32:00 -0500 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKROJW032073; Thu, 14 Jan 2021 20:31:59 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma04wdc.us.ibm.com with ESMTP id 35y449fskh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:59 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVwua23986446 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:58 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 18A306E050; Thu, 14 Jan 2021 20:31:58 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA6386E05B; Thu, 14 Jan 2021 20:31:57 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:57 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 19/21] ibmvfc: purge scsi channels after transport loss/reset Date: Thu, 14 Jan 2021 14:31:46 -0600 Message-Id: <20210114203148.246656-20-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 bulkscore=0 suspectscore=0 spamscore=0 impostorscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Grab the queue and list lock for each Sub-CRQ and add any uncompleted events to the host purge list. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 5ca8fcafd1d5..d314dffaafc4 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1056,7 +1056,13 @@ static void ibmvfc_fail_request(struct ibmvfc_event *evt, int error_code) static void ibmvfc_purge_requests(struct ibmvfc_host *vhost, int error_code) { struct ibmvfc_event *evt, *pos; + struct ibmvfc_queue *queues = vhost->scsi_scrqs.scrqs; unsigned long flags; + int hwqs = 0; + int i; + + if (vhost->using_channels) + hwqs = vhost->scsi_scrqs.active_queues; ibmvfc_dbg(vhost, "Purging all requests\n"); spin_lock_irqsave(&vhost->crq.l_lock, flags); @@ -1064,6 +1070,16 @@ static void ibmvfc_purge_requests(struct ibmvfc_host *vhost, int error_code) ibmvfc_fail_request(evt, error_code); list_splice_init(&vhost->crq.sent, &vhost->purge); spin_unlock_irqrestore(&vhost->crq.l_lock, flags); + + for (i = 0; i < hwqs; i++) { + spin_lock_irqsave(queues[i].q_lock, flags); + spin_lock(&queues[i].l_lock); + list_for_each_entry_safe(evt, pos, &queues[i].sent, queue_list) + ibmvfc_fail_request(evt, error_code); + list_splice_init(&queues[i].sent, &vhost->purge); + spin_unlock(&queues[i].l_lock); + spin_unlock_irqrestore(queues[i].q_lock, flags); + } } /** From patchwork Thu Jan 14 20:31:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 363254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EF1FC4332E for ; Thu, 14 Jan 2021 20:33:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 70CB523A5F for ; Thu, 14 Jan 2021 20:33:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728654AbhANUdL (ORCPT ); Thu, 14 Jan 2021 15:33:11 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:53546 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726066AbhANUcp (ORCPT ); Thu, 14 Jan 2021 15:32:45 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK2slJ051361; Thu, 14 Jan 2021 15:32:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=dj9oH9mDaFvH1GVNXBhap4vp+1FQO0wVABQYlHPF47o=; b=eP1dXuIzAtFLGDEgsXlNxStkGf9fhROQNgSrBBGbeLWJ0ssyUdLWx3K3slCtrVY9YhqG TC8zgi7+T+pf9pKCySsL3lBs16LTJSa3SLhy6vuMDl+C+boLALw0lRxQP/4l8xoo1ShF ksK11Qo9htFX10BnfH4z5XEQ29uxo0xF5QGXZxUObu8BNo6U6HOkSQxU3XrxUg5z9jUU OtKWwthgqS2Rsy9rL0lHzK/HFbxbdX+kVcOCGdxI3WMwbYJZBqVsvKGVFFDUsQ06QZC6 LK1nLlEy8x6aHx5tyC3FHrpxfAx1kQ4JvlXNgPo4QKU3EiY3kY+IzWjHUMDVXHlYukfO wA== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 362v29sgj4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:32:01 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKHJNa004698; Thu, 14 Jan 2021 20:32:00 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma02wdc.us.ibm.com with ESMTP id 35y449fryn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:31:59 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVwDX24510782 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:58 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 83C5F6E050; Thu, 14 Jan 2021 20:31:58 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2796D6E052; Thu, 14 Jan 2021 20:31:58 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:58 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v5 20/21] ibmvfc: enable MQ and set reasonable defaults Date: Thu, 14 Jan 2021 14:31:47 -0600 Message-Id: <20210114203148.246656-21-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0 priorityscore=1501 mlxlogscore=856 malwarescore=0 impostorscore=0 spamscore=0 clxscore=1015 bulkscore=0 adultscore=0 phishscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Turn on MQ by default and set sane values for the upper limit on hw queues for the scsi host, and number of hw scsi channels to request from the partner VIOS. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index c3bb83c9d8a6..0391cb746d0b 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -41,9 +41,9 @@ #define IBMVFC_DEFAULT_LOG_LEVEL 2 #define IBMVFC_MAX_CDB_LEN 16 #define IBMVFC_CLS3_ERROR 0 -#define IBMVFC_MQ 0 -#define IBMVFC_SCSI_CHANNELS 0 -#define IBMVFC_SCSI_HW_QUEUES 1 +#define IBMVFC_MQ 1 +#define IBMVFC_SCSI_CHANNELS 8 +#define IBMVFC_SCSI_HW_QUEUES 8 #define IBMVFC_MIG_NO_SUB_TO_CRQ 0 #define IBMVFC_MIG_NO_N_TO_M 0 From patchwork Thu Jan 14 20:31:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 364247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 212DFC433DB for ; Thu, 14 Jan 2021 20:35:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E567523A81 for ; Thu, 14 Jan 2021 20:35:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730225AbhANUfQ (ORCPT ); Thu, 14 Jan 2021 15:35:16 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:10652 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726693AbhANUfN (ORCPT ); Thu, 14 Jan 2021 15:35:13 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10EK2riw051326; Thu, 14 Jan 2021 15:32:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=KJuvTZ9SlAAfX4HEuXhK5kdGQ8DRK12f0BqL9jyuIlo=; b=S9ZbKLoy8dGOXJVHzQ9RcaQlRWloyxj9+ckDIIIYeRuVXqFXWN1HjmLKX3bO6nYU13l2 LeyzeetHTI91IrKpsYkQ+cztm68/332PJF9UKBmQX2N77lCUYeaCkSG/V668LslUcX9V 4eIC0iA2IJ4Uf9mgg6Sf6Yk+OVD5fDbbAolCu/9hcL4zEAjrRdoZET0Nn9xJDYPbO7VV hqyP9FS1HL9c/4+J2JDodZZTtiL3G2+q84kvLeFdmxtsLIPD0gDvpEVtioCzCfBEkpeu CUQTQBy2jWtSXCLeALUwi250H5VG273H4qqNW36C9BLgY0KLoAcZsT/0lRAw5av39ifH 2w== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 362v29sgje-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 15:32:01 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10EKS7RL026094; Thu, 14 Jan 2021 20:32:00 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma03dal.us.ibm.com with ESMTP id 35y449snet-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Jan 2021 20:32:00 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10EKVxbi26083768 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Jan 2021 20:31:59 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E78F46E052; Thu, 14 Jan 2021 20:31:58 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 929B16E04E; Thu, 14 Jan 2021 20:31:58 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 14 Jan 2021 20:31:58 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v5 21/21] ibmvfc: provide modules parameters for MQ settings Date: Thu, 14 Jan 2021 14:31:48 -0600 Message-Id: <20210114203148.246656-22-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210114203148.246656-1-tyreld@linux.ibm.com> References: <20210114203148.246656-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-14_07:2021-01-14,2021-01-14 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0 priorityscore=1501 mlxlogscore=999 malwarescore=0 impostorscore=0 spamscore=0 clxscore=1015 bulkscore=0 adultscore=0 phishscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101140115 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add the various module parameter toggles for adjusting the MQ characteristics at boot/load time as well as a device attribute for changing the client scsi channel request amount. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 75 ++++++++++++++++++++++++++++++---- drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 67 insertions(+), 9 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index d314dffaafc4..7097028d4cb6 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -40,6 +40,12 @@ static unsigned int disc_threads = IBMVFC_MAX_DISC_THREADS; static unsigned int ibmvfc_debug = IBMVFC_DEBUG; static unsigned int log_level = IBMVFC_DEFAULT_LOG_LEVEL; static unsigned int cls3_error = IBMVFC_CLS3_ERROR; +static unsigned int mq_enabled = IBMVFC_MQ; +static unsigned int nr_scsi_hw_queues = IBMVFC_SCSI_HW_QUEUES; +static unsigned int nr_scsi_channels = IBMVFC_SCSI_CHANNELS; +static unsigned int mig_channels_only = IBMVFC_MIG_NO_SUB_TO_CRQ; +static unsigned int mig_no_less_channels = IBMVFC_MIG_NO_N_TO_M; + static LIST_HEAD(ibmvfc_head); static DEFINE_SPINLOCK(ibmvfc_driver_lock); static struct scsi_transport_template *ibmvfc_transport_template; @@ -49,6 +55,22 @@ MODULE_AUTHOR("Brian King "); MODULE_LICENSE("GPL"); MODULE_VERSION(IBMVFC_DRIVER_VERSION); +module_param_named(mq, mq_enabled, uint, S_IRUGO); +MODULE_PARM_DESC(mq, "Enable multiqueue support. " + "[Default=" __stringify(IBMVFC_MQ) "]"); +module_param_named(scsi_host_queues, nr_scsi_hw_queues, uint, S_IRUGO); +MODULE_PARM_DESC(scsi_host_queues, "Number of SCSI Host submission queues. " + "[Default=" __stringify(IBMVFC_SCSI_HW_QUEUES) "]"); +module_param_named(scsi_hw_channels, nr_scsi_channels, uint, S_IRUGO); +MODULE_PARM_DESC(scsi_hw_channels, "Number of hw scsi channels to request. " + "[Default=" __stringify(IBMVFC_SCSI_CHANNELS) "]"); +module_param_named(mig_channels_only, mig_channels_only, uint, S_IRUGO); +MODULE_PARM_DESC(mig_channels_only, "Prevent migration to non-channelized system. " + "[Default=" __stringify(IBMVFC_MIG_NO_SUB_TO_CRQ) "]"); +module_param_named(mig_no_less_channels, mig_no_less_channels, uint, S_IRUGO); +MODULE_PARM_DESC(mig_no_less_channels, "Prevent migration to system with less channels. " + "[Default=" __stringify(IBMVFC_MIG_NO_N_TO_M) "]"); + module_param_named(init_timeout, init_timeout, uint, S_IRUGO | S_IWUSR); MODULE_PARM_DESC(init_timeout, "Initialization timeout in seconds. " "[Default=" __stringify(IBMVFC_INIT_TIMEOUT) "]"); @@ -926,7 +948,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) crq->cur = 0; if (vhost->scsi_scrqs.scrqs) { - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + for (i = 0; i < nr_scsi_hw_queues; i++) { scrq = &vhost->scsi_scrqs.scrqs[i]; spin_lock(scrq->q_lock); memset(scrq->msgs.scrq, 0, PAGE_SIZE); @@ -3397,6 +3419,37 @@ static ssize_t ibmvfc_store_log_level(struct device *dev, return strlen(buf); } +static ssize_t ibmvfc_show_scsi_channels(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct Scsi_Host *shost = class_to_shost(dev); + struct ibmvfc_host *vhost = shost_priv(shost); + unsigned long flags = 0; + int len; + + spin_lock_irqsave(shost->host_lock, flags); + len = snprintf(buf, PAGE_SIZE, "%d\n", vhost->client_scsi_channels); + spin_unlock_irqrestore(shost->host_lock, flags); + return len; +} + +static ssize_t ibmvfc_store_scsi_channels(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct Scsi_Host *shost = class_to_shost(dev); + struct ibmvfc_host *vhost = shost_priv(shost); + unsigned long flags = 0; + unsigned int channels; + + spin_lock_irqsave(shost->host_lock, flags); + channels = simple_strtoul(buf, NULL, 10); + vhost->client_scsi_channels = min(channels, nr_scsi_hw_queues); + ibmvfc_hard_reset_host(vhost); + spin_unlock_irqrestore(shost->host_lock, flags); + return strlen(buf); +} + static DEVICE_ATTR(partition_name, S_IRUGO, ibmvfc_show_host_partition_name, NULL); static DEVICE_ATTR(device_name, S_IRUGO, ibmvfc_show_host_device_name, NULL); static DEVICE_ATTR(port_loc_code, S_IRUGO, ibmvfc_show_host_loc_code, NULL); @@ -3405,6 +3458,8 @@ static DEVICE_ATTR(npiv_version, S_IRUGO, ibmvfc_show_host_npiv_version, NULL); static DEVICE_ATTR(capabilities, S_IRUGO, ibmvfc_show_host_capabilities, NULL); static DEVICE_ATTR(log_level, S_IRUGO | S_IWUSR, ibmvfc_show_log_level, ibmvfc_store_log_level); +static DEVICE_ATTR(nr_scsi_channels, S_IRUGO | S_IWUSR, + ibmvfc_show_scsi_channels, ibmvfc_store_scsi_channels); #ifdef CONFIG_SCSI_IBMVFC_TRACE /** @@ -3461,6 +3516,7 @@ static struct device_attribute *ibmvfc_attrs[] = { &dev_attr_npiv_version, &dev_attr_capabilities, &dev_attr_log_level, + &dev_attr_nr_scsi_channels, NULL }; @@ -4866,9 +4922,9 @@ static void ibmvfc_channel_enquiry(struct ibmvfc_host *vhost) mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_ENQUIRY); mad->common.length = cpu_to_be16(sizeof(*mad)); - if (IBMVFC_MIG_NO_SUB_TO_CRQ) + if (mig_channels_only) mad->flags |= cpu_to_be32(IBMVFC_NO_CHANNELS_TO_CRQ_SUPPORT); - if (IBMVFC_MIG_NO_N_TO_M) + if (mig_no_less_channels) mad->flags |= cpu_to_be32(IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT); ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); @@ -5655,13 +5711,13 @@ static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) ENTER; - vhost->scsi_scrqs.scrqs = kcalloc(IBMVFC_SCSI_HW_QUEUES, + vhost->scsi_scrqs.scrqs = kcalloc(nr_scsi_hw_queues, sizeof(*vhost->scsi_scrqs.scrqs), GFP_KERNEL); if (!vhost->scsi_scrqs.scrqs) return -1; - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + for (i = 0; i < nr_scsi_hw_queues; i++) { if (ibmvfc_register_scsi_channel(vhost, i)) { for (j = i; j > 0; j--) ibmvfc_deregister_scsi_channel(vhost, j - 1); @@ -5685,7 +5741,7 @@ static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) if (!vhost->scsi_scrqs.scrqs) return; - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) + for (i = 0; i < nr_scsi_hw_queues; i++) ibmvfc_deregister_scsi_channel(vhost, i); kfree(vhost->scsi_scrqs.scrqs); @@ -5873,6 +5929,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) struct Scsi_Host *shost; struct device *dev = &vdev->dev; int rc = -ENOMEM; + unsigned int max_scsi_queues = IBMVFC_MAX_SCSI_QUEUES; ENTER; shost = scsi_host_alloc(&driver_template, sizeof(*vhost)); @@ -5888,7 +5945,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) shost->max_sectors = IBMVFC_MAX_SECTORS; shost->max_cmd_len = IBMVFC_MAX_CDB_LEN; shost->unique_id = shost->host_no; - shost->nr_hw_queues = IBMVFC_MQ ? IBMVFC_SCSI_HW_QUEUES : 1; + shost->nr_hw_queues = mq_enabled ? min(max_scsi_queues, nr_scsi_hw_queues) : 1; vhost = shost_priv(shost); INIT_LIST_HEAD(&vhost->targets); @@ -5900,8 +5957,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->log_level = log_level; vhost->task_set = 1; - vhost->mq_enabled = IBMVFC_MQ; - vhost->client_scsi_channels = IBMVFC_SCSI_CHANNELS; + vhost->mq_enabled = mq_enabled; + vhost->client_scsi_channels = min(shost->nr_hw_queues, nr_scsi_channels); vhost->using_channels = 0; vhost->do_enquiry = 1; diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 0391cb746d0b..19dcec3ae9ba 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -43,6 +43,7 @@ #define IBMVFC_CLS3_ERROR 0 #define IBMVFC_MQ 1 #define IBMVFC_SCSI_CHANNELS 8 +#define IBMVFC_MAX_SCSI_QUEUES 16 #define IBMVFC_SCSI_HW_QUEUES 8 #define IBMVFC_MIG_NO_SUB_TO_CRQ 0 #define IBMVFC_MIG_NO_N_TO_M 0