From patchwork Mon Jan 11 23:12:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC43EC43332 for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A9E8522CB8 for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405532AbhALAZp (ORCPT ); Mon, 11 Jan 2021 19:25:45 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:32608 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403849AbhAKXNU (ORCPT ); Mon, 11 Jan 2021 18:13:20 -0500 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN3KCS142679; Mon, 11 Jan 2021 18:12:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=qo5eJE63wZrxZvsoxpPAFKZcHlR5E2jJ3tsbMsgVvj8=; b=SXiLxgxb15MiDuyN3vRYL4CLEOG6NZs6/6y0r46sVQLVx6Ehr8J4jZ0ZdWXXgHMSOH+E JRpgxbWOOJicz2HcHFunIQLSAE32AqfaSuU2spHRicPNk36PWvkumtXInbuf423DAAhX YhURC9qu3X6aBZ1kvMDu0v218iJwY4ZRC3dQHxQUTNOsDo4T4RMW9we7+ebv52wlkeon xvW7m7Ijibdz+7ryEq/KMEN70TTx9iiGBuPzseRMyCMfkgVYiWlR6Ue8JgXb7lZo3Lp8 KCmeCK0GGdvu2Y1h0/kvhjkFT37+gcy5EvAPuHfEdNz/4AiOXhvnNbgOgFhlU3zFyVRZ EA== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 361001r8ps-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:30 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN9qtq009897; Mon, 11 Jan 2021 23:12:29 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma02wdc.us.ibm.com with ESMTP id 35y448sy9r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:29 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCSuq28180980 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:28 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 48C4C7805C; Mon, 11 Jan 2021 23:12:28 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DB4E47805F; Mon, 11 Jan 2021 23:12:27 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:27 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 01/21] ibmvfc: add vhost fields and defaults for MQ enablement Date: Mon, 11 Jan 2021 17:12:05 -0600 Message-Id: <20210111231225.105347-2-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_34:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 clxscore=1015 suspectscore=0 phishscore=0 impostorscore=0 mlxlogscore=999 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110130 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce several new vhost fields for managing MQ state of the adapter as well as initial defaults for MQ enablement. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 8 ++++++++ drivers/scsi/ibmvscsi/ibmvfc.h | 9 +++++++++ 2 files changed, 17 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index ba95438a8912..9200fe49c57e 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3302,6 +3302,7 @@ static struct scsi_host_template driver_template = { .max_sectors = IBMVFC_MAX_SECTORS, .shost_attrs = ibmvfc_attrs, .track_queue_depth = 1, + .host_tagset = 1, }; /** @@ -5290,6 +5291,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) shost->max_sectors = IBMVFC_MAX_SECTORS; shost->max_cmd_len = IBMVFC_MAX_CDB_LEN; shost->unique_id = shost->host_no; + shost->nr_hw_queues = IBMVFC_MQ ? IBMVFC_SCSI_HW_QUEUES : 1; vhost = shost_priv(shost); INIT_LIST_HEAD(&vhost->targets); @@ -5300,6 +5302,12 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->partition_number = -1; vhost->log_level = log_level; vhost->task_set = 1; + + vhost->mq_enabled = IBMVFC_MQ; + vhost->client_scsi_channels = IBMVFC_SCSI_CHANNELS; + vhost->using_channels = 0; + vhost->do_enquiry = 1; + strcpy(vhost->partition_name, "UNKNOWN"); init_waitqueue_head(&vhost->work_wait_q); init_waitqueue_head(&vhost->init_wait_q); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 632e977449c5..dd6d89292867 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -41,6 +41,11 @@ #define IBMVFC_DEFAULT_LOG_LEVEL 2 #define IBMVFC_MAX_CDB_LEN 16 #define IBMVFC_CLS3_ERROR 0 +#define IBMVFC_MQ 0 +#define IBMVFC_SCSI_CHANNELS 0 +#define IBMVFC_SCSI_HW_QUEUES 1 +#define IBMVFC_MIG_NO_SUB_TO_CRQ 0 +#define IBMVFC_MIG_NO_N_TO_M 0 /* * Ensure we have resources for ERP and initialization: @@ -840,6 +845,10 @@ struct ibmvfc_host { int delay_init; int scan_complete; int logged_in; + int mq_enabled; + int using_channels; + int do_enquiry; + int client_scsi_channels; int aborting_passthru; int events_to_log; #define IBMVFC_AE_LINKUP 0x0001 From patchwork Mon Jan 11 23:12:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AEEAC43381 for ; Tue, 12 Jan 2021 00:31:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D746B22D08 for ; Tue, 12 Jan 2021 00:31:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391764AbhALAZq (ORCPT ); Mon, 11 Jan 2021 19:25:46 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:46372 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403864AbhAKXNV (ORCPT ); Mon, 11 Jan 2021 18:13:21 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BNBrbs017938; Mon, 11 Jan 2021 18:12:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=tW30Dz7n8q+6qhgZvSJ2mdmZP94XSZx4NCukQuMrLos=; b=r3PmpfOuffHCHW/Mya3jDFb5U8ZFAMwJJCyaCIJrOVxbJwxc2K7lF29qxHZ3iyQT3AYm FQl3qahFR4K/wA54X0LDGKPQL0n+ZaIrIDm+85IIC3E2/hHnKrkZmwbP5BeB+juYaghR wFk44a9tF/zZiEdqFFHQsxAjCEYeYGV0sKjsUNqzmwDRK0PA+zDiAvprtv631iJyIPKy mompwZeYbWflFX9hTGq+i3z1PMSWvrbxdj0kt/ytoQVvFuNbgJvQ34SaZKKREzqsOaHf rk1Bnrw6JswYrAghRFbGKwpbZxV0ECfHOpam+kvnhQdN+SF4dbqE+KiZqf5h82N3HZp2 Ew== Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 36104wg083-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:31 -0500 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN8Who022027; Mon, 11 Jan 2021 23:12:30 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma04dal.us.ibm.com with ESMTP id 35y448xfh4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:30 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCSUM28180986 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:28 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BDE4578064; Mon, 11 Jan 2021 23:12:28 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5C56678063; Mon, 11 Jan 2021 23:12:28 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:28 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 02/21] ibmvfc: move event pool init/free routines Date: Mon, 11 Jan 2021 17:12:06 -0600 Message-Id: <20210111231225.105347-3-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 mlxscore=0 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 suspectscore=0 mlxlogscore=947 clxscore=1015 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The next patch in this series reworks the event pool allocation calls to happen within the individual queue allocation routines instead of as independent calls. Move the init/free routines earlier in ibmvfc.c to prevent undefined reference errors when calling these functions from the queue allocation code. No functional change. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 151 +++++++++++++++++---------------- 1 file changed, 76 insertions(+), 75 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 9200fe49c57e..cd9273a5fadb 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -716,6 +716,82 @@ static int ibmvfc_send_crq_init_complete(struct ibmvfc_host *vhost) return ibmvfc_send_crq(vhost, 0xC002000000000000LL, 0); } +/** + * ibmvfc_init_event_pool - Allocates and initializes the event pool for a host + * @vhost: ibmvfc host who owns the event pool + * + * Returns zero on success. + **/ +static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, + struct ibmvfc_queue *queue) +{ + int i; + struct ibmvfc_event_pool *pool = &queue->evt_pool; + + ENTER; + pool->size = max_requests + IBMVFC_NUM_INTERNAL_REQ; + pool->events = kcalloc(pool->size, sizeof(*pool->events), GFP_KERNEL); + if (!pool->events) + return -ENOMEM; + + pool->iu_storage = dma_alloc_coherent(vhost->dev, + pool->size * sizeof(*pool->iu_storage), + &pool->iu_token, 0); + + if (!pool->iu_storage) { + kfree(pool->events); + return -ENOMEM; + } + + INIT_LIST_HEAD(&queue->sent); + INIT_LIST_HEAD(&queue->free); + spin_lock_init(&queue->l_lock); + + for (i = 0; i < pool->size; ++i) { + struct ibmvfc_event *evt = &pool->events[i]; + + atomic_set(&evt->free, 1); + evt->crq.valid = 0x80; + evt->crq.ioba = cpu_to_be64(pool->iu_token + (sizeof(*evt->xfer_iu) * i)); + evt->xfer_iu = pool->iu_storage + i; + evt->vhost = vhost; + evt->queue = queue; + evt->ext_list = NULL; + list_add_tail(&evt->queue_list, &queue->free); + } + + LEAVE; + return 0; +} + +/** + * ibmvfc_free_event_pool - Frees memory of the event pool of a host + * @vhost: ibmvfc host who owns the event pool + * + **/ +static void ibmvfc_free_event_pool(struct ibmvfc_host *vhost, + struct ibmvfc_queue *queue) +{ + int i; + struct ibmvfc_event_pool *pool = &queue->evt_pool; + + ENTER; + for (i = 0; i < pool->size; ++i) { + list_del(&pool->events[i].queue_list); + BUG_ON(atomic_read(&pool->events[i].free) != 1); + if (pool->events[i].ext_list) + dma_pool_free(vhost->sg_pool, + pool->events[i].ext_list, + pool->events[i].ext_list_token); + } + + kfree(pool->events); + dma_free_coherent(vhost->dev, + pool->size * sizeof(*pool->iu_storage), + pool->iu_storage, pool->iu_token); + LEAVE; +} + /** * ibmvfc_free_queue - Deallocate queue * @vhost: ibmvfc host struct @@ -1312,81 +1388,6 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) strncpy(login_info->drc_name, location, IBMVFC_MAX_NAME); } -/** - * ibmvfc_init_event_pool - Allocates and initializes the event pool for a host - * @vhost: ibmvfc host who owns the event pool - * - * Returns zero on success. - **/ -static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, - struct ibmvfc_queue *queue) -{ - int i; - struct ibmvfc_event_pool *pool = &queue->evt_pool; - - ENTER; - pool->size = max_requests + IBMVFC_NUM_INTERNAL_REQ; - pool->events = kcalloc(pool->size, sizeof(*pool->events), GFP_KERNEL); - if (!pool->events) - return -ENOMEM; - - pool->iu_storage = dma_alloc_coherent(vhost->dev, - pool->size * sizeof(*pool->iu_storage), - &pool->iu_token, 0); - - if (!pool->iu_storage) { - kfree(pool->events); - return -ENOMEM; - } - - INIT_LIST_HEAD(&queue->sent); - INIT_LIST_HEAD(&queue->free); - spin_lock_init(&queue->l_lock); - - for (i = 0; i < pool->size; ++i) { - struct ibmvfc_event *evt = &pool->events[i]; - atomic_set(&evt->free, 1); - evt->crq.valid = 0x80; - evt->crq.ioba = cpu_to_be64(pool->iu_token + (sizeof(*evt->xfer_iu) * i)); - evt->xfer_iu = pool->iu_storage + i; - evt->vhost = vhost; - evt->queue = queue; - evt->ext_list = NULL; - list_add_tail(&evt->queue_list, &queue->free); - } - - LEAVE; - return 0; -} - -/** - * ibmvfc_free_event_pool - Frees memory of the event pool of a host - * @vhost: ibmvfc host who owns the event pool - * - **/ -static void ibmvfc_free_event_pool(struct ibmvfc_host *vhost, - struct ibmvfc_queue *queue) -{ - int i; - struct ibmvfc_event_pool *pool = &queue->evt_pool; - - ENTER; - for (i = 0; i < pool->size; ++i) { - list_del(&pool->events[i].queue_list); - BUG_ON(atomic_read(&pool->events[i].free) != 1); - if (pool->events[i].ext_list) - dma_pool_free(vhost->sg_pool, - pool->events[i].ext_list, - pool->events[i].ext_list_token); - } - - kfree(pool->events); - dma_free_coherent(vhost->dev, - pool->size * sizeof(*pool->iu_storage), - pool->iu_storage, pool->iu_token); - LEAVE; -} - /** * ibmvfc_get_event - Gets the next free event in pool * @vhost: ibmvfc host struct From patchwork Mon Jan 11 23:12:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 293AAC433E6 for ; Tue, 12 Jan 2021 00:31:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07AF822CF7 for ; Tue, 12 Jan 2021 00:31:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391760AbhALAZq (ORCPT ); Mon, 11 Jan 2021 19:25:46 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:34718 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403861AbhAKXNV (ORCPT ); Mon, 11 Jan 2021 18:13:21 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN1QGt093644; Mon, 11 Jan 2021 18:12:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Z0MKHlExerIGTDZ+RFFcZDlZHDy3QWjCBB0jo4OYJqs=; b=hbtQbDgSQQWLx1ClzTuhlRiXljh+zs8HLlMOSYCRhXxaUDzHnhmEYg+TbUEUy5+TDePB oyes9R291cJfW83uI3/sbd8lRUi4i9ddPO2xNQrX4om5FNueCWpk1KiPms5u/u6x8+7h pGFcREhHBGVm/kvTXmmgemChao6ETAtYHgFEbaJS1wXym3XRWJ8qq59uRi1G9crGGbzj OfG4aRhn9n5fBkiSW5knsOyxlJZB0TMh2sxBuA9rbsTynVepXxdlxRuqLthwLPckfebW 6nDGXccGeCyMJ9EMTh+41FQPtd9NRWZ0RgY10FRQZvN0NBa1hJQrM/plvMjObSkKD21T 3Q== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 360wgfmeg3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:31 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN9uPa009926; Mon, 11 Jan 2021 23:12:30 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma02wdc.us.ibm.com with ESMTP id 35y448sy9w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:30 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCT4k27263276 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:29 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3EC8A78063; Mon, 11 Jan 2021 23:12:29 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D15507805E; Mon, 11 Jan 2021 23:12:28 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:28 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 03/21] ibmvfc: init/free event pool during queue allocation/free Date: Mon, 11 Jan 2021 17:12:07 -0600 Message-Id: <20210111231225.105347-4-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 spamscore=0 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The event pool and CRQ used to be separate entities of the adapter host structure and as such were allocated and freed independently of each other. Recent work as defined a generic queue structure with an event pool specific to each queue. As such the event pool for each queue shouldn't be allocated/freed independently, but instead performed as part of the queue allocation/free routines. Move the calls to ibmvfc_event_pool_{init|free} into ibmvfc_{alloc|free}_queue respectively. The only functional change here is that the CRQ cannot be released in ibmvfc_remove until after the event pool has been successfully purged since releasing the queue will also free the event pool. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index cd9273a5fadb..9330f5a65a7e 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -807,6 +807,8 @@ static void ibmvfc_free_queue(struct ibmvfc_host *vhost, dma_unmap_single(dev, queue->msg_token, PAGE_SIZE, DMA_BIDIRECTIONAL); free_page((unsigned long)queue->msgs.handle); queue->msgs.handle = NULL; + + ibmvfc_free_event_pool(vhost, queue); } /** @@ -5019,6 +5021,10 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, switch (fmt) { case IBMVFC_CRQ_FMT: fmt_size = sizeof(*queue->msgs.crq); + if (ibmvfc_init_event_pool(vhost, queue)) { + dev_err(dev, "Couldn't initialize event pool.\n"); + return -ENOMEM; + } break; case IBMVFC_ASYNC_FMT: fmt_size = sizeof(*queue->msgs.async); @@ -5333,13 +5339,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) goto kill_kthread; } - if ((rc = ibmvfc_init_event_pool(vhost, &vhost->crq))) { - dev_err(dev, "Couldn't initialize event pool. rc=%d\n", rc); - goto release_crq; - } - if ((rc = scsi_add_host(shost, dev))) - goto release_event_pool; + goto release_crq; fc_host_dev_loss_tmo(shost) = IBMVFC_DEV_LOSS_TMO; @@ -5362,8 +5363,6 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) remove_shost: scsi_remove_host(shost); -release_event_pool: - ibmvfc_free_event_pool(vhost, &vhost->crq); release_crq: ibmvfc_release_crq_queue(vhost); kill_kthread: @@ -5398,7 +5397,6 @@ static int ibmvfc_remove(struct vio_dev *vdev) spin_unlock_irqrestore(vhost->host->host_lock, flags); ibmvfc_wait_while_resetting(vhost); - ibmvfc_release_crq_queue(vhost); kthread_stop(vhost->work_thread); fc_remove_host(vhost->host); scsi_remove_host(vhost->host); @@ -5408,7 +5406,7 @@ static int ibmvfc_remove(struct vio_dev *vdev) list_splice_init(&vhost->purge, &purge); spin_unlock_irqrestore(vhost->host->host_lock, flags); ibmvfc_complete_purge(&purge); - ibmvfc_free_event_pool(vhost, &vhost->crq); + ibmvfc_release_crq_queue(vhost); ibmvfc_free_mem(vhost); spin_lock(&ibmvfc_driver_lock); From patchwork Mon Jan 11 23:12:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D410C4332E for ; Tue, 12 Jan 2021 00:30:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F127522CB8 for ; Tue, 12 Jan 2021 00:30:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405543AbhALAZt (ORCPT ); Mon, 11 Jan 2021 19:25:49 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:22758 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403943AbhAKXOu (ORCPT ); Mon, 11 Jan 2021 18:14:50 -0500 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN1fbS094000; Mon, 11 Jan 2021 18:12:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=QjcsI1qOMV2f2DMfiZGn6+ursQYzl5WtmahCDPBRtb4=; b=ITWcoSx898ZkAWnxWO6P7zT1k+EtL7IkFsWwLup68su8Vmg/vQX3goLU1b3zEJ7kzwyt 453ylCBVBIo4hQpXeZi5lSk+D+sDJuM27JXRufso+aK9f890T1qRa20QKPHPlpO1dt4u E1DWtDIrOyAErgPVkPiDyCDaEID5vrqh+fRKfxt4QenDpLm3r/7tza92vkhzl5ROxBuk f4iRjl0HFosg8GonEKZNdERYqz418ZHAKMsfnXJcHIVuJduREA6wC2opcQBVn2cObbbx 0JwxLZ/YmNg0BUWk0BOGq7VMewKFUYdl4rN760S4iDfa8gag3voL5aibRNPZpWJzQzMq RQ== Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0b-001b2d01.pphosted.com with ESMTP id 360yexs1by-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:32 -0500 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN8UwZ022002; Mon, 11 Jan 2021 23:12:31 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma04dal.us.ibm.com with ESMTP id 35y448xfh9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:31 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCT9328508434 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:29 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B4DD878063; Mon, 11 Jan 2021 23:12:29 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 527727805F; Mon, 11 Jan 2021 23:12:29 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:29 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 04/21] ibmvfc: add size parameter to ibmvfc_init_event_pool Date: Mon, 11 Jan 2021 17:12:08 -0600 Message-Id: <20210111231225.105347-5-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 mlxscore=0 spamscore=0 bulkscore=0 adultscore=0 suspectscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org With the upcoming addition of Sub-CRQs the event pool size may vary per-queue. Add a size parameter to ibmvfc_init_event_pool such that different size event pools can be requested by ibmvfc_alloc_queue. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 9330f5a65a7e..524e81164d70 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -723,19 +723,23 @@ static int ibmvfc_send_crq_init_complete(struct ibmvfc_host *vhost) * Returns zero on success. **/ static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, - struct ibmvfc_queue *queue) + struct ibmvfc_queue *queue, + unsigned int size) { int i; struct ibmvfc_event_pool *pool = &queue->evt_pool; ENTER; - pool->size = max_requests + IBMVFC_NUM_INTERNAL_REQ; - pool->events = kcalloc(pool->size, sizeof(*pool->events), GFP_KERNEL); + if (!size) + return 0; + + pool->size = size; + pool->events = kcalloc(size, sizeof(*pool->events), GFP_KERNEL); if (!pool->events) return -ENOMEM; pool->iu_storage = dma_alloc_coherent(vhost->dev, - pool->size * sizeof(*pool->iu_storage), + size * sizeof(*pool->iu_storage), &pool->iu_token, 0); if (!pool->iu_storage) { @@ -747,7 +751,7 @@ static int ibmvfc_init_event_pool(struct ibmvfc_host *vhost, INIT_LIST_HEAD(&queue->free); spin_lock_init(&queue->l_lock); - for (i = 0; i < pool->size; ++i) { + for (i = 0; i < size; ++i) { struct ibmvfc_event *evt = &pool->events[i]; atomic_set(&evt->free, 1); @@ -5013,6 +5017,7 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, { struct device *dev = vhost->dev; size_t fmt_size; + unsigned int pool_size = 0; ENTER; spin_lock_init(&queue->_lock); @@ -5021,10 +5026,7 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, switch (fmt) { case IBMVFC_CRQ_FMT: fmt_size = sizeof(*queue->msgs.crq); - if (ibmvfc_init_event_pool(vhost, queue)) { - dev_err(dev, "Couldn't initialize event pool.\n"); - return -ENOMEM; - } + pool_size = max_requests + IBMVFC_NUM_INTERNAL_REQ; break; case IBMVFC_ASYNC_FMT: fmt_size = sizeof(*queue->msgs.async); @@ -5034,6 +5036,11 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, return -EINVAL; } + if (ibmvfc_init_event_pool(vhost, queue, pool_size)) { + dev_err(dev, "Couldn't initialize event pool.\n"); + return -ENOMEM; + } + queue->msgs.handle = (void *)get_zeroed_page(GFP_KERNEL); if (!queue->msgs.handle) return -ENOMEM; From patchwork Mon Jan 11 23:12:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55E4AC4332B for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 397DE22CB8 for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405536AbhALAZq (ORCPT ); Mon, 11 Jan 2021 19:25:46 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:3756 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403856AbhAKXNU (ORCPT ); Mon, 11 Jan 2021 18:13:20 -0500 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN9Fe8028249; Mon, 11 Jan 2021 18:12:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=BcDVb3bASKP8n3XTdtz5NuaFmqzBKGe0LTF+tKXgA6s=; b=dJC8UE+Qn8ezpMR9v4a1FBDNruf36aK7UkDppWX7QLY2DjmvD5uFxMmTtRkfhka6JrGx MLt5aP134olB584DSObpeyisXAsFWg39F5zLcuqYHwpMv8WGJBNDS6ZTUD50tD1dzHXz X2ZJlAQAfOHodhdmXcJBsVXW33wGAVEAKDhH3zcfM6k8Ms2TxxxdGP+whThEDQ9K7DwQ uBoKnWMaIelS76l4GtxinOBlCe2jXlExZy+HgBRqABpfi+5WF5m6FwUdyyee6lT6I+dZ yBkt7ITuUlUKKgC3YgZjjrDHlIkcx5PaBbXH7pfF0G9ScXXhG/8EQ/XyOrWKKWLRtRtF +Q== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 360yc7ry3c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:32 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN8CXs008286; Mon, 11 Jan 2021 23:12:32 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma03dal.us.ibm.com with ESMTP id 35y448xcp0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:31 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCUjP27656678 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:30 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3E7BA78066; Mon, 11 Jan 2021 23:12:30 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C7B6B7805E; Mon, 11 Jan 2021 23:12:29 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:29 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 05/21] ibmvfc: define hcall wrapper for registering a Sub-CRQ Date: Mon, 11 Jan 2021 17:12:09 -0600 Message-Id: <20210111231225.105347-6-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_34:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 clxscore=1015 suspectscore=0 priorityscore=1501 impostorscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110130 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Sub-CRQs are registred with firmware via a hypercall. Abstract that interface into a simpler helper function. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 524e81164d70..612c7f3d7bd3 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -138,6 +138,20 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *); static const char *unknown_error = "unknown error"; +static long h_reg_sub_crq(unsigned long unit_address, unsigned long ioba, + unsigned long length, unsigned long *cookie, + unsigned long *irq) +{ + unsigned long retbuf[PLPAR_HCALL_BUFSIZE]; + long rc; + + rc = plpar_hcall(H_REG_SUB_CRQ, retbuf, unit_address, ioba, length); + *cookie = retbuf[0]; + *irq = retbuf[1]; + + return rc; +} + static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags) { u64 host_caps = be64_to_cpu(vhost->login_buf->resp.capabilities); From patchwork Mon Jan 11 23:12:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEAC8C43331 for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A8F622CF7 for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405533AbhALAZp (ORCPT ); Mon, 11 Jan 2021 19:25:45 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:32696 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403853AbhAKXNU (ORCPT ); Mon, 11 Jan 2021 18:13:20 -0500 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN3Jtv109839; Mon, 11 Jan 2021 18:12:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=KuKVw+l4AKfXXo1oLHlINLaq9JT/aeLO017xRvNUuos=; b=Mh6tAdxax+d4LBQv47QWEwanub4jsCePsvswQ/3fLtE+n8aXEw6AyGT6z1RUoNC5GRxe MRnsnWkmF58qcmYAJClEI51rXhmMirI5ikE0KeWLm7HRelRS5RhFPZIHLTkTDjh6MCIe VtdVoAGw6tmCEQaRbzJ4Ugyv60rIz9GP/kSiM9n85Aso//l0/8IPZLOh4txvecSItIgg mO8hvQFy82Dxf6TS+QQbVVdhF+3QT/5DVvnUvIwPajg9u49SC8HeTc3U9QGpDrEIfY/R gIUrtW2Bl4AmVyMIUnVbYheMzPKi2zGvBh2kVdA/gi1EQEttC31PaTnXf5iou/LY44B/ /g== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 360y9ksbuq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:33 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN7v6d022232; Mon, 11 Jan 2021 23:12:32 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma01dal.us.ibm.com with ESMTP id 35y448xesk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:32 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCUBX28246430 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:30 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B3DBA78068; Mon, 11 Jan 2021 23:12:30 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 51B647805C; Mon, 11 Jan 2021 23:12:30 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:30 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 06/21] ibmvfc: add Subordinate CRQ definitions Date: Mon, 11 Jan 2021 17:12:10 -0600 Message-Id: <20210111231225.105347-7-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 phishscore=0 adultscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 mlxscore=0 spamscore=0 mlxlogscore=999 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110130 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Subordinate Command Response Queues (Sub CRQ) are used in conjunction with the primary CRQ when more than one queue is needed by the virtual IO adapter. Recent phyp firmware versions support Sub CRQ's with ibmvfc adapters. This feature is a prerequisite for supporting multiple hardware backed submission queues in the vfc adapter. The Sub CRQ command element differs from the standard CRQ in that it is 32bytes long as opposed to 16bytes for the latter. Despite this extra 16bytes the ibmvfc protocol will use the original CRQ command element mapped to the first 16bytes of the Sub CRQ element initially. Add definitions for the Sub CRQ command element and queue. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index dd6d89292867..b9eed05c165f 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -650,6 +650,11 @@ struct ibmvfc_crq { volatile __be64 ioba; } __packed __aligned(8); +struct ibmvfc_sub_crq { + struct ibmvfc_crq crq; + __be64 reserved[2]; +} __packed __aligned(8); + enum ibmvfc_ae_link_state { IBMVFC_AE_LS_LINK_UP = 0x01, IBMVFC_AE_LS_LINK_BOUNCED = 0x02, @@ -761,12 +766,14 @@ struct ibmvfc_event_pool { enum ibmvfc_msg_fmt { IBMVFC_CRQ_FMT = 0, IBMVFC_ASYNC_FMT, + IBMVFC_SUB_CRQ_FMT, }; union ibmvfc_msgs { void *handle; struct ibmvfc_crq *crq; struct ibmvfc_async_crq *async; + struct ibmvfc_sub_crq *scrq; }; struct ibmvfc_queue { @@ -781,6 +788,20 @@ struct ibmvfc_queue { struct list_head sent; struct list_head free; spinlock_t l_lock; + + /* Sub-CRQ fields */ + struct ibmvfc_host *vhost; + unsigned long cookie; + unsigned long vios_cookie; + unsigned long hw_irq; + unsigned long irq; + unsigned long hwq_id; + char name[32]; +}; + +struct ibmvfc_scsi_channels { + struct ibmvfc_queue *scrqs; + unsigned int active_queues; }; enum ibmvfc_host_action { From patchwork Mon Jan 11 23:12:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73A11C433E9 for ; Tue, 12 Jan 2021 00:30:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4CF0322CB8 for ; Tue, 12 Jan 2021 00:30:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405541AbhALAZs (ORCPT ); Mon, 11 Jan 2021 19:25:48 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:46724 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403934AbhAKXOn (ORCPT ); Mon, 11 Jan 2021 18:14:43 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN1RE0093692; Mon, 11 Jan 2021 18:12:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=PuJ0PDc+UyiDlIECxar4D5rNl7YB2feQmh7l6C/SZIo=; b=hN8Ix5KmGljYHNh8LLVF/eRlvqXykcIYDBt/SNL1xD0Iilaw9O4Xp9OnDib83/vjOCln cNEMk7LCma45emZS+hPfcMoEfjSReMxcq51mHZmxNOBM4hcN86rdGok8Z/Sy4qE9t+Nc Mlrwcc2lmjoAVJx+DgZ786WguYZPR8GY/XenW7kio6cwvPqTphy87xGnHzIPia4C0l1l JKqbBZjro5vzITlsSyoVsFsiaHBdiJh+3OBeZgp+FZlreE8AcxhxN1n9E1cFFtN9LNIy KhulnOmWTKig78rZpx9ziQvMnnAk2Don/ZIQS4lvO53Ep8CqsK8kg5zraViN6SeGsqe+ lQ== Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 360wgfmegt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:33 -0500 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN7H5B018708; Mon, 11 Jan 2021 23:12:32 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma04wdc.us.ibm.com with ESMTP id 35y448sy6j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:32 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCVbG23134510 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:31 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3452E78063; Mon, 11 Jan 2021 23:12:31 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C61777805C; Mon, 11 Jan 2021 23:12:30 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:30 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 07/21] ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels Date: Mon, 11 Jan 2021 17:12:11 -0600 Message-Id: <20210111231225.105347-8-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 spamscore=0 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Allocate a set of Sub-CRQs in advance. During channel setup the client and VIOS negotiate the number of queues the VIOS supports and the number that the client desires to request. Its possible that the final channel resources allocated is less than requested, but the client is still responsible for sending handles for every queue it is hoping for. Also, provide deallocation cleanup routines. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 125 +++++++++++++++++++++++++++++++++ drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 126 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 612c7f3d7bd3..a198e118887d 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -895,6 +895,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) unsigned long flags; struct vio_dev *vdev = to_vio_dev(vhost->dev); struct ibmvfc_queue *crq = &vhost->crq; + struct ibmvfc_queue *scrq; + int i; /* Close the CRQ */ do { @@ -912,6 +914,16 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) memset(crq->msgs.crq, 0, PAGE_SIZE); crq->cur = 0; + if (vhost->scsi_scrqs.scrqs) { + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + scrq = &vhost->scsi_scrqs.scrqs[i]; + spin_lock(scrq->q_lock); + memset(scrq->msgs.scrq, 0, PAGE_SIZE); + scrq->cur = 0; + spin_unlock(scrq->q_lock); + } + } + /* And re-open it again */ rc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address, crq->msg_token, PAGE_SIZE); @@ -5045,6 +5057,11 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, case IBMVFC_ASYNC_FMT: fmt_size = sizeof(*queue->msgs.async); break; + case IBMVFC_SUB_CRQ_FMT: + fmt_size = sizeof(*queue->msgs.scrq); + /* We need one extra event for Cancel Commands */ + pool_size = max_requests + 1; + break; default: dev_warn(dev, "Unknown command/response queue message format: %d\n", fmt); return -EINVAL; @@ -5136,6 +5153,107 @@ static int ibmvfc_init_crq(struct ibmvfc_host *vhost) return retrc; } +static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, + int index) +{ + struct device *dev = vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + struct ibmvfc_queue *scrq = &vhost->scsi_scrqs.scrqs[index]; + int rc = -ENOMEM; + + ENTER; + + if (ibmvfc_alloc_queue(vhost, scrq, IBMVFC_SUB_CRQ_FMT)) + return -ENOMEM; + + rc = h_reg_sub_crq(vdev->unit_address, scrq->msg_token, PAGE_SIZE, + &scrq->cookie, &scrq->hw_irq); + + if (rc) { + dev_warn(dev, "Error registering sub-crq: %d\n", rc); + if (rc == H_PARAMETER) + dev_warn_once(dev, "Firmware may not support MQ\n"); + goto reg_failed; + } + + scrq->hwq_id = index; + scrq->vhost = vhost; + + LEAVE; + return 0; + +reg_failed: + ibmvfc_free_queue(vhost, scrq); + LEAVE; + return rc; +} + +static void ibmvfc_deregister_scsi_channel(struct ibmvfc_host *vhost, int index) +{ + struct device *dev = vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + struct ibmvfc_queue *scrq = &vhost->scsi_scrqs.scrqs[index]; + long rc; + + ENTER; + + do { + rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, + scrq->cookie); + } while (rc == H_BUSY || H_IS_LONG_BUSY(rc)); + + if (rc) + dev_err(dev, "Failed to free sub-crq[%d]: rc=%ld\n", index, rc); + + ibmvfc_free_queue(vhost, scrq); + LEAVE; +} + +static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) +{ + int i, j; + + ENTER; + + vhost->scsi_scrqs.scrqs = kcalloc(IBMVFC_SCSI_HW_QUEUES, + sizeof(*vhost->scsi_scrqs.scrqs), + GFP_KERNEL); + if (!vhost->scsi_scrqs.scrqs) + return -1; + + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + if (ibmvfc_register_scsi_channel(vhost, i)) { + for (j = i; j > 0; j--) + ibmvfc_deregister_scsi_channel(vhost, j - 1); + kfree(vhost->scsi_scrqs.scrqs); + vhost->scsi_scrqs.scrqs = NULL; + vhost->scsi_scrqs.active_queues = 0; + LEAVE; + return -1; + } + } + + LEAVE; + return 0; +} + +static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) +{ + int i; + + ENTER; + if (!vhost->scsi_scrqs.scrqs) + return; + + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) + ibmvfc_deregister_scsi_channel(vhost, i); + + kfree(vhost->scsi_scrqs.scrqs); + vhost->scsi_scrqs.scrqs = NULL; + vhost->scsi_scrqs.active_queues = 0; + LEAVE; +} + /** * ibmvfc_free_mem - Free memory for vhost * @vhost: ibmvfc host struct @@ -5371,6 +5489,12 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) goto remove_shost; } + if (vhost->mq_enabled) { + rc = ibmvfc_init_sub_crqs(vhost); + if (rc) + dev_warn(dev, "Failed to allocate Sub-CRQs. rc=%d\n", rc); + } + if (shost_to_fc_host(shost)->rqst_q) blk_queue_max_segments(shost_to_fc_host(shost)->rqst_q, 1); dev_set_drvdata(dev, vhost); @@ -5427,6 +5551,7 @@ static int ibmvfc_remove(struct vio_dev *vdev) list_splice_init(&vhost->purge, &purge); spin_unlock_irqrestore(vhost->host->host_lock, flags); ibmvfc_complete_purge(&purge); + ibmvfc_release_sub_crqs(vhost); ibmvfc_release_crq_queue(vhost); ibmvfc_free_mem(vhost); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index b9eed05c165f..bdafe9956649 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -850,6 +850,7 @@ struct ibmvfc_host { mempool_t *tgt_pool; struct ibmvfc_queue crq; struct ibmvfc_queue async_crq; + struct ibmvfc_scsi_channels scsi_scrqs; struct ibmvfc_npiv_login login_info; union ibmvfc_npiv_login_data *login_buf; dma_addr_t login_buf_dma; From patchwork Mon Jan 11 23:12:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D74CDC433E9 for ; Tue, 12 Jan 2021 00:25:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B04FE22D57 for ; Tue, 12 Jan 2021 00:25:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405539AbhALAZs (ORCPT ); Mon, 11 Jan 2021 19:25:48 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:23178 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403932AbhAKXOl (ORCPT ); Mon, 11 Jan 2021 18:14:41 -0500 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN9FR9028234; Mon, 11 Jan 2021 18:12:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=zItf6+mbSfLvIL9pxQZIi/E7XRG+Y4KsHH2EKTlW9iM=; b=AJDI7Ph1hI7r2Sl2nOeovWhDSA1hT0AgMFpUzNFAkj8PglGsdAQKa8N1yRsD0uS9CRK2 AAODp/jRVuf5bXFHUKtAMou4mbBQUxwsgp0YgbVfRYuClt3fxsfO0xPzte06Dc1+X9Ya flHatr+aOVs6V7++/pF9W2W3q4wqHGVfmMKrj1mD3JjPoT4PNRVFNqs/acacabtWIoSn 12qmi01l6ZwzvqLSz7M84o25cDDQKEZHyuWlvh2wPa7UYOYfNNHRR00p/aXnqu1uppF3 rjiJY7d7BuXIfVyTty+ch9kn5KDfzQIg3HeXmGWBh2QXE/7kSdt4V2r8Zpp6Mrbv6wOX 3Q== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 360yc7ry3t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:36 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN7S5I010344; Mon, 11 Jan 2021 23:12:32 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma01wdc.us.ibm.com with ESMTP id 35y448sys5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:32 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCVFD27656670 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:31 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B38D478063; Mon, 11 Jan 2021 23:12:31 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4840F7805C; Mon, 11 Jan 2021 23:12:31 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:31 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 08/21] ibmvfc: add Sub-CRQ IRQ enable/disable routine Date: Mon, 11 Jan 2021 17:12:12 -0600 Message-Id: <20210111231225.105347-9-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_34:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=838 mlxscore=0 lowpriorityscore=0 clxscore=1015 suspectscore=0 priorityscore=1501 impostorscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110130 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Each Sub-CRQ has its own interrupt. A hypercall is required to toggle the IRQ state. Provide the necessary mechanism via a helper function. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index a198e118887d..5d7ada0ed0d6 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3465,6 +3465,26 @@ static void ibmvfc_tasklet(void *data) } } +static int ibmvfc_toggle_scrq_irq(struct ibmvfc_queue *scrq, int enable) +{ + struct device *dev = scrq->vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + unsigned long rc; + int irq_action = H_ENABLE_VIO_INTERRUPT; + + if (!enable) + irq_action = H_DISABLE_VIO_INTERRUPT; + + rc = plpar_hcall_norets(H_VIOCTL, vdev->unit_address, irq_action, + scrq->hw_irq, 0, 0); + + if (rc) + dev_err(dev, "Couldn't %s sub-crq[%lu] irq. rc=%ld\n", + enable ? "enable" : "disable", scrq->hwq_id, rc); + + return rc; +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Mon Jan 11 23:12:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361586 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2AF4C433DB for ; Tue, 12 Jan 2021 00:31:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9FBC722D08 for ; Tue, 12 Jan 2021 00:31:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391762AbhALAZq (ORCPT ); Mon, 11 Jan 2021 19:25:46 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:36022 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403869AbhAKXNV (ORCPT ); Mon, 11 Jan 2021 18:13:21 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN1QpE093616; Mon, 11 Jan 2021 18:12:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=0HO1nQX0OZEtB+IaScUuAUpOS+cC9/9gLWrf8WnhoXY=; b=eYlC6ZFTW3ltO+mAvE+GtxKHw8AAZm8EPHl0mGmdgSrYK32HeDlVB4osOtavtn0dpUxe 3bduO/f7+WplAp4ITRXm7QeLcQ+Zzl1OE2Ydunqeg0F7M04xhlUhhztNp5cYxqpbYNon gThW8I2duvTAlNC4sPZXrd9HyDSxQI9/WLYFDSBVcGPZf//lPvgxXWF9ybX0itcQcXdw XqTGNklmPLjz6uxuCegARAEUcJqqBU990p58zrxkd7gVTsWYv7L00LcOG7YjXRsDdRWq WUvxNT7MtMpYXCI3ixg6SHbajeoN5mD1HE1R2f0TbcQ/uPvWjWc2Xa/JsVV2GhWNrnGQ LA== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 360wgfmehh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:34 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN805p008151; Mon, 11 Jan 2021 23:12:34 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma03dal.us.ibm.com with ESMTP id 35y448xcp3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:34 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCW8L6422890 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:32 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2A7BD78067; Mon, 11 Jan 2021 23:12:32 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C6FC778064; Mon, 11 Jan 2021 23:12:31 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:31 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 09/21] ibmvfc: add handlers to drain and complete Sub-CRQ responses Date: Mon, 11 Jan 2021 17:12:13 -0600 Message-Id: <20210111231225.105347-10-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 spamscore=0 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The logic for iterating over the Sub-CRQ responses is similiar to that of the primary CRQ. Add the necessary handlers for processing those responses. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 86 ++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 5d7ada0ed0d6..f3cd092478ee 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3485,6 +3485,92 @@ static int ibmvfc_toggle_scrq_irq(struct ibmvfc_queue *scrq, int enable) return rc; } +static void ibmvfc_handle_scrq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost, + struct list_head *evt_doneq) +{ + struct ibmvfc_event *evt = (struct ibmvfc_event *)be64_to_cpu(crq->ioba); + + switch (crq->valid) { + case IBMVFC_CRQ_CMD_RSP: + break; + case IBMVFC_CRQ_XPORT_EVENT: + return; + default: + dev_err(vhost->dev, "Got and invalid message type 0x%02x\n", crq->valid); + return; + } + + /* The only kind of payload CRQs we should get are responses to + * things we send. Make sure this response is to something we + * actually sent + */ + if (unlikely(!ibmvfc_valid_event(&evt->queue->evt_pool, evt))) { + dev_err(vhost->dev, "Returned correlation_token 0x%08llx is invalid!\n", + crq->ioba); + return; + } + + if (unlikely(atomic_read(&evt->free))) { + dev_err(vhost->dev, "Received duplicate correlation_token 0x%08llx!\n", + crq->ioba); + return; + } + + spin_lock(&evt->queue->l_lock); + list_move_tail(&evt->queue_list, evt_doneq); + spin_unlock(&evt->queue->l_lock); +} + +static struct ibmvfc_crq *ibmvfc_next_scrq(struct ibmvfc_queue *scrq) +{ + struct ibmvfc_crq *crq; + + crq = &scrq->msgs.scrq[scrq->cur].crq; + if (crq->valid & 0x80) { + if (++scrq->cur == scrq->size) + scrq->cur = 0; + rmb(); + } else + crq = NULL; + + return crq; +} + +static void ibmvfc_drain_sub_crq(struct ibmvfc_queue *scrq) +{ + struct ibmvfc_crq *crq; + struct ibmvfc_event *evt, *temp; + unsigned long flags; + int done = 0; + LIST_HEAD(evt_doneq); + + spin_lock_irqsave(scrq->q_lock, flags); + while (!done) { + while ((crq = ibmvfc_next_scrq(scrq)) != NULL) { + ibmvfc_handle_scrq(crq, scrq->vhost, &evt_doneq); + crq->valid = 0; + wmb(); + } + + ibmvfc_toggle_scrq_irq(scrq, 1); + if ((crq = ibmvfc_next_scrq(scrq)) != NULL) { + ibmvfc_toggle_scrq_irq(scrq, 0); + ibmvfc_handle_scrq(crq, scrq->vhost, &evt_doneq); + crq->valid = 0; + wmb(); + } else + done = 1; + } + spin_unlock_irqrestore(scrq->q_lock, flags); + + list_for_each_entry_safe(evt, temp, &evt_doneq, queue_list) { + del_timer(&evt->timer); + list_del(&evt->queue_list); + ibmvfc_trc_end(evt); + evt->done(evt); + } +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Mon Jan 11 23:12:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3EDDC4332D for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6F69622CB8 for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405534AbhALAZp (ORCPT ); Mon, 11 Jan 2021 19:25:45 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:65212 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403844AbhAKXNU (ORCPT ); Mon, 11 Jan 2021 18:13:20 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN2UOp175352; Mon, 11 Jan 2021 18:12:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=y2i4GWq5LEr5rklEQ6slWMtaroYCAfGe74ncTmgMpHs=; b=iJTiM3e3q/m54PXJUvSrsCAoFFyOx/upTQSbhRAMO0F84qvH6vE37kkUkwfIK8k4njou vvYSz3nLA7xcBJVJ9V8AqHQ6LKS2WejQEt74cdGy3FPoysiemoeWGPOqWmPAE/CMT3kS Gnuz7WeesXWS8uU3CaFPHXbI/cSjruKaMTy37aiWG87OyHd8RKTVh7w5sgHoqmtnVX6V kkOjFWGkUAxhPPlunc6pyfFIxVfl6Txe8fSXB5I5eE7Yn4Nql5p05by6UOY/C+3XXGJF Jhn8l88nUdFGTJKBzVzDDpfx7QxeNYGA0tfaO4/IIKDoYUURS6TWv/KVqbb1OURHOUWd Kw== Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 360yvkrf42-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:35 -0500 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN89Rf011995; Mon, 11 Jan 2021 23:12:34 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma05wdc.us.ibm.com with ESMTP id 35y448sxff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:34 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCWs28323354 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:32 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B28357805C; Mon, 11 Jan 2021 23:12:32 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 479D078064; Mon, 11 Jan 2021 23:12:32 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:32 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 10/21] ibmvfc: define Sub-CRQ interrupt handler routine Date: Mon, 11 Jan 2021 17:12:14 -0600 Message-Id: <20210111231225.105347-11-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 lowpriorityscore=0 phishscore=0 priorityscore=1501 suspectscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Simple handler that calls Sub-CRQ drain routine directly. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index f3cd092478ee..51bcafad9490 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3571,6 +3571,16 @@ static void ibmvfc_drain_sub_crq(struct ibmvfc_queue *scrq) } } +static irqreturn_t ibmvfc_interrupt_scsi(int irq, void *scrq_instance) +{ + struct ibmvfc_queue *scrq = (struct ibmvfc_queue *)scrq_instance; + + ibmvfc_toggle_scrq_irq(scrq, 0); + ibmvfc_drain_sub_crq(scrq); + + return IRQ_HANDLED; +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Mon Jan 11 23:12:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362576 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1768EC433E0 for ; Tue, 12 Jan 2021 00:30:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E038922CF7 for ; Tue, 12 Jan 2021 00:30:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405540AbhALAZs (ORCPT ); Mon, 11 Jan 2021 19:25:48 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:40414 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403937AbhAKXOn (ORCPT ); Mon, 11 Jan 2021 18:14:43 -0500 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN1fA7094032; Mon, 11 Jan 2021 18:12:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=hZeV9JzufbxYww9v6JS4tA3P/nR0IZfx3kmIVf3DC8U=; b=VG7ab5UoE+/4X3JX3wTBQQ9+8yiSJOF8x5fPEPsd/A5u/hTKq0NTKjw8JtOZbyC23g+X a6FvlAjM5ijiSWyxqCmjVNo7pSobH2Jm8J8Q+EFZ/2WwoHuzU1VRCk/6UkGg3AWLYDHG rzIW08lfFo9/Dm5Xzpn6AmiZG3lO5Qi4r8+O2Uf2yRInql+x0BS1J1IhC2Tgyhfd7lIW kbW2HLZRJxHhl7H0luCT3Nr/THj9EUeJ/J6VTiz5OHPf6FqAKs0FWj36J6ciUIru8v8m G8YFdeWWsffaG7emJUf6L61bCCTBho6pNM19RK15uVY69wRTZMQgjMW9RARd4piedt9S 7g== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0b-001b2d01.pphosted.com with ESMTP id 360yexs1dp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:37 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN7PLV010308; Mon, 11 Jan 2021 23:12:36 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma01wdc.us.ibm.com with ESMTP id 35y448sys9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:36 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCXVm37159268 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:33 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 32ACF78064; Mon, 11 Jan 2021 23:12:33 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C573D78067; Mon, 11 Jan 2021 23:12:32 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:32 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 11/21] ibmvfc: map/request irq and register Sub-CRQ interrupt handler Date: Mon, 11 Jan 2021 17:12:15 -0600 Message-Id: <20210111231225.105347-12-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 mlxscore=0 spamscore=0 bulkscore=0 adultscore=0 suspectscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Create an irq mapping for the hw_irq number provided from phyp firmware. Request an irq assigned our Sub-CRQ interrupt handler. Unmap these irqs at Sub-CRQ teardown. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 51bcafad9490..d3d7c6b53d4f 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -5292,12 +5292,34 @@ static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, goto reg_failed; } + scrq->irq = irq_create_mapping(NULL, scrq->hw_irq); + + if (!scrq->irq) { + rc = -EINVAL; + dev_err(dev, "Error mapping sub-crq[%d] irq\n", index); + goto irq_failed; + } + + snprintf(scrq->name, sizeof(scrq->name), "ibmvfc-%x-scsi%d", + vdev->unit_address, index); + rc = request_irq(scrq->irq, ibmvfc_interrupt_scsi, 0, scrq->name, scrq); + + if (rc) { + dev_err(dev, "Couldn't register sub-crq[%d] irq\n", index); + irq_dispose_mapping(scrq->irq); + goto irq_failed; + } + scrq->hwq_id = index; scrq->vhost = vhost; LEAVE; return 0; +irq_failed: + do { + plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, scrq->cookie); + } while (rc == H_BUSY || H_IS_LONG_BUSY(rc)); reg_failed: ibmvfc_free_queue(vhost, scrq); LEAVE; @@ -5313,6 +5335,9 @@ static void ibmvfc_deregister_scsi_channel(struct ibmvfc_host *vhost, int index) ENTER; + free_irq(scrq->irq, scrq); + irq_dispose_mapping(scrq->irq); + do { rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, scrq->cookie); From patchwork Mon Jan 11 23:12:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361584 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7191FC4332E for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 521F222CF7 for ; Tue, 12 Jan 2021 00:33:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405535AbhALAZp (ORCPT ); Mon, 11 Jan 2021 19:25:45 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:31920 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403848AbhAKXNU (ORCPT ); Mon, 11 Jan 2021 18:13:20 -0500 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN3Ifb140805; Mon, 11 Jan 2021 18:12:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=huvnYSSCUo/Im06yfp4ZIAX216Nn8Rmul7bljlbP4Vc=; b=DzEF1qUYqBdwRbL3r3Og7tfuinGbVJ9Rpvu0dtkKpAWkUXNnBJvQGFC+jxwfeRq3pxGa NQduer95E0mi0WXBZ1pTH/oM5YTaF3PkYHftoeB0lp6dD53dXWp7vZJI0e61XqW/tLtF 8bhub/ShaAxnv9bdcKIh8ZUCNIht06vIG973S3A85A4SU2Xe2ca88dNxSHLGmt88P0+K boYiT4nqHDwFonzUEL2OSMjwTBurTxm7z6tFBDTQjAnH0GbFofAZ+w8upXEPHzVqVTcK 1//7RIiLIBOyumJAVhhfuVH5CfTG1fsSbWyTDQg15/hru0nrynJ3am4dquSGA/a5laNJ mQ== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 361001r8rt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:35 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN8CXt008286; Mon, 11 Jan 2021 23:12:35 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma03dal.us.ibm.com with ESMTP id 35y448xcp8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:35 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCX3j23790018 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:33 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A78ED7805C; Mon, 11 Jan 2021 23:12:33 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 464BB7805F; Mon, 11 Jan 2021 23:12:33 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:33 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 12/21] ibmvfc: implement channel enquiry and setup commands Date: Mon, 11 Jan 2021 17:12:16 -0600 Message-Id: <20210111231225.105347-13-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_34:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 clxscore=1015 suspectscore=0 phishscore=0 impostorscore=0 mlxlogscore=999 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110130 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org New NPIV_ENQUIRY_CHANNEL and NPIV_SETUP_CHANNEL management datagrams (MADs) were defined in a previous patchset. If the client advertises a desire to use channels and the partner VIOS is channel capable then the client must proceed with channel enquiry to determine the maximum number of channels the VIOS is capable of providing, and registering SubCRQs via channel setup with the VIOS immediately following NPIV Login. This handshaking should not be performed for subsequent NPIV Logins unless the CRQ connection has been reset. Implement these two new MADs and issue them following a successful NPIV login where the VIOS has set the SUPPORT_CHANNELS capability bit in the NPIV Login response. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 135 ++++++++++++++++++++++++++++++++- drivers/scsi/ibmvscsi/ibmvfc.h | 3 + 2 files changed, 136 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index d3d7c6b53d4f..a00f38558613 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -909,6 +909,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) spin_lock(vhost->crq.q_lock); vhost->state = IBMVFC_NO_CRQ; vhost->logged_in = 0; + vhost->do_enquiry = 1; + vhost->using_channels = 0; /* Clean out the queue */ memset(crq->msgs.crq, 0, PAGE_SIZE); @@ -4586,6 +4588,118 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost) ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); } +static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) +{ + struct ibmvfc_host *vhost = evt->vhost; + u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); + int level = IBMVFC_DEFAULT_LOG_LEVEL; + + ibmvfc_free_event(evt); + + switch (mad_status) { + case IBMVFC_MAD_SUCCESS: + ibmvfc_dbg(vhost, "Channel Setup succeded\n"); + vhost->do_enquiry = 0; + break; + case IBMVFC_MAD_FAILED: + level += ibmvfc_retry_host_init(vhost); + ibmvfc_log(vhost, level, "Channel Setup failed\n"); + fallthrough; + case IBMVFC_MAD_DRIVER_FAILED: + return; + default: + dev_err(vhost->dev, "Invalid Channel Setup response: 0x%x\n", + mad_status); + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); + return; + } + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); + wake_up(&vhost->work_wait_q); +} + +static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) +{ + struct ibmvfc_channel_setup_mad *mad; + struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf; + struct ibmvfc_event *evt = ibmvfc_get_event(&vhost->crq); + + memset(setup_buf, 0, sizeof(*setup_buf)); + setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + + ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); + mad = &evt->iu.channel_setup; + memset(mad, 0, sizeof(*mad)); + mad->common.version = cpu_to_be32(1); + mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_SETUP); + mad->common.length = cpu_to_be16(sizeof(*mad)); + mad->buffer.va = cpu_to_be64(vhost->channel_setup_dma); + mad->buffer.len = cpu_to_be32(sizeof(*vhost->channel_setup_buf)); + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); + + if (!ibmvfc_send_event(evt, vhost, default_timeout)) + ibmvfc_dbg(vhost, "Sent channel setup\n"); + else + ibmvfc_link_down(vhost, IBMVFC_LINK_DOWN); +} + +static void ibmvfc_channel_enquiry_done(struct ibmvfc_event *evt) +{ + struct ibmvfc_host *vhost = evt->vhost; + struct ibmvfc_channel_enquiry *rsp = &evt->xfer_iu->channel_enquiry; + u32 mad_status = be16_to_cpu(rsp->common.status); + int level = IBMVFC_DEFAULT_LOG_LEVEL; + + switch (mad_status) { + case IBMVFC_MAD_SUCCESS: + ibmvfc_dbg(vhost, "Channel Enquiry succeeded\n"); + vhost->max_vios_scsi_channels = be32_to_cpu(rsp->num_scsi_subq_channels); + ibmvfc_free_event(evt); + break; + case IBMVFC_MAD_FAILED: + level += ibmvfc_retry_host_init(vhost); + ibmvfc_log(vhost, level, "Channel Enquiry failed\n"); + fallthrough; + case IBMVFC_MAD_DRIVER_FAILED: + ibmvfc_free_event(evt); + return; + default: + dev_err(vhost->dev, "Invalid Channel Enquiry response: 0x%x\n", + mad_status); + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); + ibmvfc_free_event(evt); + return; + } + + ibmvfc_channel_setup(vhost); +} + +static void ibmvfc_channel_enquiry(struct ibmvfc_host *vhost) +{ + struct ibmvfc_channel_enquiry *mad; + struct ibmvfc_event *evt = ibmvfc_get_event(&vhost->crq); + + ibmvfc_init_event(evt, ibmvfc_channel_enquiry_done, IBMVFC_MAD_FORMAT); + mad = &evt->iu.channel_enquiry; + memset(mad, 0, sizeof(*mad)); + mad->common.version = cpu_to_be32(1); + mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_ENQUIRY); + mad->common.length = cpu_to_be16(sizeof(*mad)); + + if (IBMVFC_MIG_NO_SUB_TO_CRQ) + mad->flags |= cpu_to_be32(IBMVFC_NO_CHANNELS_TO_CRQ_SUPPORT); + if (IBMVFC_MIG_NO_N_TO_M) + mad->flags |= cpu_to_be32(IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT); + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); + + if (!ibmvfc_send_event(evt, vhost, default_timeout)) + ibmvfc_dbg(vhost, "Send channel enquiry\n"); + else + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); +} + /** * ibmvfc_npiv_login_done - Completion handler for NPIV Login * @evt: ibmvfc event struct @@ -4667,8 +4781,14 @@ static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt) vhost->host->can_queue = be32_to_cpu(rsp->max_cmds) - IBMVFC_NUM_INTERNAL_REQ; vhost->host->max_sectors = npiv_max_sectors; - ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); - wake_up(&vhost->work_wait_q); + + if (ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPORT_CHANNELS) && vhost->do_enquiry) { + ibmvfc_channel_enquiry(vhost); + } else { + vhost->do_enquiry = 0; + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); + wake_up(&vhost->work_wait_q); + } } /** @@ -5477,9 +5597,20 @@ static int ibmvfc_alloc_mem(struct ibmvfc_host *vhost) goto free_trace; } + vhost->channel_setup_buf = dma_alloc_coherent(dev, sizeof(*vhost->channel_setup_buf), + &vhost->channel_setup_dma, + GFP_KERNEL); + + if (!vhost->channel_setup_buf) { + dev_err(dev, "Couldn't allocate Channel Setup buffer\n"); + goto free_tgt_pool; + } + LEAVE; return 0; +free_tgt_pool: + mempool_destroy(vhost->tgt_pool); free_trace: kfree(vhost->trace); free_disc_buffer: diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index bdafe9956649..3d76cd3c1fd9 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -854,10 +854,13 @@ struct ibmvfc_host { struct ibmvfc_npiv_login login_info; union ibmvfc_npiv_login_data *login_buf; dma_addr_t login_buf_dma; + struct ibmvfc_channel_setup *channel_setup_buf; + dma_addr_t channel_setup_dma; int disc_buf_sz; int log_level; struct ibmvfc_discover_targets_entry *disc_buf; struct mutex passthru_mutex; + int max_vios_scsi_channels; int task_set; int init_retries; int discovery_threads; From patchwork Mon Jan 11 23:12:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8E92C433DB for ; Tue, 12 Jan 2021 00:31:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 822DE22D08 for ; Tue, 12 Jan 2021 00:31:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403861AbhALAZr (ORCPT ); Mon, 11 Jan 2021 19:25:47 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:26504 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403876AbhAKXNX (ORCPT ); Mon, 11 Jan 2021 18:13:23 -0500 Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN2UGm146087; Mon, 11 Jan 2021 18:12:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=djK3tr0YW8t8Pyhv6WxA+bpEQW8q4wEvYVFB5wuM0Uw=; b=sgJLMJbzaLbFXZ5xwGVXiKtXFonyhVix8aAhZlgpOLR1XoXPKtJhnjLfLPiurGmbZ6kV /LFwyrRL+cmgkTqrLuX6Ggn5v/d3neQLDX26DYRmE25VAZ4PeAaTZIxrj0Ercg6nToxC NQZm9Mhg61vTfG4GsiMZnpLb98m0J0dmBHvehgrYdC5a3xoMD379cRg7762OON5WceGr 4lTp4x1C2r3ww9urHW6udCsxyINv4hhZQZeZ7k7zNsNUln0mi74/X6c05bLfK2DrbyZD WaNgJP/TwUtwvd/wZpIcURIi94pxZ2m5Y04pvkNWp/4FJDgCa8VzTL3WLOVdS0g804dD fQ== Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 360y93s7xh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:36 -0500 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN7Fx1018701; Mon, 11 Jan 2021 23:12:35 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma04wdc.us.ibm.com with ESMTP id 35y448sy6u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:35 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCYTp23790060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:34 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 286A078066; Mon, 11 Jan 2021 23:12:34 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BB0437805F; Mon, 11 Jan 2021 23:12:33 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:33 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 13/21] ibmvfc: advertise client support for using hardware channels Date: Mon, 11 Jan 2021 17:12:17 -0600 Message-Id: <20210111231225.105347-14-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 adultscore=0 bulkscore=0 suspectscore=0 mlxscore=0 spamscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 phishscore=0 mlxlogscore=999 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Previous patches have plumbed the necessary Sub-CRQ interface and channel negotiation MADs to fully channelize via hardware backed queues. Advertise client support via NPIV Login capability IBMVFC_CAN_USE_CHANNELS when the client bits have MQ enabled via vhost->mq_enabled, or when channels were already in use during a subsequent NPIV Login. The later is required because channel support is only renegotiated after a CRQ pair is broken. Simple NPIV Logout/Logins require the client to continue to advertise the channel capability until the CRQ pair between the client is broken. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index a00f38558613..0653d52d4ea0 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1410,6 +1410,10 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) login_info->max_cmds = cpu_to_be32(max_requests + IBMVFC_NUM_INTERNAL_REQ); login_info->capabilities = cpu_to_be64(IBMVFC_CAN_MIGRATE | IBMVFC_CAN_SEND_VF_WWPN); + + if (vhost->mq_enabled || vhost->using_channels) + login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_CHANNELS); + login_info->async.va = cpu_to_be64(vhost->async_crq.msg_token); login_info->async.len = cpu_to_be32(async_crq->size * sizeof(*async_crq->msgs.async)); From patchwork Mon Jan 11 23:12:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D647C433E0 for ; Tue, 12 Jan 2021 00:33:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7680722CB8 for ; Tue, 12 Jan 2021 00:33:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405537AbhALAZq (ORCPT ); Mon, 11 Jan 2021 19:25:46 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:11588 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403871AbhAKXNV (ORCPT ); Mon, 11 Jan 2021 18:13:21 -0500 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN3Isd109804; Mon, 11 Jan 2021 18:12:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=nLItLeTuMlDJB3G/lzt/ljE/4A0U0/fxclf93e4Nm4g=; b=L3g27lSruPTGMLuJCTa2hgrOAQi14F95aHTXKNzWrYrgqiRf41wuolIVU358QFAPNL+i mLa7OGLenSzVLS0bzPuMQtwB0qm/BaM1JTM3l9AtP2DGoeLZrS5I6bSdeqsNI0xoYYmQ anedBgHOc9QpP7nqT2ZbxdEzeYicjlQfrnrnjbjlsl0RLrtm7+zTefjLqHgFqmAIc46K NvjR1qsnOHHJ+q1rbfseVWIyLThd5k8wRHzaNFY7/Q9jCjRH6uQhMuccVS3kfrYKPHlv blJ8syXsjBy8qOQtkbVHwS09AovLhkMulKpVpWnyAp0t9WE9V3mKfIx7ZVdRjD2keXVw Ig== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 360y9ksbvt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:37 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN9s7R009902; Mon, 11 Jan 2021 23:12:36 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma02wdc.us.ibm.com with ESMTP id 35y448syaa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:35 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCYxF18940354 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:34 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9CCC978063; Mon, 11 Jan 2021 23:12:34 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3B9557805C; Mon, 11 Jan 2021 23:12:34 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:34 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 14/21] ibmvfc: set and track hw queue in ibmvfc_event struct Date: Mon, 11 Jan 2021 17:12:18 -0600 Message-Id: <20210111231225.105347-15-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 phishscore=0 adultscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 mlxscore=0 spamscore=0 mlxlogscore=999 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110130 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Extract the hwq id from a SCSI command and store it in the ibmvfc_event structure to identify which Sub-CRQ to send the command down when channels are being utilized. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 5 +++++ drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 0653d52d4ea0..3f3cc37a263f 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1483,6 +1483,7 @@ static void ibmvfc_init_event(struct ibmvfc_event *evt, evt->_done = done; evt->done = ibmvfc_locked_done; } + evt->hwq = 0; } /** @@ -1839,6 +1840,8 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) struct ibmvfc_cmd *vfc_cmd; struct ibmvfc_fcp_cmd_iu *iu; struct ibmvfc_event *evt; + u32 tag_and_hwq = blk_mq_unique_tag(cmnd->request); + u16 hwq = blk_mq_unique_tag_to_hwq(tag_and_hwq); int rc; if (unlikely((rc = fc_remote_port_chkready(rport))) || @@ -1865,6 +1868,8 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) } vfc_cmd->correlation = cpu_to_be64((u64)evt); + if (vhost->using_channels) + evt->hwq = hwq % vhost->scsi_scrqs.active_queues; if (likely(!(rc = ibmvfc_map_sg_data(cmnd, evt, vfc_cmd, vhost->dev)))) return ibmvfc_send_event(evt, vhost, 0); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 3d76cd3c1fd9..2dbce7071409 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -753,6 +753,7 @@ struct ibmvfc_event { struct completion comp; struct completion *eh_comp; struct timer_list timer; + u16 hwq; }; /* a pool of event structs for use */ From patchwork Mon Jan 11 23:12:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0113C43332 for ; Tue, 12 Jan 2021 00:30:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A96CC22CF7 for ; Tue, 12 Jan 2021 00:30:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405542AbhALAZt (ORCPT ); Mon, 11 Jan 2021 19:25:49 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:12754 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403938AbhAKXOs (ORCPT ); Mon, 11 Jan 2021 18:14:48 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN1QnQ093660; Mon, 11 Jan 2021 18:12:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=tRxjrYeZ9O21Nei950L0agvbsa0y2iWrGrLttEO7+gU=; b=s05XxKIty3kqsE9Bh4XGLX4AqENk1qPcoeYznAaOVHF2pYlgSxGEcXsZ2+/7HhILOW9x EV/JTQo5idHLzw1mJ/ImeFf6VoQnd4W8OCez7VHDSfOgz+5Q1dsjUeXxJk68Llb5XjsH Id6f8lMuuUdRWzGuGrCrcstn+c5Vs+jSFMtRJUVnqRj0EjL6dpBCUyaggKJVHPvVUtvY iSBF0f4hBL23z3Ft0m0Gpf5BDXiioi2ORnM1Uzi3LbDZn3tmlMZ5yaKJZtBnfu34Hca9 aTD3dXmeLjHTVQroUrKyfiM1Grrt8TbOQew+vg4VCf1tZEiRz8LCHcEH8FsRHqDenLJJ BQ== Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0b-001b2d01.pphosted.com with ESMTP id 360wgfmeja-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:37 -0500 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN81ms025543; Mon, 11 Jan 2021 23:12:37 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma02dal.us.ibm.com with ESMTP id 35y4496e2n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:36 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCZHC29360520 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:35 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1D9AA78063; Mon, 11 Jan 2021 23:12:35 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B05527805C; Mon, 11 Jan 2021 23:12:34 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:34 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 15/21] ibmvfc: send commands down HW Sub-CRQ when channelized Date: Mon, 11 Jan 2021 17:12:19 -0600 Message-Id: <20210111231225.105347-16-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 spamscore=0 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org When the client has negotiated the use of channels all vfcFrames are required to go down a Sub-CRQ channel or it is a protocoal violation. If the adapter state is channelized submit vfcFrames to the appropriate Sub-CRQ via the h_send_sub_crq() helper. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 39 ++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 3f3cc37a263f..865b87881d86 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -704,6 +704,15 @@ static int ibmvfc_send_crq(struct ibmvfc_host *vhost, u64 word1, u64 word2) return plpar_hcall_norets(H_SEND_CRQ, vdev->unit_address, word1, word2); } +static int ibmvfc_send_sub_crq(struct ibmvfc_host *vhost, u64 cookie, u64 word1, + u64 word2, u64 word3, u64 word4) +{ + struct vio_dev *vdev = to_vio_dev(vhost->dev); + + return plpar_hcall_norets(H_SEND_SUB_CRQ, vdev->unit_address, cookie, + word1, word2, word3, word4); +} + /** * ibmvfc_send_crq_init - Send a CRQ init message * @vhost: ibmvfc host struct @@ -1623,8 +1632,17 @@ static int ibmvfc_send_event(struct ibmvfc_event *evt, mb(); - if ((rc = ibmvfc_send_crq(vhost, be64_to_cpu(crq_as_u64[0]), - be64_to_cpu(crq_as_u64[1])))) { + if (evt->queue->fmt == IBMVFC_SUB_CRQ_FMT) + rc = ibmvfc_send_sub_crq(vhost, + evt->queue->vios_cookie, + be64_to_cpu(crq_as_u64[0]), + be64_to_cpu(crq_as_u64[1]), + 0, 0); + else + rc = ibmvfc_send_crq(vhost, be64_to_cpu(crq_as_u64[0]), + be64_to_cpu(crq_as_u64[1])); + + if (rc) { list_del(&evt->queue_list); spin_unlock_irqrestore(&evt->queue->l_lock, flags); del_timer(&evt->timer); @@ -1842,6 +1860,7 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) struct ibmvfc_event *evt; u32 tag_and_hwq = blk_mq_unique_tag(cmnd->request); u16 hwq = blk_mq_unique_tag_to_hwq(tag_and_hwq); + u16 scsi_channel; int rc; if (unlikely((rc = fc_remote_port_chkready(rport))) || @@ -1852,7 +1871,13 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) } cmnd->result = (DID_OK << 16); - evt = ibmvfc_get_event(&vhost->crq); + if (vhost->using_channels) { + scsi_channel = hwq % vhost->scsi_scrqs.active_queues; + evt = ibmvfc_get_event(&vhost->scsi_scrqs.scrqs[scsi_channel]); + evt->hwq = hwq % vhost->scsi_scrqs.active_queues; + } else + evt = ibmvfc_get_event(&vhost->crq); + ibmvfc_init_event(evt, ibmvfc_scsi_done, IBMVFC_CMD_FORMAT); evt->cmnd = cmnd; @@ -1868,8 +1893,6 @@ static int ibmvfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) } vfc_cmd->correlation = cpu_to_be64((u64)evt); - if (vhost->using_channels) - evt->hwq = hwq % vhost->scsi_scrqs.active_queues; if (likely(!(rc = ibmvfc_map_sg_data(cmnd, evt, vfc_cmd, vhost->dev)))) return ibmvfc_send_event(evt, vhost, 0); @@ -2200,7 +2223,11 @@ static int ibmvfc_reset_device(struct scsi_device *sdev, int type, char *desc) spin_lock_irqsave(vhost->host->host_lock, flags); if (vhost->state == IBMVFC_ACTIVE) { - evt = ibmvfc_get_event(&vhost->crq); + if (vhost->using_channels) + evt = ibmvfc_get_event(&vhost->scsi_scrqs.scrqs[0]); + else + evt = ibmvfc_get_event(&vhost->crq); + ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_CMD_FORMAT); tmf = ibmvfc_init_vfc_cmd(evt, sdev); iu = ibmvfc_get_fcp_iu(vhost, tmf); From patchwork Mon Jan 11 23:12:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361588 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 110ECC433E6 for ; Tue, 12 Jan 2021 00:31:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7BDE22CB8 for ; Tue, 12 Jan 2021 00:31:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403869AbhALAZr (ORCPT ); Mon, 11 Jan 2021 19:25:47 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:65332 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403877AbhAKXNY (ORCPT ); Mon, 11 Jan 2021 18:13:24 -0500 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BMo3GR040130; Mon, 11 Jan 2021 18:12:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=+PynfPSrmwGMPIiDCYzkA6Jm1QrL5Oi+bjFx8FNL/4M=; b=VOgpuEAZNiCbGG4WEJTsjiYFFbMciT6PhnNKe8hz8VRAW1mr1JFoeVg4EwGkynTwtf5K Dor61d9I9Rj329AnNJRO7DOI+pRKRBHVYcWJHu8dsTpwO0Clmt7Ax9M5flWTluH5xPbj FG7oI5cSDqzQvcSgWwLbvTbTVSD3BXGqD2t+vRnuIS5dmvbTESt6jGdHUdGWdSArU05S XHoPd7uQWzYIdDtix/dftDhZYQ8vTXFwF9SW0V35FxwAyuRLUt0sjalXr9E/JtpYHI6M RrRJFFLGoGzbobBv6vq3IBrcUaPK+9873jD2TNFewzWew+fma+lkwxpEI7WCjmWphYit 2g== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 360ytk8gjv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:37 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN80g3008161; Mon, 11 Jan 2021 23:12:37 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma03dal.us.ibm.com with ESMTP id 35y448xcpf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:37 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCZOL25952696 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:35 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 962E378064; Mon, 11 Jan 2021 23:12:35 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 324077805C; Mon, 11 Jan 2021 23:12:35 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:35 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 16/21] ibmvfc: register Sub-CRQ handles with VIOS during channel setup Date: Mon, 11 Jan 2021 17:12:20 -0600 Message-Id: <20210111231225.105347-17-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 spamscore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 malwarescore=0 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org If the ibmvfc client adapter requests channels it must submit a number of Sub-CRQ handles matching the number of channels being requested. The VIOS in its response will overwrite the actual number of channel resources allocated which may be less than what was requested. The client then must store the VIOS Sub-CRQ handle for each queue. This VIOS handle is needed as a parameter with h_send_sub_crq(). Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 865b87881d86..578e27180f10 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -4627,15 +4627,35 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost) static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) { struct ibmvfc_host *vhost = evt->vhost; + struct ibmvfc_channel_setup *setup = vhost->channel_setup_buf; + struct ibmvfc_scsi_channels *scrqs = &vhost->scsi_scrqs; u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); int level = IBMVFC_DEFAULT_LOG_LEVEL; + int flags, active_queues, i; ibmvfc_free_event(evt); switch (mad_status) { case IBMVFC_MAD_SUCCESS: ibmvfc_dbg(vhost, "Channel Setup succeded\n"); + flags = be32_to_cpu(setup->flags); vhost->do_enquiry = 0; + active_queues = be32_to_cpu(setup->num_scsi_subq_channels); + scrqs->active_queues = active_queues; + + if (flags & IBMVFC_CHANNELS_CANCELED) { + ibmvfc_dbg(vhost, "Channels Canceled\n"); + vhost->using_channels = 0; + } else { + if (active_queues) + vhost->using_channels = 1; + for (i = 0; i < active_queues; i++) + scrqs->scrqs[i].vios_cookie = + be64_to_cpu(setup->channel_handles[i]); + + ibmvfc_dbg(vhost, "Using %u channels\n", + vhost->scsi_scrqs.active_queues); + } break; case IBMVFC_MAD_FAILED: level += ibmvfc_retry_host_init(vhost); @@ -4659,9 +4679,19 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) struct ibmvfc_channel_setup_mad *mad; struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf; struct ibmvfc_event *evt = ibmvfc_get_event(&vhost->crq); + struct ibmvfc_scsi_channels *scrqs = &vhost->scsi_scrqs; + unsigned int num_channels = + min(vhost->client_scsi_channels, vhost->max_vios_scsi_channels); + int i; memset(setup_buf, 0, sizeof(*setup_buf)); - setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + if (num_channels == 0) + setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + else { + setup_buf->num_scsi_subq_channels = cpu_to_be32(num_channels); + for (i = 0; i < num_channels; i++) + setup_buf->channel_handles[i] = cpu_to_be64(scrqs->scrqs[i].cookie); + } ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); mad = &evt->iu.channel_setup; From patchwork Mon Jan 11 23:12:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F6BCC433E0 for ; Tue, 12 Jan 2021 00:31:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 17D4522D08 for ; Tue, 12 Jan 2021 00:31:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391766AbhALAZr (ORCPT ); Mon, 11 Jan 2021 19:25:47 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:23044 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403872AbhAKXNX (ORCPT ); Mon, 11 Jan 2021 18:13:23 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN2UBG175401; Mon, 11 Jan 2021 18:12:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=GmNvDU70WRRDHGJO5ab1fmTFybGNSQl2UdTKfI7awiA=; b=tJgxcxYtkfnXs6IVD/eBdT0dQl+tDnTPOk6AFTab02hgb+Tauz/9fKDeWfWvGM2T14dn aqZMipMs3zqICG/tSn3cCvOPQ/mfaROcc/fEqJtPjilDUkgYMNMnJW+hGwNzrlxCytzL 0xvGgzyD4jPDraTzexOSAcdnFmqD1BIJwl7+W/YV2IiEpUOpCuRWSbVXQfUzbWteYq9F 0JEPV3KkUdO3GyRVeULyGAjzXup7q41Zxa4YNgk+R4i2po0lQiTABl0gHPonSKh7Pa1U 1iIvVJj50I6oLKBKbo8d5AIOetDuND7K3Y3YQhiCmakTmzr7zTapHQy0xg+YrFwCH2xf SQ== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 360yvkrf51-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:38 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN7xjI008136; Mon, 11 Jan 2021 23:12:37 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma03dal.us.ibm.com with ESMTP id 35y448xcph-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:37 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCa7v23855526 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:36 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 164407805F; Mon, 11 Jan 2021 23:12:36 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A962C78067; Mon, 11 Jan 2021 23:12:35 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:35 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 17/21] ibmvfc: add cancel mad initialization helper Date: Mon, 11 Jan 2021 17:12:21 -0600 Message-Id: <20210111231225.105347-18-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 lowpriorityscore=0 phishscore=0 priorityscore=1501 suspectscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add a helper routine for initializing a Cancel MAD. This will be useful for a channelized client that needs to send Cancel commands down every channel commands were sent for a particular LUN. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 68 ++++++++++++++++++++-------------- 1 file changed, 40 insertions(+), 28 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 578e27180f10..b0b0212344f3 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -2379,6 +2379,45 @@ static int ibmvfc_wait_for_ops(struct ibmvfc_host *vhost, void *device, return SUCCESS; } +static struct ibmvfc_event *ibmvfc_init_tmf(struct ibmvfc_queue *queue, + struct scsi_device *sdev, + int type) +{ + struct ibmvfc_host *vhost = shost_priv(sdev->host); + struct scsi_target *starget = scsi_target(sdev); + struct fc_rport *rport = starget_to_rport(starget); + struct ibmvfc_event *evt; + struct ibmvfc_tmf *tmf; + + evt = ibmvfc_get_event(queue); + ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_MAD_FORMAT); + + tmf = &evt->iu.tmf; + memset(tmf, 0, sizeof(*tmf)); + if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN)) { + tmf->common.version = cpu_to_be32(2); + tmf->target_wwpn = cpu_to_be64(rport->port_name); + } else { + tmf->common.version = cpu_to_be32(1); + } + tmf->common.opcode = cpu_to_be32(IBMVFC_TMF_MAD); + tmf->common.length = cpu_to_be16(sizeof(*tmf)); + tmf->scsi_id = cpu_to_be64(rport->port_id); + int_to_scsilun(sdev->lun, &tmf->lun); + if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS)) + type &= ~IBMVFC_TMF_SUPPRESS_ABTS; + if (vhost->state == IBMVFC_ACTIVE) + tmf->flags = cpu_to_be32((type | IBMVFC_TMF_LUA_VALID)); + else + tmf->flags = cpu_to_be32(((type & IBMVFC_TMF_SUPPRESS_ABTS) | IBMVFC_TMF_LUA_VALID)); + tmf->cancel_key = cpu_to_be32((unsigned long)sdev->hostdata); + tmf->my_cancel_key = cpu_to_be32((unsigned long)starget->hostdata); + + init_completion(&evt->comp); + + return evt; +} + /** * ibmvfc_cancel_all - Cancel all outstanding commands to the device * @sdev: scsi device to cancel commands @@ -2393,9 +2432,6 @@ static int ibmvfc_wait_for_ops(struct ibmvfc_host *vhost, void *device, static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) { struct ibmvfc_host *vhost = shost_priv(sdev->host); - struct scsi_target *starget = scsi_target(sdev); - struct fc_rport *rport = starget_to_rport(starget); - struct ibmvfc_tmf *tmf; struct ibmvfc_event *evt, *found_evt; union ibmvfc_iu rsp; int rsp_rc = -EBUSY; @@ -2422,32 +2458,8 @@ static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) } if (vhost->logged_in) { - evt = ibmvfc_get_event(&vhost->crq); - ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_MAD_FORMAT); - - tmf = &evt->iu.tmf; - memset(tmf, 0, sizeof(*tmf)); - if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN)) { - tmf->common.version = cpu_to_be32(2); - tmf->target_wwpn = cpu_to_be64(rport->port_name); - } else { - tmf->common.version = cpu_to_be32(1); - } - tmf->common.opcode = cpu_to_be32(IBMVFC_TMF_MAD); - tmf->common.length = cpu_to_be16(sizeof(*tmf)); - tmf->scsi_id = cpu_to_be64(rport->port_id); - int_to_scsilun(sdev->lun, &tmf->lun); - if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS)) - type &= ~IBMVFC_TMF_SUPPRESS_ABTS; - if (vhost->state == IBMVFC_ACTIVE) - tmf->flags = cpu_to_be32((type | IBMVFC_TMF_LUA_VALID)); - else - tmf->flags = cpu_to_be32(((type & IBMVFC_TMF_SUPPRESS_ABTS) | IBMVFC_TMF_LUA_VALID)); - tmf->cancel_key = cpu_to_be32((unsigned long)sdev->hostdata); - tmf->my_cancel_key = cpu_to_be32((unsigned long)starget->hostdata); - + evt = ibmvfc_init_tmf(&vhost->crq, sdev, type); evt->sync_iu = &rsp; - init_completion(&evt->comp); rsp_rc = ibmvfc_send_event(evt, vhost, default_timeout); } From patchwork Mon Jan 11 23:12:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A28EBC433E0 for ; Tue, 12 Jan 2021 00:30:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7CEF422CF7 for ; Tue, 12 Jan 2021 00:30:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403877AbhALAZr (ORCPT ); Mon, 11 Jan 2021 19:25:47 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:48194 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403882AbhAKXNY (ORCPT ); Mon, 11 Jan 2021 18:13:24 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN28GO093004; Mon, 11 Jan 2021 18:12:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=cUd44LP3mhyibI01umpjv/0DBGZmG1N1SO7aI/f5DIo=; b=Bm+EDDK5KVDvyYUOtJDNwlSoVyDddzYP6GWx8Q1tqKGYl+SMiHsbknI9HLfLUWLzWZ11 6h9TymthkxZFkMtYWUU5Dj+WUGYmsYpmOJ4FetDy9ju/MF5zftJsJfGFa9aVRSLTmmS9 HN6+Mv2ADRRpFgSYTG98893i2qD0g/a5KOeaitcat+rGAEmKJ6hmt5Daw5+ZyDnNIT80 b/EiOqf99jqGgNZArVl9V2N6/U+0fChejma5KtCvjwnuiskJZvznw+J4V6fIJsI7PB5z G98NVxYO5jxrDq9hDqgnmaz+tGMz4BHNb92EKCgq2aAeaPRh75j6NGWA9t2szPQJadYt Lw== Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 360y91sb59-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:39 -0500 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN78Gs004814; Mon, 11 Jan 2021 23:12:37 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma03wdc.us.ibm.com with ESMTP id 35y448syqs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:37 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCa0N21430686 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:36 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8B30C7805E; Mon, 11 Jan 2021 23:12:36 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 29DAB78060; Mon, 11 Jan 2021 23:12:36 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:36 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 18/21] ibmvfc: send Cancel MAD down each hw scsi channel Date: Mon, 11 Jan 2021 17:12:22 -0600 Message-Id: <20210111231225.105347-19-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=999 malwarescore=0 priorityscore=1501 phishscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 spamscore=0 mlxscore=0 bulkscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In general the client needs to send Cancel MADs and task management commands down the same channel as the command(s) intended to cancel or abort. The client assigns cancel keys per LUN and thus must send a Cancel down each channel commands were submitted for that LUN. Further, the client then must wait for those cancel completions prior to submitting a LUN RESET or ABORT TASK SET. Add a cancel rsp iu syncronization field to the ibmvfc_queue struct such that the cancel routine can sync the cancel response to each queue that requires a cancel command. Build a list of each cancel event sent and wait for the completion of each submitted cancel. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 106 +++++++++++++++++++++++++++++---- drivers/scsi/ibmvscsi/ibmvfc.h | 3 + 2 files changed, 97 insertions(+), 12 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index b0b0212344f3..24e1278acfeb 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -2418,18 +2418,79 @@ static struct ibmvfc_event *ibmvfc_init_tmf(struct ibmvfc_queue *queue, return evt; } -/** - * ibmvfc_cancel_all - Cancel all outstanding commands to the device - * @sdev: scsi device to cancel commands - * @type: type of error recovery being performed - * - * This sends a cancel to the VIOS for the specified device. This does - * NOT send any abort to the actual device. That must be done separately. - * - * Returns: - * 0 on success / other on failure - **/ -static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) +static int ibmvfc_cancel_all_mq(struct scsi_device *sdev, int type) +{ + struct ibmvfc_host *vhost = shost_priv(sdev->host); + struct ibmvfc_event *evt, *found_evt, *temp; + struct ibmvfc_queue *queues = vhost->scsi_scrqs.scrqs; + unsigned long flags; + int num_hwq, i; + LIST_HEAD(cancelq); + u16 status; + + ENTER; + spin_lock_irqsave(vhost->host->host_lock, flags); + num_hwq = vhost->scsi_scrqs.active_queues; + for (i = 0; i < num_hwq; i++) { + spin_lock(queues[i].q_lock); + spin_lock(&queues[i].l_lock); + found_evt = NULL; + list_for_each_entry(evt, &queues[i].sent, queue_list) { + if (evt->cmnd && evt->cmnd->device == sdev) { + found_evt = evt; + break; + } + } + spin_unlock(&queues[i].l_lock); + + if (!found_evt) + continue; + + if (vhost->logged_in) { + evt = ibmvfc_init_tmf(&queues[i], sdev, type); + evt->sync_iu = &queues[i].cancel_rsp; + ibmvfc_send_event(evt, vhost, default_timeout); + list_add_tail(&evt->cancel, &cancelq); + } + } + + for (i = 0; i < num_hwq; i++) + spin_unlock(queues[i].q_lock); + spin_unlock_irqrestore(vhost->host->host_lock, flags); + + if (list_empty(&cancelq)) { + if (vhost->log_level > IBMVFC_DEFAULT_LOG_LEVEL) + sdev_printk(KERN_INFO, sdev, "No events found to cancel\n"); + return 0; + } + + sdev_printk(KERN_INFO, sdev, "Cancelling outstanding commands.\n"); + + list_for_each_entry_safe(evt, temp, &cancelq, cancel) { + wait_for_completion(&evt->comp); + status = be16_to_cpu(evt->queue->cancel_rsp.mad_common.status); + ibmvfc_free_event(evt); + + if (status != IBMVFC_MAD_SUCCESS) { + sdev_printk(KERN_WARNING, sdev, "Cancel failed with rc=%x\n", status); + switch (status) { + case IBMVFC_MAD_DRIVER_FAILED: + case IBMVFC_MAD_CRQ_ERROR: + /* Host adapter most likely going through reset, return success to + * the caller will wait for the command being cancelled to get returned + */ + break; + default: + break; + } + } + } + + sdev_printk(KERN_INFO, sdev, "Successfully cancelled outstanding commands\n"); + return 0; +} + +static int ibmvfc_cancel_all_sq(struct scsi_device *sdev, int type) { struct ibmvfc_host *vhost = shost_priv(sdev->host); struct ibmvfc_event *evt, *found_evt; @@ -2498,6 +2559,27 @@ static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) return 0; } +/** + * ibmvfc_cancel_all - Cancel all outstanding commands to the device + * @sdev: scsi device to cancel commands + * @type: type of error recovery being performed + * + * This sends a cancel to the VIOS for the specified device. This does + * NOT send any abort to the actual device. That must be done separately. + * + * Returns: + * 0 on success / other on failure + **/ +static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) +{ + struct ibmvfc_host *vhost = shost_priv(sdev->host); + + if (vhost->mq_enabled && vhost->using_channels) + return ibmvfc_cancel_all_mq(sdev, type); + else + return ibmvfc_cancel_all_sq(sdev, type); +} + /** * ibmvfc_match_key - Match function for specified cancel key * @evt: ibmvfc event struct diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 2dbce7071409..c3bb83c9d8a6 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -737,6 +737,7 @@ struct ibmvfc_target { /* a unit of work for the hosting partition */ struct ibmvfc_event { struct list_head queue_list; + struct list_head cancel; struct ibmvfc_host *vhost; struct ibmvfc_queue *queue; struct ibmvfc_target *tgt; @@ -790,6 +791,8 @@ struct ibmvfc_queue { struct list_head free; spinlock_t l_lock; + union ibmvfc_iu cancel_rsp; + /* Sub-CRQ fields */ struct ibmvfc_host *vhost; unsigned long cookie; From patchwork Mon Jan 11 23:12:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5391DC433E6 for ; Tue, 12 Jan 2021 00:30:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2F42422CF7 for ; Tue, 12 Jan 2021 00:30:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405538AbhALAZs (ORCPT ); Mon, 11 Jan 2021 19:25:48 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:17604 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2403902AbhAKXN1 (ORCPT ); Mon, 11 Jan 2021 18:13:27 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN1Q9l093675; Mon, 11 Jan 2021 18:12:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=OCeqXqTg76Y0WcH/KI/byd9/8vHZqFTCU6yOhsYJqt8=; b=Wf4VU+0rh3h1LzOaPCugHkxlezrad9NmmDbC+er4jDxv5GhnAsiqIdxL0wczk+0Gj1gL uVKiavzYHTqIr7lrQ5hXxLQOUFXNQ49ye3uebPe0RDzOk/9otJbodgQQaigmiK9GlP3a Gd3xdgKJcGYSi3Vd1affXtQRJKUetb2Ful9GnlauOr8pTkkE6iFIWcPDEb6cLlN+nLYN hvcxLYdj+WGXM9P6JVkLHCCsBQ2mT1oty+Aon07LRuazVOlKt83ysWpKiESfuwDcCzXi lLIM8t0w/GO9EBm4FJxIC7vAmNJtRCiL0qVD9rb6ogZEvv6tbQaBBKTOzujfGgfvJj0P 4g== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0b-001b2d01.pphosted.com with ESMTP id 360wgfmekf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:41 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN7Q1I010322; Mon, 11 Jan 2021 23:12:40 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma01wdc.us.ibm.com with ESMTP id 35y448sysj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:40 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCbfl27656688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:37 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B84C7805C; Mon, 11 Jan 2021 23:12:37 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9E71678060; Mon, 11 Jan 2021 23:12:36 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:36 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 19/21] ibmvfc: purge scsi channels after transport loss/reset Date: Mon, 11 Jan 2021 17:12:23 -0600 Message-Id: <20210111231225.105347-20-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 spamscore=0 adultscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Grab the queue and list lock for each Sub-CRQ and add any uncompleted events to the host purge list. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 24e1278acfeb..b413f5da71ce 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1056,7 +1056,13 @@ static void ibmvfc_fail_request(struct ibmvfc_event *evt, int error_code) static void ibmvfc_purge_requests(struct ibmvfc_host *vhost, int error_code) { struct ibmvfc_event *evt, *pos; + struct ibmvfc_queue *queues = vhost->scsi_scrqs.scrqs; unsigned long flags; + int hwqs = 0; + int i; + + if (vhost->using_channels) + hwqs = vhost->scsi_scrqs.active_queues; ibmvfc_dbg(vhost, "Purging all requests\n"); spin_lock_irqsave(&vhost->crq.l_lock, flags); @@ -1064,6 +1070,16 @@ static void ibmvfc_purge_requests(struct ibmvfc_host *vhost, int error_code) ibmvfc_fail_request(evt, error_code); list_splice_init(&vhost->crq.sent, &vhost->purge); spin_unlock_irqrestore(&vhost->crq.l_lock, flags); + + for (i = 0; i < hwqs; i++) { + spin_lock_irqsave(queues[i].q_lock, flags); + spin_lock(&queues[i].l_lock); + list_for_each_entry_safe(evt, pos, &queues[i].sent, queue_list) + ibmvfc_fail_request(evt, error_code); + list_splice_init(&queues[i].sent, &vhost->purge); + spin_unlock(&queues[i].l_lock); + spin_unlock_irqrestore(queues[i].q_lock, flags); + } } /** From patchwork Mon Jan 11 23:12:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 361592 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD0A3C433E6 for ; Tue, 12 Jan 2021 00:25:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E89422D49 for ; Tue, 12 Jan 2021 00:25:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403872AbhALAZr (ORCPT ); Mon, 11 Jan 2021 19:25:47 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:58832 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403885AbhAKXNY (ORCPT ); Mon, 11 Jan 2021 18:13:24 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN2810093026; Mon, 11 Jan 2021 18:12:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=e/g278L8vvPF3KlzSvgAs3eL73hhc7XdsrgFqfIR1kA=; b=GPfnEaRmiEVX58UoMX/5hXTM1BeisX91LCAADANJkRpaKr3F8oSUaP7kE9cpcCWdyHWs xog0GfbJcHz9HrMa9os5BqzZ2cLhYYmAzBLecbphQrx5C5DNwuhOldABRkjt4x3qAh3x wWudq2ttkCqOJT4A/c+KKIXNyyLqz0YsulRXbgumTY0RKS+aTkz4EUhOmsJrdUGCwj1T rH81HU6jTvDl1KBb6VUQvHDxoUqWmBtUIUX0OgjNOkOfs9GpGSLa8t0oFEtra8FLCFxr /m6AvsqQpTAun/CChKnMn3BCUhV77ctfCXNuJQ6Z62IrC7gIf3J4m03dS8ZKHJP8A3AQ vQ== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 360y91sb5k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:40 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN9s7U009902; Mon, 11 Jan 2021 23:12:39 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma02wdc.us.ibm.com with ESMTP id 35y448syah-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:39 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCbQR22086016 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:37 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 898C27805F; Mon, 11 Jan 2021 23:12:37 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1EB3E7805E; Mon, 11 Jan 2021 23:12:37 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:37 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 20/21] ibmvfc: enable MQ and set reasonable defaults Date: Mon, 11 Jan 2021 17:12:24 -0600 Message-Id: <20210111231225.105347-21-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_32:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=841 malwarescore=0 priorityscore=1501 phishscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 spamscore=0 mlxscore=0 bulkscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110126 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Turn on MQ by default and set sane values for the upper limit on hw queues for the scsi host, and number of hw scsi channels to request from the partner VIOS. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index c3bb83c9d8a6..0391cb746d0b 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -41,9 +41,9 @@ #define IBMVFC_DEFAULT_LOG_LEVEL 2 #define IBMVFC_MAX_CDB_LEN 16 #define IBMVFC_CLS3_ERROR 0 -#define IBMVFC_MQ 0 -#define IBMVFC_SCSI_CHANNELS 0 -#define IBMVFC_SCSI_HW_QUEUES 1 +#define IBMVFC_MQ 1 +#define IBMVFC_SCSI_CHANNELS 8 +#define IBMVFC_SCSI_HW_QUEUES 8 #define IBMVFC_MIG_NO_SUB_TO_CRQ 0 #define IBMVFC_MIG_NO_N_TO_M 0 From patchwork Mon Jan 11 23:12:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 362578 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C42E8C43381 for ; Tue, 12 Jan 2021 00:25:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 96A1422D58 for ; Tue, 12 Jan 2021 00:25:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403885AbhALAZs (ORCPT ); Mon, 11 Jan 2021 19:25:48 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:55234 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403889AbhAKXNZ (ORCPT ); Mon, 11 Jan 2021 18:13:25 -0500 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 10BN3Gmv039990; Mon, 11 Jan 2021 18:12:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=NTQh5Mca0fUVUHflOLrQ8uuJoSPpDULjBgxVxWMPUSI=; b=NfrwnJb0lHkQgIJucUKtK2eC11XvShqxQewA68VZdiEL9USDSm3T1aPJwS3zKQNd+1qK 2rqbEPxPOz4HOfcAtQC7Mj/UkB8lpntVvFq6DzPz2N8gRp1mL83bWMgcz5G839VzAh5o Zh98nYbOjsik5q1IauGPJ7hd6RhEn+oqiKYlxa4gARWSTWSmr8pV9Q1W2D+/l+FvyMln +CSgzQMNzW8O3VIvRRv/J4ewVUn/EVq1bZTio28qKXW3NIdt0aKMXdXgMghNYWxnsm3e AQXB1KDCwqF+IL1bgQQafvTqUl26oSIymaCpQ1meEC4r8Uj2wJEFnGqArqcNR/lpGBv5 aQ== Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 360xh4tnnj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 18:12:40 -0500 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 10BN8Uwc022002; Mon, 11 Jan 2021 23:12:39 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma04dal.us.ibm.com with ESMTP id 35y448xfj2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 11 Jan 2021 23:12:39 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 10BNCc5Q9830812 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 11 Jan 2021 23:12:38 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 13ABC7805C; Mon, 11 Jan 2021 23:12:38 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9CEA878060; Mon, 11 Jan 2021 23:12:37 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Mon, 11 Jan 2021 23:12:37 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v4 21/21] ibmvfc: provide modules parameters for MQ settings Date: Mon, 11 Jan 2021 17:12:25 -0600 Message-Id: <20210111231225.105347-22-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210111231225.105347-1-tyreld@linux.ibm.com> References: <20210111231225.105347-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-11_34:2021-01-11,2021-01-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0 malwarescore=0 adultscore=0 mlxlogscore=999 bulkscore=0 impostorscore=0 phishscore=0 spamscore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101110130 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add the various module parameter toggles for adjusting the MQ characteristics at boot/load time as well as a device attribute for changing the client scsi channel request amount. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 75 ++++++++++++++++++++++++++++++---- 1 file changed, 66 insertions(+), 9 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index b413f5da71ce..6a64735d9652 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -40,6 +40,12 @@ static unsigned int disc_threads = IBMVFC_MAX_DISC_THREADS; static unsigned int ibmvfc_debug = IBMVFC_DEBUG; static unsigned int log_level = IBMVFC_DEFAULT_LOG_LEVEL; static unsigned int cls3_error = IBMVFC_CLS3_ERROR; +static unsigned int mq_enabled = IBMVFC_MQ; +static unsigned int nr_scsi_hw_queues = IBMVFC_SCSI_HW_QUEUES; +static unsigned int nr_scsi_channels = IBMVFC_SCSI_CHANNELS; +static unsigned int mig_channels_only = IBMVFC_MIG_NO_SUB_TO_CRQ; +static unsigned int mig_no_less_channels = IBMVFC_MIG_NO_N_TO_M; + static LIST_HEAD(ibmvfc_head); static DEFINE_SPINLOCK(ibmvfc_driver_lock); static struct scsi_transport_template *ibmvfc_transport_template; @@ -49,6 +55,22 @@ MODULE_AUTHOR("Brian King "); MODULE_LICENSE("GPL"); MODULE_VERSION(IBMVFC_DRIVER_VERSION); +module_param_named(mq, mq_enabled, uint, S_IRUGO); +MODULE_PARM_DESC(mq, "Enable multiqueue support. " + "[Default=" __stringify(IBMVFC_MQ) "]"); +module_param_named(scsi_host_queues, nr_scsi_hw_queues, uint, S_IRUGO); +MODULE_PARM_DESC(scsi_host_queues, "Number of SCSI Host submission queues. " + "[Default=" __stringify(IBMVFC_SCSI_HW_QUEUES) "]"); +module_param_named(scsi_hw_channels, nr_scsi_channels, uint, S_IRUGO); +MODULE_PARM_DESC(scsi_hw_channels, "Number of hw scsi channels to request. " + "[Default=" __stringify(IBMVFC_SCSI_CHANNELS) "]"); +module_param_named(mig_channels_only, mig_channels_only, uint, S_IRUGO); +MODULE_PARM_DESC(mig_channels_only, "Prevent migration to non-channelized system. " + "[Default=" __stringify(IBMVFC_MIG_NO_SUB_TO_CRQ) "]"); +module_param_named(mig_no_less_channels, mig_no_less_channels, uint, S_IRUGO); +MODULE_PARM_DESC(mig_no_less_channels, "Prevent migration to system with less channels. " + "[Default=" __stringify(IBMVFC_MIG_NO_N_TO_M) "]"); + module_param_named(init_timeout, init_timeout, uint, S_IRUGO | S_IWUSR); MODULE_PARM_DESC(init_timeout, "Initialization timeout in seconds. " "[Default=" __stringify(IBMVFC_INIT_TIMEOUT) "]"); @@ -926,7 +948,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) crq->cur = 0; if (vhost->scsi_scrqs.scrqs) { - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + for (i = 0; i < nr_scsi_hw_queues; i++) { scrq = &vhost->scsi_scrqs.scrqs[i]; spin_lock(scrq->q_lock); memset(scrq->msgs.scrq, 0, PAGE_SIZE); @@ -3394,6 +3416,37 @@ static ssize_t ibmvfc_store_log_level(struct device *dev, return strlen(buf); } +static ssize_t ibmvfc_show_scsi_channels(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct Scsi_Host *shost = class_to_shost(dev); + struct ibmvfc_host *vhost = shost_priv(shost); + unsigned long flags = 0; + int len; + + spin_lock_irqsave(shost->host_lock, flags); + len = snprintf(buf, PAGE_SIZE, "%d\n", vhost->client_scsi_channels); + spin_unlock_irqrestore(shost->host_lock, flags); + return len; +} + +static ssize_t ibmvfc_store_scsi_channels(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct Scsi_Host *shost = class_to_shost(dev); + struct ibmvfc_host *vhost = shost_priv(shost); + unsigned long flags = 0; + unsigned int channels; + + spin_lock_irqsave(shost->host_lock, flags); + channels = simple_strtoul(buf, NULL, 10); + vhost->client_scsi_channels = min(channels, nr_scsi_hw_queues); + ibmvfc_hard_reset_host(vhost); + spin_unlock_irqrestore(shost->host_lock, flags); + return strlen(buf); +} + static DEVICE_ATTR(partition_name, S_IRUGO, ibmvfc_show_host_partition_name, NULL); static DEVICE_ATTR(device_name, S_IRUGO, ibmvfc_show_host_device_name, NULL); static DEVICE_ATTR(port_loc_code, S_IRUGO, ibmvfc_show_host_loc_code, NULL); @@ -3402,6 +3455,8 @@ static DEVICE_ATTR(npiv_version, S_IRUGO, ibmvfc_show_host_npiv_version, NULL); static DEVICE_ATTR(capabilities, S_IRUGO, ibmvfc_show_host_capabilities, NULL); static DEVICE_ATTR(log_level, S_IRUGO | S_IWUSR, ibmvfc_show_log_level, ibmvfc_store_log_level); +static DEVICE_ATTR(nr_scsi_channels, S_IRUGO | S_IWUSR, + ibmvfc_show_scsi_channels, ibmvfc_store_scsi_channels); #ifdef CONFIG_SCSI_IBMVFC_TRACE /** @@ -3458,6 +3513,7 @@ static struct device_attribute *ibmvfc_attrs[] = { &dev_attr_npiv_version, &dev_attr_capabilities, &dev_attr_log_level, + &dev_attr_nr_scsi_channels, NULL }; @@ -4863,9 +4919,9 @@ static void ibmvfc_channel_enquiry(struct ibmvfc_host *vhost) mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_ENQUIRY); mad->common.length = cpu_to_be16(sizeof(*mad)); - if (IBMVFC_MIG_NO_SUB_TO_CRQ) + if (mig_channels_only) mad->flags |= cpu_to_be32(IBMVFC_NO_CHANNELS_TO_CRQ_SUPPORT); - if (IBMVFC_MIG_NO_N_TO_M) + if (mig_no_less_channels) mad->flags |= cpu_to_be32(IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT); ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); @@ -5652,13 +5708,13 @@ static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) ENTER; - vhost->scsi_scrqs.scrqs = kcalloc(IBMVFC_SCSI_HW_QUEUES, + vhost->scsi_scrqs.scrqs = kcalloc(nr_scsi_hw_queues, sizeof(*vhost->scsi_scrqs.scrqs), GFP_KERNEL); if (!vhost->scsi_scrqs.scrqs) return -1; - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + for (i = 0; i < nr_scsi_hw_queues; i++) { if (ibmvfc_register_scsi_channel(vhost, i)) { for (j = i; j > 0; j--) ibmvfc_deregister_scsi_channel(vhost, j - 1); @@ -5682,7 +5738,7 @@ static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) if (!vhost->scsi_scrqs.scrqs) return; - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) + for (i = 0; i < nr_scsi_hw_queues; i++) ibmvfc_deregister_scsi_channel(vhost, i); kfree(vhost->scsi_scrqs.scrqs); @@ -5880,12 +5936,13 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) shost->transportt = ibmvfc_transport_template; shost->can_queue = max_requests; + shost->can_queue = (max_requests / nr_scsi_hw_queues); shost->max_lun = max_lun; shost->max_id = max_targets; shost->max_sectors = IBMVFC_MAX_SECTORS; shost->max_cmd_len = IBMVFC_MAX_CDB_LEN; shost->unique_id = shost->host_no; - shost->nr_hw_queues = IBMVFC_MQ ? IBMVFC_SCSI_HW_QUEUES : 1; + shost->nr_hw_queues = mq_enabled ? nr_scsi_hw_queues : 1; vhost = shost_priv(shost); INIT_LIST_HEAD(&vhost->targets); @@ -5897,8 +5954,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->log_level = log_level; vhost->task_set = 1; - vhost->mq_enabled = IBMVFC_MQ; - vhost->client_scsi_channels = IBMVFC_SCSI_CHANNELS; + vhost->mq_enabled = mq_enabled; + vhost->client_scsi_channels = min(nr_scsi_hw_queues, nr_scsi_channels); vhost->using_channels = 0; vhost->do_enquiry = 1;