From patchwork Wed Sep 1 18:05:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 505463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FF4AC4320E for ; Wed, 1 Sep 2021 18:05:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C1DD60525 for ; Wed, 1 Sep 2021 18:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345472AbhIASGz (ORCPT ); Wed, 1 Sep 2021 14:06:55 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:60426 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235904AbhIASGy (ORCPT ); Wed, 1 Sep 2021 14:06:54 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 181I2eFk031076 for ; Wed, 1 Sep 2021 14:05:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=gNwRShj1fCPOjbvcZzcbmqfKAtdoDisxZvVIpOU9ySc=; b=htDFt8TfRxY/e4d20J60uPEgZtdVBNxxUELzslQadvGIlPrhpmRQ5ZhQuNC79tljxQnR mTaa2QIugWtAVHuU0BSnt3drQVE6tmWNvQT2FFAlfyd/hay4Csz42FoTHWygpmQT0Iy9 5TxBvgsFTKPbrnQGOQ+M+whdBvs/PX7c70peGm1dL51BTDUDxZZDk/nM1P6gdtsPy+U0 LXsOTS04db6cHyfN8hYrFt6VHMrPGfjG1RQxbV36F1sS66d8reLmUohWP5MNSUb9JZQS mbc/bU9SP5RkpHWuYnqGBMNU2YZIjtMBVj8O1cqQExwZhcy/3pDkiqYyDYVTtEo4wpGv Dw== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 3ate1arss6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 14:05:56 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 181I43LI023180 for ; Wed, 1 Sep 2021 18:05:56 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma03dal.us.ibm.com with ESMTP id 3atdxbgwwx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 18:05:56 +0000 Received: from b01ledav002.gho.pok.ibm.com (b01ledav002.gho.pok.ibm.com [9.57.199.107]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 181I5sNF38928746 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Sep 2021 18:05:54 GMT Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 71CA312405B; Wed, 1 Sep 2021 18:05:54 +0000 (GMT) Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5C48212405A; Wed, 1 Sep 2021 18:05:53 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.160.152.143]) by b01ledav002.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 1 Sep 2021 18:05:53 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next v2 1/9] ibmvnic: Consolidate code in replenish_rx_pool() Date: Wed, 1 Sep 2021 11:05:43 -0700 Message-Id: <20210901180551.150126-2-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901180551.150126-1-sukadev@linux.ibm.com> References: <20210901180551.150126-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: BCPkvWMpjVyN71YmsmEKXYCiZuBu_pHn X-Proofpoint-GUID: BCPkvWMpjVyN71YmsmEKXYCiZuBu_pHn X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-09-01_05:2021-09-01,2021-09-01 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 lowpriorityscore=0 bulkscore=0 clxscore=1015 suspectscore=0 spamscore=0 malwarescore=0 adultscore=0 phishscore=0 impostorscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000 definitions=main-2109010104 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org For better readability, consolidate related code in replenish_rx_pool() and add some comments. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index a775c69e4fd7..e8b1231be485 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -371,6 +371,8 @@ static void replenish_rx_pool(struct ibmvnic_adapter *adapter, } index = pool->free_map[pool->next_free]; + pool->free_map[pool->next_free] = IBMVNIC_INVALID_MAP; + pool->next_free = (pool->next_free + 1) % pool->size; if (pool->rx_buff[index].skb) dev_err(dev, "Inconsistent free_map!\n"); @@ -380,14 +382,15 @@ static void replenish_rx_pool(struct ibmvnic_adapter *adapter, dst = pool->long_term_buff.buff + offset; memset(dst, 0, pool->buff_size); dma_addr = pool->long_term_buff.addr + offset; - pool->rx_buff[index].data = dst; - pool->free_map[pool->next_free] = IBMVNIC_INVALID_MAP; + /* add the skb to an rx_buff in the pool */ + pool->rx_buff[index].data = dst; pool->rx_buff[index].dma = dma_addr; pool->rx_buff[index].skb = skb; pool->rx_buff[index].pool_index = pool->index; pool->rx_buff[index].size = pool->buff_size; + /* queue the rx_buff for the next send_subcrq_indirect */ sub_crq = &ind_bufp->indir_arr[ind_bufp->index++]; memset(sub_crq, 0, sizeof(*sub_crq)); sub_crq->rx_add.first = IBMVNIC_CRQ_CMD; @@ -405,7 +408,8 @@ static void replenish_rx_pool(struct ibmvnic_adapter *adapter, shift = 8; #endif sub_crq->rx_add.len = cpu_to_be32(pool->buff_size << shift); - pool->next_free = (pool->next_free + 1) % pool->size; + + /* if send_subcrq_indirect queue is full, flush to VIOS */ if (ind_bufp->index == IBMVNIC_MAX_IND_DESCS || i == count - 1) { lpar_rc = From patchwork Wed Sep 1 18:05:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 505462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A29ECC4320A for ; Wed, 1 Sep 2021 18:06:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F3A660E98 for ; Wed, 1 Sep 2021 18:06:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346521AbhIASG7 (ORCPT ); Wed, 1 Sep 2021 14:06:59 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:53122 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345478AbhIASG4 (ORCPT ); Wed, 1 Sep 2021 14:06:56 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 181I3gQV187461 for ; Wed, 1 Sep 2021 14:06:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=M6wOHtzBzQR8PzwJII4hlzFN1NNx5ohs41Ao5oYuLo4=; b=hWgIWQi2V/Teb2wjr3DWHQLXXvPh5ow54sRME2Tbm0BRsxYHH2iAcOpP7t5wEb3qFw1p UjVlpt09bN8Vl3ll62LhuFDpz0eKyM2pZyIVOYBlroJ7pbRjRRsbXJqnzKFhgAqSjXjD JR2Ilqpm/MSxxW+QzMH1/+YxluJjxk4cX7Ch6lWzvf5s9pZX6HbIqK4qzvjcsCcpE9Nl DczrrQw/I7JPshR+hL409xFX8+FqrWpCpuCNLjzUd9gHoF5cgckpxpeVV71Pk/0hBBdi RBWf2NWEurzt/CIhlJ0EjySARonpHyuoRp+58BuP/h1ui19Lz7k9AE1W+sFJaGssemq4 Ug== Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 3ated288bs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 14:05:59 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 181I2mXe007583 for ; Wed, 1 Sep 2021 18:05:58 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma05wdc.us.ibm.com with ESMTP id 3atdxd0r6t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 18:05:58 +0000 Received: from b01ledav002.gho.pok.ibm.com (b01ledav002.gho.pok.ibm.com [9.57.199.107]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 181I5vj78126988 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Sep 2021 18:05:57 GMT Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2F32512405A; Wed, 1 Sep 2021 18:05:57 +0000 (GMT) Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1ACF2124054; Wed, 1 Sep 2021 18:05:56 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.160.152.143]) by b01ledav002.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 1 Sep 2021 18:05:55 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next v2 3/9] ibmvnic: Use/rename local vars in init_rx_pools Date: Wed, 1 Sep 2021 11:05:45 -0700 Message-Id: <20210901180551.150126-4-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901180551.150126-1-sukadev@linux.ibm.com> References: <20210901180551.150126-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: hre7hIjIbnUzoX1UhB9N1IK8J0_QmSxu X-Proofpoint-ORIG-GUID: hre7hIjIbnUzoX1UhB9N1IK8J0_QmSxu X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-09-01_05:2021-09-01,2021-09-01 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 clxscore=1015 priorityscore=1501 phishscore=0 spamscore=0 impostorscore=0 bulkscore=0 adultscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000 definitions=main-2109010104 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To make the code more readable, use/rename some local variables. Basically we have a set of pools, num_pools. Each pool has a set of buffers, pool_size and each buffer is of size buff_size. pool_size is a bit ambiguous (whether size in bytes or buffers). Add a comment in the header file to make it explicit. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 17 +++++++++-------- drivers/net/ethernet/ibm/ibmvnic.h | 2 +- 2 files changed, 10 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index cd04c3eb6c41..688489475791 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -620,14 +620,16 @@ static int init_rx_pools(struct net_device *netdev) struct ibmvnic_adapter *adapter = netdev_priv(netdev); struct device *dev = &adapter->vdev->dev; struct ibmvnic_rx_pool *rx_pool; - int rxadd_subcrqs; + u64 num_pools; + u64 pool_size; /* # of buffers in one pool */ u64 buff_size; int i, j; - rxadd_subcrqs = adapter->num_active_rx_scrqs; + num_pools = adapter->num_active_rx_scrqs; + pool_size = adapter->req_rx_add_entries_per_subcrq; buff_size = adapter->cur_rx_buf_sz; - adapter->rx_pool = kcalloc(rxadd_subcrqs, + adapter->rx_pool = kcalloc(num_pools, sizeof(struct ibmvnic_rx_pool), GFP_KERNEL); if (!adapter->rx_pool) { @@ -638,17 +640,16 @@ static int init_rx_pools(struct net_device *netdev) /* Set num_active_rx_pools early. If we fail below after partial * allocation, release_rx_pools() will know how many to look for. */ - adapter->num_active_rx_pools = rxadd_subcrqs; + adapter->num_active_rx_pools = num_pools; - for (i = 0; i < rxadd_subcrqs; i++) { + for (i = 0; i < num_pools; i++) { rx_pool = &adapter->rx_pool[i]; netdev_dbg(adapter->netdev, "Initializing rx_pool[%d], %lld buffs, %lld bytes each\n", - i, adapter->req_rx_add_entries_per_subcrq, - buff_size); + i, pool_size, buff_size); - rx_pool->size = adapter->req_rx_add_entries_per_subcrq; + rx_pool->size = pool_size; rx_pool->index = i; rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); rx_pool->active = 1; diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h index 22df602323bc..5652566818fb 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.h +++ b/drivers/net/ethernet/ibm/ibmvnic.h @@ -827,7 +827,7 @@ struct ibmvnic_rx_buff { struct ibmvnic_rx_pool { struct ibmvnic_rx_buff *rx_buff; - int size; + int size; /* # of buffers in the pool */ int index; int buff_size; atomic_t available; From patchwork Wed Sep 1 18:05:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 505461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08ECDC432BE for ; Wed, 1 Sep 2021 18:06:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DCEB660F6F for ; Wed, 1 Sep 2021 18:06:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346674AbhIASHE (ORCPT ); Wed, 1 Sep 2021 14:07:04 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:38606 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345478AbhIASHA (ORCPT ); Wed, 1 Sep 2021 14:07:00 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 181I3QqO095708 for ; Wed, 1 Sep 2021 14:06:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=/STvjzj5KsmkCt1yq1JYgvvnD8RXosDpF7js/9V3idA=; b=Mnp7GxBQbT/3IU1Vkh1pIpt0cCN8xFoXm8///3s35qdT5dXa0rLMW6OYKam6xW/p/Gh6 G6c1z9FiMuuVxhQknrBjnuPN+HvTQubKOaFgbWvCJIYA7t36wMHCUkprwXP9cXJ2qne5 hBGP+of1RvIDQprwx2fZvANbyWmBzuCgo4CliE6kYYBjb9/+fjOcYCrJ8bH3o4MFVoxg LKoGYy7psFWTSb9AL0huiPT7Y6NfLKWeBBHcxMUIT68+1eqnrIKPrcpce46EGn9F1a1s GZ4Z38TwEQu1bolGfhB3XfYOr/kJNCWm+Zsloc73FTFrmOOwnkMBlprJUqlc4NL0zPXf kw== Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 3ate1ngrq7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 14:06:02 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 181I2nId007636 for ; Wed, 1 Sep 2021 18:06:01 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma05wdc.us.ibm.com with ESMTP id 3atdxd0r7p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 18:06:01 +0000 Received: from b01ledav002.gho.pok.ibm.com (b01ledav002.gho.pok.ibm.com [9.57.199.107]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 181I60da30998926 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Sep 2021 18:06:00 GMT Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2C84A12405B; Wed, 1 Sep 2021 18:06:00 +0000 (GMT) Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E913B124052; Wed, 1 Sep 2021 18:05:58 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.160.152.143]) by b01ledav002.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 1 Sep 2021 18:05:58 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next v2 5/9] ibmvnic: init_tx_pools move loop-invariant code Date: Wed, 1 Sep 2021 11:05:47 -0700 Message-Id: <20210901180551.150126-6-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901180551.150126-1-sukadev@linux.ibm.com> References: <20210901180551.150126-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 1voxR6bbwFzm-dBb1vt_LIRfOuORkNyA X-Proofpoint-GUID: 1voxR6bbwFzm-dBb1vt_LIRfOuORkNyA X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-09-01_05:2021-09-01,2021-09-01 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 phishscore=0 mlxlogscore=999 priorityscore=1501 impostorscore=0 adultscore=0 suspectscore=0 mlxscore=0 bulkscore=0 malwarescore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000 definitions=main-2109010104 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In init_tx_pools() move some loop-invariant code out of the loop. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 97041b319beb..bb9b8aec9c9b 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -843,11 +843,10 @@ static int init_tx_pools(struct net_device *netdev) * allocation, release_tx_pools() will know how many to look for. */ adapter->num_active_tx_pools = num_pools; + buff_size = adapter->req_mtu + VLAN_HLEN; + buff_size = ALIGN(buff_size, L1_CACHE_BYTES); for (i = 0; i < num_pools; i++) { - buff_size = adapter->req_mtu + VLAN_HLEN; - buff_size = ALIGN(buff_size, L1_CACHE_BYTES); - dev_dbg(dev, "Init tx pool %d [%llu, %llu]\n", i, adapter->req_tx_entries_per_subcrq, buff_size); From patchwork Wed Sep 1 18:05:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 505460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7758DC4320E for ; Wed, 1 Sep 2021 18:06:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 575CC61075 for ; Wed, 1 Sep 2021 18:06:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346708AbhIASHL (ORCPT ); Wed, 1 Sep 2021 14:07:11 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:11534 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346661AbhIASHD (ORCPT ); Wed, 1 Sep 2021 14:07:03 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 181I4QCC135911 for ; Wed, 1 Sep 2021 14:06:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Ftenm1l+iGD+nmAxRHfECJIdXtjMa7SfDZvMHSNvO60=; b=d5QWWJcgmtCMt96TE9XJWBkTv6zeaRJ0ilQod2tk4fOy06n5l/AEvLV+Iaxtfe0+S3b/ 4u5V0XzSLbv+v4sTRajrF6E4vUFgORUX8Z4innnG4NlJ02pGokkUKeTxHIIsxGpv9cZd R/YxcghXtfqzmQ0+qe8ODWzx4fbuhzYt++kGl3fY4cT7LU9faLaWoyZm3dKzxsW7WkzT jSS5JQfwW5SWm1kxkFS8xPKCZ3bkrd3C+0Q/umh2evl7ScSyILAtuP94XDng+fXuITaq ZW31KAbLqC5FnQ15jn4khCfSndo5mDly5gKoUV9ClVLuZerRm40mClXysQ3fPnIp8CuU rg== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 3ate208s2x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 14:06:05 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 181I2jEf029132 for ; Wed, 1 Sep 2021 18:06:04 GMT Received: from b01cxnp22035.gho.pok.ibm.com (b01cxnp22035.gho.pok.ibm.com [9.57.198.25]) by ppma01wdc.us.ibm.com with ESMTP id 3atdxc8rhh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 18:06:04 +0000 Received: from b01ledav002.gho.pok.ibm.com (b01ledav002.gho.pok.ibm.com [9.57.199.107]) by b01cxnp22035.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 181I63x340108462 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Sep 2021 18:06:03 GMT Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 250E512405B; Wed, 1 Sep 2021 18:06:03 +0000 (GMT) Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E8B78124052; Wed, 1 Sep 2021 18:06:01 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.160.152.143]) by b01ledav002.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 1 Sep 2021 18:06:01 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next v2 7/9] ibmvnic: Reuse LTB when possible Date: Wed, 1 Sep 2021 11:05:49 -0700 Message-Id: <20210901180551.150126-8-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901180551.150126-1-sukadev@linux.ibm.com> References: <20210901180551.150126-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 2HzeDyX7tZ5qv1RoGjk0lrkoTASwLNdT X-Proofpoint-ORIG-GUID: 2HzeDyX7tZ5qv1RoGjk0lrkoTASwLNdT X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-09-01_05:2021-09-01,2021-09-01 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 phishscore=0 lowpriorityscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 bulkscore=0 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000 definitions=main-2109010104 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Reuse the long term buffer during a reset as long as its size has not changed. If the size has changed, free it and allocate a new one of the appropriate size. When we do this, alloc_long_term_buff() and reset_long_term_buff() become identical. Drop reset_long_term_buff(). Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- Changelog[v2]: [Jakub Kicinski] Fix kdoc issues --- drivers/net/ethernet/ibm/ibmvnic.c | 137 ++++++++++++++++------------- 1 file changed, 74 insertions(+), 63 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 4603597a9c10..dafb36690fdc 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -108,6 +108,8 @@ static int init_crq_queue(struct ibmvnic_adapter *adapter); static int send_query_phys_parms(struct ibmvnic_adapter *adapter); static void ibmvnic_tx_scrq_clean_buffer(struct ibmvnic_adapter *adapter, struct ibmvnic_sub_crq_queue *tx_scrq); +static void free_long_term_buff(struct ibmvnic_adapter *adapter, + struct ibmvnic_long_term_buff *ltb); struct ibmvnic_stat { char name[ETH_GSTRING_LEN]; @@ -214,23 +216,77 @@ static int ibmvnic_wait_for_completion(struct ibmvnic_adapter *adapter, return -ETIMEDOUT; } +/** + * reuse_ltb() - Check if a long term buffer can be reused + * @ltb: The long term buffer to be checked + * @size: The size of the long term buffer. + * + * An LTB can be reused unless its size has changed. + * + * Return: Return true if the LTB can be reused, false otherwise. + */ +static bool reuse_ltb(struct ibmvnic_long_term_buff *ltb, int size) +{ + return (ltb->buff && ltb->size == size); +} + +/** + * alloc_long_term_buff() - Allocate a long term buffer (LTB) + * + * @adapter: ibmvnic adapter associated to the LTB + * @ltb: container object for the LTB + * @size: size of the LTB + * + * Allocate an LTB of the specified size and notify VIOS. + * + * If the given @ltb already has the correct size, reuse it. Otherwise if + * its non-NULL, free it. Then allocate a new one of the correct size. + * Notify the VIOS either way since we may now be working with a new VIOS. + * + * Allocating larger chunks of memory during resets, specially LPM or under + * low memory situations can cause resets to fail/timeout and for LPAR to + * lose connectivity. So hold onto the LTB even if we fail to communicate + * with the VIOS and reuse it on next open. Free LTB when adapter is closed. + * + * Return: 0 if we were able to allocate the LTB and notify the VIOS and + * a negative value otherwise. + */ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter, struct ibmvnic_long_term_buff *ltb, int size) { struct device *dev = &adapter->vdev->dev; int rc; - ltb->size = size; - ltb->buff = dma_alloc_coherent(dev, ltb->size, <b->addr, - GFP_KERNEL); + if (!reuse_ltb(ltb, size)) { + dev_dbg(dev, + "LTB size changed from 0x%llx to 0x%x, reallocating\n", + ltb->size, size); + free_long_term_buff(adapter, ltb); + } - if (!ltb->buff) { - dev_err(dev, "Couldn't alloc long term buffer\n"); - return -ENOMEM; + if (ltb->buff) { + dev_dbg(dev, "Reusing LTB [map %d, size 0x%llx]\n", + ltb->map_id, ltb->size); + } else { + ltb->buff = dma_alloc_coherent(dev, size, <b->addr, + GFP_KERNEL); + if (!ltb->buff) { + dev_err(dev, "Couldn't alloc long term buffer\n"); + return -ENOMEM; + } + ltb->size = size; + + ltb->map_id = find_first_zero_bit(adapter->map_ids, + MAX_MAP_ID); + bitmap_set(adapter->map_ids, ltb->map_id, 1); + + dev_dbg(dev, + "Allocated new LTB [map %d, size 0x%llx]\n", + ltb->map_id, ltb->size); } - ltb->map_id = find_first_zero_bit(adapter->map_ids, - MAX_MAP_ID); - bitmap_set(adapter->map_ids, ltb->map_id, 1); + + /* Ensure ltb is zeroed - specially when reusing it. */ + memset(ltb->buff, 0, ltb->size); mutex_lock(&adapter->fw_lock); adapter->fw_done_rc = 0; @@ -257,10 +313,7 @@ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter, } rc = 0; out: - if (rc) { - dma_free_coherent(dev, ltb->size, ltb->buff, ltb->addr); - ltb->buff = NULL; - } + /* don't free LTB on communication error - see function header */ mutex_unlock(&adapter->fw_lock); return rc; } @@ -290,43 +343,6 @@ static void free_long_term_buff(struct ibmvnic_adapter *adapter, ltb->map_id = 0; } -static int reset_long_term_buff(struct ibmvnic_adapter *adapter, - struct ibmvnic_long_term_buff *ltb) -{ - struct device *dev = &adapter->vdev->dev; - int rc; - - memset(ltb->buff, 0, ltb->size); - - mutex_lock(&adapter->fw_lock); - adapter->fw_done_rc = 0; - - reinit_completion(&adapter->fw_done); - rc = send_request_map(adapter, ltb->addr, ltb->size, ltb->map_id); - if (rc) { - mutex_unlock(&adapter->fw_lock); - return rc; - } - - rc = ibmvnic_wait_for_completion(adapter, &adapter->fw_done, 10000); - if (rc) { - dev_info(dev, - "Reset failed, long term map request timed out or aborted\n"); - mutex_unlock(&adapter->fw_lock); - return rc; - } - - if (adapter->fw_done_rc) { - dev_info(dev, - "Reset failed, attempting to free and reallocate buffer\n"); - free_long_term_buff(adapter, ltb); - mutex_unlock(&adapter->fw_lock); - return alloc_long_term_buff(adapter, ltb, ltb->size); - } - mutex_unlock(&adapter->fw_lock); - return 0; -} - static void deactivate_rx_pools(struct ibmvnic_adapter *adapter) { int i; @@ -548,18 +564,10 @@ static int reset_rx_pools(struct ibmvnic_adapter *adapter) netdev_dbg(adapter->netdev, "Re-setting rx_pool[%d]\n", i); - if (rx_pool->buff_size != buff_size) { - free_long_term_buff(adapter, &rx_pool->long_term_buff); - rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); - rc = alloc_long_term_buff(adapter, - &rx_pool->long_term_buff, - rx_pool->size * - rx_pool->buff_size); - } else { - rc = reset_long_term_buff(adapter, - &rx_pool->long_term_buff); - } - + rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); + rc = alloc_long_term_buff(adapter, + &rx_pool->long_term_buff, + rx_pool->size * rx_pool->buff_size); if (rc) return rc; @@ -694,9 +702,12 @@ static int init_rx_pools(struct net_device *netdev) static int reset_one_tx_pool(struct ibmvnic_adapter *adapter, struct ibmvnic_tx_pool *tx_pool) { + struct ibmvnic_long_term_buff *ltb; int rc, i; - rc = reset_long_term_buff(adapter, &tx_pool->long_term_buff); + ltb = &tx_pool->long_term_buff; + + rc = alloc_long_term_buff(adapter, ltb, ltb->size); if (rc) return rc; From patchwork Wed Sep 1 18:05:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 505459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFB03C4320A for ; Wed, 1 Sep 2021 18:06:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B15060E98 for ; Wed, 1 Sep 2021 18:06:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346727AbhIASHQ (ORCPT ); Wed, 1 Sep 2021 14:07:16 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:47248 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1346707AbhIASHG (ORCPT ); Wed, 1 Sep 2021 14:07:06 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 181I2pxY047469 for ; Wed, 1 Sep 2021 14:06:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=9RTzSLJRzQr1R6OZD1Kd9en2LTJQUwi92pdoKS6C3SM=; b=bsunqPDSM67M9iRpeUcahj6KkJ4yceTQmlQnaIzwU2luYEW4TOba0R2sjmHHumg9JUOQ QsfmBYtaDPX9dvTQJXUarfxNbg1DLySzIrQ0dLGd+13cOEqoxrKSgYc+8xR5M3tX7532 +Rfu/OSqq4dQAv+cTix7WxrFPRMr6imzPKPAZn7qmQvkNSx9XvrvkFPRVT7deFwBiavc I7HVrdv3JGIqdFDqt4FwH9c3ZXb9YDEDcY9134jJF7JDwP8TivHwfi1oaAZEvWQlecw/ kyXLOj0OlM+LBpoM3ir3lCqwvxZ1sA9AOTdZX9jGsrQKzCzMZbm31dU5v+MNkavElZmu XQ== Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0b-001b2d01.pphosted.com with ESMTP id 3ate1brtjy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 14:06:08 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 181I3unf005299 for ; Wed, 1 Sep 2021 18:06:08 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02dal.us.ibm.com with ESMTP id 3atdxcrx7s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 01 Sep 2021 18:06:08 +0000 Received: from b01ledav002.gho.pok.ibm.com (b01ledav002.gho.pok.ibm.com [9.57.199.107]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 181I66us34472224 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Sep 2021 18:06:06 GMT Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B1CF12405A; Wed, 1 Sep 2021 18:06:06 +0000 (GMT) Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E30B912405C; Wed, 1 Sep 2021 18:06:04 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.160.152.143]) by b01ledav002.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 1 Sep 2021 18:06:04 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next v2 9/9] ibmvnic: Reuse tx pools when possible Date: Wed, 1 Sep 2021 11:05:51 -0700 Message-Id: <20210901180551.150126-10-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210901180551.150126-1-sukadev@linux.ibm.com> References: <20210901180551.150126-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: CcfARlvtfLUHHMN1blYqCuW9rENHPBA0 X-Proofpoint-ORIG-GUID: CcfARlvtfLUHHMN1blYqCuW9rENHPBA0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-09-01_05:2021-09-01,2021-09-01 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=0 lowpriorityscore=0 mlxscore=0 adultscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 priorityscore=1501 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000 definitions=main-2109010104 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Rather than releasing the tx pools on every close and reallocating them on open, reuse the tx pools unless the pool parameters (number of pools, size of each pool or size of each buffer in a pool) have changed. If the pool parameters changed, then release the old pools (if any) and allocate new ones. Specifically release tx pools, if: - adapter is removed, - pool parameters change during reset, - we encounter an error when opening the adapter in response to a user request (in ibmvnic_open()). and don't release them: - in __ibmvnic_close() or - on errors in __ibmvnic_open() in the hope that we can reuse them during this or next reset. With these changes reset_tx_pools() can be dropped because its optimization is now included in init_tx_pools() itself. cleanup_tx_pools() releases all the skbs associated with the pool and is called from ibmvnic_cleanup(), which is called on every reset. Since we want to reuse skbs across resets, move cleanup_tx_pools() out of ibmvnic_cleanup() and call it only when user closes the adapter. Add two new adapter fields, ->prev_mtu, ->prev_tx_pool_size to track the previous values and use them to decide whether to reuse or realloc the pools. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- Changelog[v2]: [Jakub Kicinski] Fix kdoc issues --- drivers/net/ethernet/ibm/ibmvnic.c | 215 +++++++++++++++++++---------- drivers/net/ethernet/ibm/ibmvnic.h | 2 + 2 files changed, 147 insertions(+), 70 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 05aabb0effef..d1be883f933a 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -767,53 +767,6 @@ static int init_rx_pools(struct net_device *netdev) return -1; } -static int reset_one_tx_pool(struct ibmvnic_adapter *adapter, - struct ibmvnic_tx_pool *tx_pool) -{ - struct ibmvnic_long_term_buff *ltb; - int rc, i; - - ltb = &tx_pool->long_term_buff; - - rc = alloc_long_term_buff(adapter, ltb, ltb->size); - if (rc) - return rc; - - memset(tx_pool->tx_buff, 0, - tx_pool->num_buffers * - sizeof(struct ibmvnic_tx_buff)); - - for (i = 0; i < tx_pool->num_buffers; i++) - tx_pool->free_map[i] = i; - - tx_pool->consumer_index = 0; - tx_pool->producer_index = 0; - - return 0; -} - -static int reset_tx_pools(struct ibmvnic_adapter *adapter) -{ - int tx_scrqs; - int i, rc; - - if (!adapter->tx_pool) - return -1; - - tx_scrqs = adapter->num_active_tx_pools; - for (i = 0; i < tx_scrqs; i++) { - ibmvnic_tx_scrq_clean_buffer(adapter, adapter->tx_scrq[i]); - rc = reset_one_tx_pool(adapter, &adapter->tso_pool[i]); - if (rc) - return rc; - rc = reset_one_tx_pool(adapter, &adapter->tx_pool[i]); - if (rc) - return rc; - } - - return 0; -} - static void release_vpd_data(struct ibmvnic_adapter *adapter) { if (!adapter->vpd) @@ -859,13 +812,13 @@ static void release_tx_pools(struct ibmvnic_adapter *adapter) kfree(adapter->tso_pool); adapter->tso_pool = NULL; adapter->num_active_tx_pools = 0; + adapter->prev_tx_pool_size = 0; } static int init_one_tx_pool(struct net_device *netdev, struct ibmvnic_tx_pool *tx_pool, int pool_size, int buf_size) { - struct ibmvnic_adapter *adapter = netdev_priv(netdev); int i; tx_pool->tx_buff = kcalloc(pool_size, @@ -874,13 +827,12 @@ static int init_one_tx_pool(struct net_device *netdev, if (!tx_pool->tx_buff) return -1; - if (alloc_long_term_buff(adapter, &tx_pool->long_term_buff, - pool_size * buf_size)) - return -1; - tx_pool->free_map = kcalloc(pool_size, sizeof(int), GFP_KERNEL); - if (!tx_pool->free_map) + if (!tx_pool->free_map) { + kfree(tx_pool->tx_buff); + tx_pool->tx_buff = NULL; return -1; + } for (i = 0; i < pool_size; i++) tx_pool->free_map[i] = i; @@ -893,6 +845,62 @@ static int init_one_tx_pool(struct net_device *netdev, return 0; } +/** + * reuse_tx_pools() - Check if the existing tx pools can be reused. + * @adapter: ibmvnic adapter + * + * Check if the existing tx pools in the adapter can be reused. The + * pools can be reused if the pool parameters (number of pools, + * number of buffers in the pool and mtu) have not changed. + * + * NOTE: This assumes that all pools have the same number of buffers + * which is the case currently. If that changes, we must fix this. + * + * Return: true if the tx pools can be reused, false otherwise. + */ +static bool reuse_tx_pools(struct ibmvnic_adapter *adapter) +{ + u64 old_num_pools, new_num_pools; + u64 old_pool_size, new_pool_size; + u64 old_mtu, new_mtu; + + if (!adapter->tx_pool) + return false; + + old_num_pools = adapter->num_active_tx_pools; + new_num_pools = adapter->num_active_tx_scrqs; + old_pool_size = adapter->prev_tx_pool_size; + new_pool_size = adapter->req_tx_entries_per_subcrq; + old_mtu = adapter->prev_mtu; + new_mtu = adapter->req_mtu; + + /* Require MTU to be exactly same to reuse pools for now */ + if (old_mtu != new_mtu) + return false; + + if (old_num_pools == new_num_pools && old_pool_size == new_pool_size) + return true; + + if (old_num_pools < adapter->min_tx_queues || + old_num_pools > adapter->max_tx_queues || + old_pool_size < adapter->min_tx_entries_per_subcrq || + old_pool_size > adapter->max_tx_entries_per_subcrq) + return false; + + return true; +} + +/** + * init_tx_pools(): Initialize the set of transmit pools in the adapter. + * @netdev: net device associated with the vnic interface + * + * Initialize the set of transmit pools in the ibmvnic adapter associated + * with the net_device @netdev. If possible, reuse the existing tx pools. + * Otherwise free any existing pools and allocate a new set of pools + * before initializing them. + * + * Return: 0 on success and negative value on error. + */ static int init_tx_pools(struct net_device *netdev) { struct ibmvnic_adapter *adapter = netdev_priv(netdev); @@ -900,7 +908,21 @@ static int init_tx_pools(struct net_device *netdev) int num_pools; u64 pool_size; /* # of buffers in pool */ u64 buff_size; - int i, rc; + int i, j, rc; + + num_pools = adapter->req_tx_queues; + + /* We must notify the VIOS about the LTB on all resets - but we only + * need to alloc/populate pools if either the number of buffers or + * size of each buffer in the pool has changed. + */ + if (reuse_tx_pools(adapter)) { + netdev_dbg(netdev, "Reusing tx pools\n"); + goto update_ltb; + } + + /* Allocate/populate the pools. */ + release_tx_pools(adapter); pool_size = adapter->req_tx_entries_per_subcrq; num_pools = adapter->num_active_tx_scrqs; @@ -925,6 +947,7 @@ static int init_tx_pools(struct net_device *netdev) * allocation, release_tx_pools() will know how many to look for. */ adapter->num_active_tx_pools = num_pools; + buff_size = adapter->req_mtu + VLAN_HLEN; buff_size = ALIGN(buff_size, L1_CACHE_BYTES); @@ -934,21 +957,73 @@ static int init_tx_pools(struct net_device *netdev) rc = init_one_tx_pool(netdev, &adapter->tx_pool[i], pool_size, buff_size); - if (rc) { - release_tx_pools(adapter); - return rc; - } + if (rc) + goto out_release; rc = init_one_tx_pool(netdev, &adapter->tso_pool[i], IBMVNIC_TSO_BUFS, IBMVNIC_TSO_BUF_SZ); - if (rc) { - release_tx_pools(adapter); - return rc; - } + if (rc) + goto out_release; + } + + adapter->prev_tx_pool_size = pool_size; + adapter->prev_mtu = adapter->req_mtu; + +update_ltb: + /* NOTE: All tx_pools have the same number of buffers (which is + * same as pool_size). All tso_pools have IBMVNIC_TSO_BUFS + * buffers (see calls init_one_tx_pool() for these). + * For consistency, we use tx_pool->num_buffers and + * tso_pool->num_buffers below. + */ + rc = -1; + for (i = 0; i < num_pools; i++) { + struct ibmvnic_tx_pool *tso_pool; + struct ibmvnic_tx_pool *tx_pool; + u32 ltb_size; + + tx_pool = &adapter->tx_pool[i]; + ltb_size = tx_pool->num_buffers * tx_pool->buf_size; + if (alloc_long_term_buff(adapter, &tx_pool->long_term_buff, + ltb_size)) + goto out; + + dev_dbg(dev, "Updated LTB for tx pool %d [%p, %d, %d]\n", + i, tx_pool->long_term_buff.buff, + tx_pool->num_buffers, tx_pool->buf_size); + + tx_pool->consumer_index = 0; + tx_pool->producer_index = 0; + + for (j = 0; j < tx_pool->num_buffers; j++) + tx_pool->free_map[j] = j; + + tso_pool = &adapter->tso_pool[i]; + ltb_size = tso_pool->num_buffers * tso_pool->buf_size; + if (alloc_long_term_buff(adapter, &tso_pool->long_term_buff, + ltb_size)) + goto out; + + dev_dbg(dev, "Updated LTB for tso pool %d [%p, %d, %d]\n", + i, tso_pool->long_term_buff.buff, + tso_pool->num_buffers, tso_pool->buf_size); + + tso_pool->consumer_index = 0; + tso_pool->producer_index = 0; + + for (j = 0; j < tso_pool->num_buffers; j++) + tso_pool->free_map[j] = j; } return 0; +out_release: + release_tx_pools(adapter); +out: + /* We failed to allocate one or more LTBs or map them on the VIOS. + * Hold onto the pools and any LTBs that we did allocate/map. + */ + return rc; } static void ibmvnic_napi_enable(struct ibmvnic_adapter *adapter) @@ -1139,8 +1214,6 @@ static void release_resources(struct ibmvnic_adapter *adapter) { release_vpd_data(adapter); - release_tx_pools(adapter); - release_napi(adapter); release_login_buffer(adapter); release_login_rsp_buffer(adapter); @@ -1413,6 +1486,7 @@ static int ibmvnic_open(struct net_device *netdev) netdev_err(netdev, "failed to initialize resources\n"); release_resources(adapter); release_rx_pools(adapter); + release_tx_pools(adapter); goto out; } } @@ -1541,8 +1615,6 @@ static void ibmvnic_cleanup(struct net_device *netdev) ibmvnic_napi_disable(adapter); ibmvnic_disable_irqs(adapter); - - clean_tx_pools(adapter); } static int __ibmvnic_close(struct net_device *netdev) @@ -1577,6 +1649,7 @@ static int ibmvnic_close(struct net_device *netdev) rc = __ibmvnic_close(netdev); ibmvnic_cleanup(netdev); clean_rx_pools(adapter); + clean_tx_pools(adapter); return rc; } @@ -2153,9 +2226,9 @@ static const char *reset_reason_to_string(enum ibmvnic_reset_reason reason) static int do_reset(struct ibmvnic_adapter *adapter, struct ibmvnic_rwi *rwi, u32 reset_state) { + struct net_device *netdev = adapter->netdev; u64 old_num_rx_queues, old_num_tx_queues; u64 old_num_rx_slots, old_num_tx_slots; - struct net_device *netdev = adapter->netdev; int rc; netdev_dbg(adapter->netdev, @@ -2305,7 +2378,6 @@ static int do_reset(struct ibmvnic_adapter *adapter, !adapter->rx_pool || !adapter->tso_pool || !adapter->tx_pool) { - release_tx_pools(adapter); release_napi(adapter); release_vpd_data(adapter); @@ -2314,9 +2386,10 @@ static int do_reset(struct ibmvnic_adapter *adapter, goto out; } else { - rc = reset_tx_pools(adapter); + rc = init_tx_pools(netdev); if (rc) { - netdev_dbg(adapter->netdev, "reset tx pools failed (%d)\n", + netdev_dbg(netdev, + "init tx pools failed (%d)\n", rc); goto out; } @@ -5661,6 +5734,7 @@ static int ibmvnic_probe(struct vio_dev *dev, const struct vio_device_id *id) init_completion(&adapter->stats_done); clear_bit(0, &adapter->resetting); adapter->prev_rx_buf_sz = 0; + adapter->prev_mtu = 0; init_success = false; do { @@ -5762,6 +5836,7 @@ static void ibmvnic_remove(struct vio_dev *dev) release_resources(adapter); release_rx_pools(adapter); + release_tx_pools(adapter); release_sub_crqs(adapter, 1); release_crq_queue(adapter); diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h index b73a1b812368..b8e42f67d897 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.h +++ b/drivers/net/ethernet/ibm/ibmvnic.h @@ -967,6 +967,7 @@ struct ibmvnic_adapter { u64 min_mtu; u64 max_mtu; u64 req_mtu; + u64 prev_mtu; u64 max_multicast_filters; u64 vlan_header_insertion; u64 rx_vlan_header_insertion; @@ -988,6 +989,7 @@ struct ibmvnic_adapter { u32 num_active_tx_pools; u32 prev_rx_pool_size; + u32 prev_tx_pool_size; u32 cur_rx_buf_sz; u32 prev_rx_buf_sz;