From patchwork Thu Nov 5 20:12:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 319762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B6DEC4741F for ; Thu, 5 Nov 2020 20:13:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E1F82074B for ; Thu, 5 Nov 2020 20:13:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="p6zNvp9m" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732253AbgKEUNa (ORCPT ); Thu, 5 Nov 2020 15:13:30 -0500 Received: from hqnvemgate26.nvidia.com ([216.228.121.65]:18627 "EHLO hqnvemgate26.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732049AbgKEUND (ORCPT ); Thu, 5 Nov 2020 15:13:03 -0500 Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Thu, 05 Nov 2020 12:13:06 -0800 Received: from sx1.mtl.com (10.124.1.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 5 Nov 2020 20:13:02 +0000 From: Saeed Mahameed To: Jakub Kicinski CC: , "David S. Miller" , "Yevgeny Kliteynik" , Alex Vesker , Mark Bloch , Saeed Mahameed Subject: [net-next v2 08/12] net/mlx5: DR, Free unused buddy ICM memory Date: Thu, 5 Nov 2020 12:12:38 -0800 Message-ID: <20201105201242.21716-9-saeedm@nvidia.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201105201242.21716-1-saeedm@nvidia.com> References: <20201105201242.21716-1-saeedm@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1604607186; bh=ee/PPwuC9tk0h0HecSEiuRm1+zIIVnA8x8tkhbhIiFs=; h=From:To:CC:Subject:Date:Message-ID:X-Mailer:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:Content-Type: X-Originating-IP:X-ClientProxiedBy; b=p6zNvp9moyqrJ4dYNONPV8Jr8iofeZD4gvB25Tei7q1wApRjUOB3RZigJPA1XhW9S /4BsXQR/sa+5s53SEaERiHSTOTK3zACHGFx4mIO13wyHfqMmv7Ad1Kh1ZaMHG9MI9M V9P3ZB4NItv79eXhjK3KDKOPVkYi/GJYxc+imoy3fv0SD3lIZfDkJBzXmRB5Og9cTY hRidNBNeSWsOEs0qnr6dc2zhDZWgG271AvObQhe2FRdxNXiI2fOHIN/BO1OeOeAVO+ 556M20mfnSKL8fDD8PviT/rta72N+WGDgSr6H+gjrnxiPYf7KC8oN9xDlSLz8nQA3Y oGus/jBPIa3iA== Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yevgeny Kliteynik Track buddy's used ICM memory, and free it if all of the buddy's memory bacame unused. Do this only for STEs. MODIFY_ACTION buddies are much smaller, so in case there is a large amount of modify_header actions, which result in large amount of MODIFY_ACTION buddies, doing this cleanup during sync will result in performance hit while not freeing significant amount of memory. Signed-off-by: Yevgeny Kliteynik Reviewed-by: Alex Vesker Reviewed-by: Mark Bloch Signed-off-by: Saeed Mahameed --- .../mellanox/mlx5/core/steering/dr_icm_pool.c | 14 ++++++++++---- .../ethernet/mellanox/mlx5/core/steering/mlx5dr.h | 1 + 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c index c49f8e86f3bc..66c24767e3b0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c @@ -175,10 +175,12 @@ get_chunk_icm_type(struct mlx5dr_icm_chunk *chunk) return chunk->buddy_mem->pool->icm_type; } -static void dr_icm_chunk_destroy(struct mlx5dr_icm_chunk *chunk) +static void dr_icm_chunk_destroy(struct mlx5dr_icm_chunk *chunk, + struct mlx5dr_icm_buddy_mem *buddy) { enum mlx5dr_icm_type icm_type = get_chunk_icm_type(chunk); + buddy->used_memory -= chunk->byte_size; list_del(&chunk->chunk_list); if (icm_type == DR_ICM_TYPE_STE) @@ -223,10 +225,10 @@ static void dr_icm_buddy_destroy(struct mlx5dr_icm_buddy_mem *buddy) struct mlx5dr_icm_chunk *chunk, *next; list_for_each_entry_safe(chunk, next, &buddy->hot_list, chunk_list) - dr_icm_chunk_destroy(chunk); + dr_icm_chunk_destroy(chunk, buddy); list_for_each_entry_safe(chunk, next, &buddy->used_list, chunk_list) - dr_icm_chunk_destroy(chunk); + dr_icm_chunk_destroy(chunk, buddy); dr_icm_pool_mr_destroy(buddy->icm_mr); @@ -267,6 +269,7 @@ dr_icm_chunk_create(struct mlx5dr_icm_pool *pool, goto out_free_chunk; } + buddy_mem_pool->used_memory += chunk->byte_size; chunk->buddy_mem = buddy_mem_pool; INIT_LIST_HEAD(&chunk->chunk_list); @@ -306,8 +309,11 @@ static int dr_icm_pool_sync_all_buddy_pools(struct mlx5dr_icm_pool *pool) mlx5dr_buddy_free_mem(buddy, chunk->seg, ilog2(chunk->num_of_entries)); pool->hot_memory_size -= chunk->byte_size; - dr_icm_chunk_destroy(chunk); + dr_icm_chunk_destroy(chunk, buddy); } + + if (!buddy->used_memory && pool->icm_type == DR_ICM_TYPE_STE) + dr_icm_buddy_destroy(buddy); } return 0; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h index a483d7de9ea6..4177786b8eaf 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h @@ -141,6 +141,7 @@ struct mlx5dr_icm_buddy_mem { /* This is the list of used chunks. HW may be accessing this memory */ struct list_head used_list; + u64 used_memory; /* Hardware may be accessing this memory but at some future, * undetermined time, it might cease to do so.