From patchwork Tue Jan 12 08:39:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 362731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61283C433E9 for ; Tue, 12 Jan 2021 08:41:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D96222E00 for ; Tue, 12 Jan 2021 08:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392386AbhALIlN (ORCPT ); Tue, 12 Jan 2021 03:41:13 -0500 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:54117 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729871AbhALIlM (ORCPT ); Tue, 12 Jan 2021 03:41:12 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 6F6A45C0200; Tue, 12 Jan 2021 03:40:06 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 12 Jan 2021 03:40:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=lSnJ06/tG6l4wYpZrWU37uPrKpNOWbnlrBtvF4KhJBw=; b=Pgo9lqvw kUU671qIEBvI8//g+DczOOVrID/NHZgibrAJmDyWu0W81XszkvYGQuS8coohh1mQ 4FpATkeFdixB6Mr3hKkjNjmz/Vh+eokXQ4jLvY2828vSrE16MxDtnq5GeMrOlvDq GNn26ZnZj8AdpkftIh/BbsCLyW3u5T6elO4xbJhqklbaTfDaIFWnmgzPKN28qDfK wey6GxB6OsZSfZnnboeEWART3X0oCIKodRRWWmXAaNzFb1Gvn/NYGKj3740LamxX wDfmo4rXrWgEi4KPq6uvfCPu+pBmS8Mls+oW1jtiC/oFX1eppXSWE35J0051/5j0 0fwkVosiXwFpJQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdehvddguddvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtke ertdertddtnecuhfhrohhmpefkughoucfutghhihhmmhgvlhcuoehiughoshgthhesihgu ohhstghhrdhorhhgqeenucggtffrrghtthgvrhhnpeduteeiveffffevleekleejffekhf ekhefgtdfftefhledvjefggfehgfevjeekhfenucfkphepkeegrddvvdelrdduheefrdeg geenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehiug hoshgthhesihguohhstghhrdhorhhg X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 06A99108005F; Tue, 12 Jan 2021 03:40:04 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, jiri@nvidia.com, danieller@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next 1/2] mlxsw: Register physical ports as a devlink resource Date: Tue, 12 Jan 2021 10:39:30 +0200 Message-Id: <20210112083931.1662874-2-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210112083931.1662874-1-idosch@idosch.org> References: <20210112083931.1662874-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Danielle Ratson The switch ASIC has a limited capacity of physical ('flavour physical' in devlink terminology) ports that it can support. While each system is brought up with a different number of ports, this number can be increased via splitting up to the ASIC's limit. Expose physical ports as a devlink resource so that user space will have visibility to the maximum number of ports that can be supported and the current occupancy. Signed-off-by: Danielle Ratson Reviewed-by: Jiri Pirko Signed-off-by: Ido Schimmel --- drivers/net/ethernet/mellanox/mlxsw/core.c | 77 ++++++++++++++++--- drivers/net/ethernet/mellanox/mlxsw/core.h | 5 ++ .../net/ethernet/mellanox/mlxsw/spectrum.h | 2 +- 3 files changed, 73 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index 685037e052af..8e8a5188c3a5 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -84,6 +84,7 @@ struct mlxsw_core { struct mlxsw_thermal *thermal; struct mlxsw_core_port *ports; unsigned int max_ports; + atomic_t active_ports_count; bool fw_flash_in_progress; struct { struct devlink_health_reporter *fw_fatal; @@ -95,9 +96,37 @@ struct mlxsw_core { }; #define MLXSW_PORT_MAX_PORTS_DEFAULT 0x40 +#define MLXSW_CORE_RESOURCE_NAME_PORTS "physical_ports" -static int mlxsw_ports_init(struct mlxsw_core *mlxsw_core) +static u64 mlxsw_ports_occ_get(void *priv) { + struct mlxsw_core *mlxsw_core = priv; + + return atomic_read(&mlxsw_core->active_ports_count); +} + +static int mlxsw_core_resources_ports_register(struct mlxsw_core *mlxsw_core) +{ + struct devlink *devlink = priv_to_devlink(mlxsw_core); + struct devlink_resource_size_params ports_num_params; + u32 max_ports; + + max_ports = mlxsw_core->max_ports - 1; + devlink_resource_size_params_init(&ports_num_params, max_ports, + max_ports, 1, + DEVLINK_RESOURCE_UNIT_ENTRY); + + return devlink_resource_register(devlink, MLXSW_CORE_RESOURCE_NAME_PORTS, + max_ports, MLXSW_CORE_RESOURCE_PORTS, + DEVLINK_RESOURCE_ID_PARENT_TOP, + &ports_num_params); +} + +static int mlxsw_ports_init(struct mlxsw_core *mlxsw_core, bool reload) +{ + struct devlink *devlink = priv_to_devlink(mlxsw_core); + int err; + /* Switch ports are numbered from 1 to queried value */ if (MLXSW_CORE_RES_VALID(mlxsw_core, MAX_SYSTEM_PORT)) mlxsw_core->max_ports = MLXSW_CORE_RES_GET(mlxsw_core, @@ -110,11 +139,30 @@ static int mlxsw_ports_init(struct mlxsw_core *mlxsw_core) if (!mlxsw_core->ports) return -ENOMEM; + if (!reload) { + err = mlxsw_core_resources_ports_register(mlxsw_core); + if (err) + goto err_resources_ports_register; + } + atomic_set(&mlxsw_core->active_ports_count, 0); + devlink_resource_occ_get_register(devlink, MLXSW_CORE_RESOURCE_PORTS, + mlxsw_ports_occ_get, mlxsw_core); + return 0; + +err_resources_ports_register: + kfree(mlxsw_core->ports); + return err; } -static void mlxsw_ports_fini(struct mlxsw_core *mlxsw_core) +static void mlxsw_ports_fini(struct mlxsw_core *mlxsw_core, bool reload) { + struct devlink *devlink = priv_to_devlink(mlxsw_core); + + devlink_resource_occ_get_unregister(devlink, MLXSW_CORE_RESOURCE_PORTS); + if (!reload) + devlink_resources_unregister(priv_to_devlink(mlxsw_core), NULL); + kfree(mlxsw_core->ports); } @@ -1897,7 +1945,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, goto err_register_resources; } - err = mlxsw_ports_init(mlxsw_core); + err = mlxsw_ports_init(mlxsw_core, reload); if (err) goto err_ports_init; @@ -1986,7 +2034,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, err_emad_init: kfree(mlxsw_core->lag.mapping); err_alloc_lag_mapping: - mlxsw_ports_fini(mlxsw_core); + mlxsw_ports_fini(mlxsw_core, reload); err_ports_init: if (!reload) devlink_resources_unregister(devlink, NULL); @@ -2056,7 +2104,7 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, devlink_unregister(devlink); mlxsw_emad_fini(mlxsw_core); kfree(mlxsw_core->lag.mapping); - mlxsw_ports_fini(mlxsw_core); + mlxsw_ports_fini(mlxsw_core, reload); if (!reload) devlink_resources_unregister(devlink, NULL); mlxsw_core->bus->fini(mlxsw_core->bus_priv); @@ -2755,16 +2803,25 @@ int mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u8 local_port, const unsigned char *switch_id, unsigned char switch_id_len) { - return __mlxsw_core_port_init(mlxsw_core, local_port, - DEVLINK_PORT_FLAVOUR_PHYSICAL, - port_number, split, split_port_subnumber, - splittable, lanes, - switch_id, switch_id_len); + int err; + + err = __mlxsw_core_port_init(mlxsw_core, local_port, + DEVLINK_PORT_FLAVOUR_PHYSICAL, + port_number, split, split_port_subnumber, + splittable, lanes, + switch_id, switch_id_len); + if (err) + return err; + + atomic_inc(&mlxsw_core->active_ports_count); + return 0; } EXPORT_SYMBOL(mlxsw_core_port_init); void mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u8 local_port) { + atomic_dec(&mlxsw_core->active_ports_count); + __mlxsw_core_port_fini(mlxsw_core, local_port); } EXPORT_SYMBOL(mlxsw_core_port_fini); diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.h b/drivers/net/ethernet/mellanox/mlxsw/core.h index 6b3ccbf6b238..8af7d9d03475 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.h +++ b/drivers/net/ethernet/mellanox/mlxsw/core.h @@ -19,6 +19,11 @@ #include "cmd.h" #include "resources.h" +enum mlxsw_core_resource_id { + MLXSW_CORE_RESOURCE_PORTS = 1, + MLXSW_CORE_RESOURCE_MAX, +}; + struct mlxsw_core; struct mlxsw_core_port; struct mlxsw_driver; diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h index a6956cfc9cb1..a3769f95a182 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h @@ -52,7 +52,7 @@ #define MLXSW_SP_RESOURCE_NAME_COUNTERS_RIF "rif" enum mlxsw_sp_resource_id { - MLXSW_SP_RESOURCE_KVD = 1, + MLXSW_SP_RESOURCE_KVD = MLXSW_CORE_RESOURCE_MAX, MLXSW_SP_RESOURCE_KVD_LINEAR, MLXSW_SP_RESOURCE_KVD_HASH_SINGLE, MLXSW_SP_RESOURCE_KVD_HASH_DOUBLE, From patchwork Tue Jan 12 08:39:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ido Schimmel X-Patchwork-Id: 362732 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DFFDC43381 for ; Tue, 12 Jan 2021 08:41:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 409AB22E01 for ; Tue, 12 Jan 2021 08:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405568AbhALIlO (ORCPT ); Tue, 12 Jan 2021 03:41:14 -0500 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:48767 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392373AbhALIlO (ORCPT ); Tue, 12 Jan 2021 03:41:14 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id EE9115C0210; Tue, 12 Jan 2021 03:40:07 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 12 Jan 2021 03:40:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=NTwxfu23AhYhT57QGnoRh9p7eGlwYLDIMWTyH0cgS6E=; b=G/LvrL74 ihTyHymlHX4RoWSCakGjJ6CRVWPMKt0kUeesJ1v7GVyb2mMc7T55Vt9quNTIUPlq ANk9p5VJIYzMsc1ENq79tUVboQHpjxFs82iswQg2uAvtECpsP1Ss4p9ff5T36CIC 3sspfMEuXFYQ006xqr3ZktUfw/H3RYXk7IhXCDUbVUCpyxQ1Tp0VHr1/Rn6bj0oy SAeI3jE7EM90bknqdE8w5ckZ7KbueP5P8a5Ir4tkDxHc+93XJSixKDKjTtTQ09wp GDt/npUdc3Lw90NVAyC/iy4RcBCP1jJee0pvNghxxvdTbLUcagWIaLTHx9E6OFHj YZs8YJkss8tlng== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdehvddguddvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtke ertdertddtnecuhfhrohhmpefkughoucfutghhihhmmhgvlhcuoehiughoshgthhesihgu ohhstghhrdhorhhgqeenucggtffrrghtthgvrhhnpeduteeiveffffevleekleejffekhf ekhefgtdfftefhledvjefggfehgfevjeekhfenucfkphepkeegrddvvdelrdduheefrdeg geenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehiug hoshgthhesihguohhstghhrdhorhhg X-ME-Proxy: Received: from shredder.mellanox.com (igld-84-229-153-44.inter.net.il [84.229.153.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 8E86A1080059; Tue, 12 Jan 2021 03:40:06 -0500 (EST) From: Ido Schimmel To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, jiri@nvidia.com, danieller@nvidia.com, mlxsw@nvidia.com, Ido Schimmel Subject: [PATCH net-next 2/2] selftests: mlxsw: Add a scale test for physical ports Date: Tue, 12 Jan 2021 10:39:31 +0200 Message-Id: <20210112083931.1662874-3-idosch@idosch.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210112083931.1662874-1-idosch@idosch.org> References: <20210112083931.1662874-1-idosch@idosch.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Danielle Ratson Query the maximum number of supported physical ports using devlink-resource and test that this number can be reached by splitting each of the splittable ports to its width. Test that an error is returned in case the maximum number is exceeded. Signed-off-by: Danielle Ratson Reviewed-by: Jiri Pirko Signed-off-by: Ido Schimmel --- .../selftests/drivers/net/mlxsw/port_scale.sh | 64 +++++++++++++++++++ .../net/mlxsw/spectrum-2/port_scale.sh | 16 +++++ .../net/mlxsw/spectrum-2/resource_scale.sh | 2 +- .../drivers/net/mlxsw/spectrum/port_scale.sh | 16 +++++ .../net/mlxsw/spectrum/resource_scale.sh | 2 +- 5 files changed, 98 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/drivers/net/mlxsw/port_scale.sh create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh diff --git a/tools/testing/selftests/drivers/net/mlxsw/port_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/port_scale.sh new file mode 100644 index 000000000000..f813ffefc07e --- /dev/null +++ b/tools/testing/selftests/drivers/net/mlxsw/port_scale.sh @@ -0,0 +1,64 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +# Test for physical ports resource. The test splits each splittable port +# to its width and checks that eventually the number of physical ports equals +# the maximum number of physical ports. + +PORT_NUM_NETIFS=0 + +port_setup_prepare() +{ + : +} + +port_cleanup() +{ + pre_cleanup + + for port in "${unsplit[@]}"; do + devlink port unsplit $port + check_err $? "Did not unsplit $netdev" + done +} + +split_all_ports() +{ + local should_fail=$1; shift + local -a unsplit + + # Loop over the splittable netdevs and create tuples of netdev along + # with its width. For example: + # '$netdev1 $count1 $netdev2 $count2...', when: + # $netdev1-2 are splittable netdevs in the device, and + # $count1-2 are the netdevs width respectively. + while read netdev count <<<$( + devlink -j port show | + jq -r '.[][] | select(.splittable==true) | "\(.netdev) \(.lanes)"' + ) + [[ ! -z $netdev ]] + do + devlink port split $netdev count $count + check_err $? "Did not split $netdev into $count" + unsplit+=( "${netdev}s0" ) + done +} + +port_test() +{ + local max_ports=$1; shift + local should_fail=$1; shift + + split_all_ports $should_fail + + occ=$(devlink -j resource show $DEVLINK_DEV \ + | jq '.[][][] | select(.name=="physical_ports") |.["occ"]') + + [[ $occ -eq $max_ports ]] + if [[ $should_fail -eq 0 ]]; then + check_err $? "Mismatch ports number: Expected $max_ports, got $occ." + else + check_err_fail $should_fail $? "Reached more ports than expected" + fi + +} diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh new file mode 100644 index 000000000000..0b71dfbbb447 --- /dev/null +++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0 +source ../port_scale.sh + +port_get_target() +{ + local should_fail=$1 + local target + + target=$(devlink_resource_size_get physical_ports) + + if ((! should_fail)); then + echo $target + else + echo $((target + 1)) + fi +} diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh index d7cf33a3f18d..4a1c9328555f 100755 --- a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh +++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh @@ -28,7 +28,7 @@ cleanup() trap cleanup EXIT -ALL_TESTS="router tc_flower mirror_gre tc_police" +ALL_TESTS="router tc_flower mirror_gre tc_police port" for current_test in ${TESTS:-$ALL_TESTS}; do source ${current_test}_scale.sh diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh new file mode 100644 index 000000000000..0b71dfbbb447 --- /dev/null +++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0 +source ../port_scale.sh + +port_get_target() +{ + local should_fail=$1 + local target + + target=$(devlink_resource_size_get physical_ports) + + if ((! should_fail)); then + echo $target + else + echo $((target + 1)) + fi +} diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh index 43f662401bc3..087a884f66cd 100755 --- a/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh +++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh @@ -22,7 +22,7 @@ cleanup() devlink_sp_read_kvd_defaults trap cleanup EXIT -ALL_TESTS="router tc_flower mirror_gre tc_police" +ALL_TESTS="router tc_flower mirror_gre tc_police port" for current_test in ${TESTS:-$ALL_TESTS}; do source ${current_test}_scale.sh