From patchwork Mon Jun 21 06:27:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joakim Zhang X-Patchwork-Id: 465649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DDD6C49361 for ; Mon, 21 Jun 2021 06:28:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 333F360FE7 for ; Mon, 21 Jun 2021 06:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229621AbhFUGbL (ORCPT ); Mon, 21 Jun 2021 02:31:11 -0400 Received: from mail-eopbgr140040.outbound.protection.outlook.com ([40.107.14.40]:33280 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230061AbhFUGbJ (ORCPT ); Mon, 21 Jun 2021 02:31:09 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N7BkM1K7x4ehNcEazj7ZI7Iz5B+H8NB+Lkctw3qKEWc0rUsphURZt8PK550CgPUEQcZJwYDTNBT9lNXbycbE9VDTjn7P5sL0YcNCE/zZUV3B7+qBs0fuj9WsIAvpoie9OasXtWbrl3Y+gWkP6ud3N44tYXs8fLN1RC3v7/i/laYFd745wTQiqQ44UURC6IBMNVtT88sVwFhP5TIrEtopJ4vZOoXxY5H+Lv9khRxUL6OxuPWNdn+26j126E4Mb/bOOBkENF+y0PvACI3aJeKcGSkGmH4qXwo/dM2cqh2JQfvLoYQf9aL9o64oU0dG6qR2j0qlcRs11AjAw/7G7gUbOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mnTSPsImtecW9CGV7Y0P0sEHtZY7xWlte6cO0d5re7I=; b=R+uFaZMlxzIe2vVLppw2E9O+DwTNjJhIT+mzoUAxJAlGrgnT52VWXsqNz+XYjOuMWrv9kx6hNCpYlSqJTxOoeastj0v27P223QpzrVbn0y8ZwuAJ8kR6wb9nd+wUBH5XrLloSxB/NHMX+SQU2QfqBcPr9ZRXvt94W1VBwr2isgJVAweT/h9XfI0gF4ewFC/V4RzzS+JrPKsBJ0PW9vbIAKWljvvx2sFTBFeqGvxxTHBEu6byg2W04LKDxCuSxsAANjCTtRwPdh1T3qtznkRKa+eZpUURRFbt5at5eR+at8svmmyu2AKK/kYf90xRJuvrBY9rR5g3FPohmNOPVz602g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mnTSPsImtecW9CGV7Y0P0sEHtZY7xWlte6cO0d5re7I=; b=haR75KNVJeLU5pmm1bXnq2d/49BkUc7XvoqMfwRWDHY/4RioSFXFmaHzLWI0Clm5hEpJtJRt/PsRXEILk2NNq8eO0dlFuXuKc5RKsyorOx6NyIxlcBFqcB/qiMiUFTSkPDJPwxYYNsvXbVg8o7zn9MxGTt2BeS16p1lSGg9BjYk= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=nxp.com; Received: from DB8PR04MB6795.eurprd04.prod.outlook.com (2603:10a6:10:fa::15) by DB8PR04MB5883.eurprd04.prod.outlook.com (2603:10a6:10:b1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.21; Mon, 21 Jun 2021 06:28:53 +0000 Received: from DB8PR04MB6795.eurprd04.prod.outlook.com ([fe80::f5e8:4886:3923:e92e]) by DB8PR04MB6795.eurprd04.prod.outlook.com ([fe80::f5e8:4886:3923:e92e%8]) with mapi id 15.20.4242.023; Mon, 21 Jun 2021 06:28:53 +0000 From: Joakim Zhang To: davem@davemloft.net, kuba@kernel.org, frieder.schrempf@kontron.de, andrew@lunn.ch Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-imx@nxp.com Subject: [PATCH V3 1/2] net: fec: add FEC_QUIRK_HAS_MULTI_QUEUES represents i.MX6SX ENET IP Date: Mon, 21 Jun 2021 14:27:36 +0800 Message-Id: <20210621062737.16896-2-qiangqing.zhang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210621062737.16896-1-qiangqing.zhang@nxp.com> References: <20210621062737.16896-1-qiangqing.zhang@nxp.com> X-Originating-IP: [119.31.174.71] X-ClientProxiedBy: SGAP274CA0002.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b6::14) To DB8PR04MB6795.eurprd04.prod.outlook.com (2603:10a6:10:fa::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (119.31.174.71) by SGAP274CA0002.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.15 via Frontend Transport; Mon, 21 Jun 2021 06:28:50 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f924e6d2-47ff-46c4-3dae-08d9347dd6c0 X-MS-TrafficTypeDiagnostic: DB8PR04MB5883: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5vKugeQ6WDJFJxEL4J7RDlawgS/AxmsRUv920pDrm3xPTCIofhnV7KkmNnt1eNcX0zEDVSt5Yq1WE1k4umlzvNvqWxqlRGddYiyFOftJHZQYKoS99TE6AVO9YoTWVN2uEZyE++ZdRYVk86BkdQIrBtYuL/LjEXV9/0kndPkw70cJb5ZCgcGUtWC9fGtoO+RDpn2F1Mix+juujsz/6tTs/OPkxfnkjjOQwlQYI05fA80nnFeWYnNoYwJmpVBBpCoSVdV5Ypeg8R6wAWPUco7ZpbE4tQKEL+Y7MbB8zDnFWn5e8HFDkD2euxVxy2L7ZpXHe2GZGpwlsWPxJSq27ySegOpUmlfWCoYigmt8H8nYOMw2Y9D9eT/Nl1ujhB4QeIxBOK3BVKHCAwlmf+2mm2tr92XEjcK1tZ4zHAz7PdXvh6lOQdpr2UWdy1eTTgNRM06gwjV2nZ7wq/9HORET7+mJlu9KovycuKvYpD5D4LBarKDpSqNs6g9aVtIEN5HmDg+88ZhbqRm55pjvoYhUR58mvTsM94aUxjJ7xXZGCLbVU1/gl6F4ByX65kerFIalT+qWEdrJ+sHskMCDPJ0LMj6AIvj5AXPhyKdZxPV8K1XfIkY4o3O/9azQpZLReHqEzsKiMLFm21SA1uw98uIW9yKFS6+qJ7PQV5b3DotLT1DrXUvlX0BqEudv2I+HEWe5qlX3CNne6q75LzVMdZx2VQI+U/WQkS9ESC2jqTJYH1B17aEFqVUxIqWtNdKZvkcL40lPsqBjS6sbpKU3T1VQpV+8c4P9cM60xvXYwDoEKbJKeI3cdY/sF24SkiOGFwFtiwg1 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB8PR04MB6795.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(376002)(136003)(346002)(39860400002)(66946007)(66556008)(66476007)(36756003)(6512007)(956004)(2616005)(966005)(478600001)(4326008)(86362001)(83380400001)(16526019)(6486002)(186003)(6666004)(38350700002)(316002)(5660300002)(8676002)(2906002)(8936002)(6506007)(38100700002)(26005)(1076003)(52116002)(69590400013); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QyTUx21fm2kQ6QIW0GNZOqrc2zEB9OvDocZ07v/63xvR5OJwNwkquM7T4X0rYd+RSAoRswLVzIy9B7WapBmK6YNwy/YHOzyS5pfMzrOtjG/ofZmMlioCS9yDTy2KdjA0nBeHxVf56ymjTJUiKChohycOc2iG7GBku7AifVWmbn0GCyGyKccVDa6YrFxpvwIhoKEhy4OtEiYaxW5k5CNcG1yPp0SoP9S6paDBjaWPuNynQJDhhSxzEgfIPPZg34ae8Oaii1c7R7vmB1ZwAPF6yuKGYru7d2EH/Z0eTCO1NUe+glfEV5keWG4vDF96vqK1TOkAymS7hSVapzFk1zJAcVxV7A84GZ8jMj2XOcd3+4UF2YqIQyyXcFH00L1v9SUY90/OpupVtsVLwxijTGGZtjiBDVsGFIj2bpQKUeCD6MvEKFFw7zjw/M5rgh7FU19oGHydVXIwQfjuYUbzXj1olxIOEqsDjmG+krT3WxEszI1wT/2lGGyVe+oaJEmHBRyDYz2ybZNxhzyL4Wb72TaOQIpwh5+op1hyb5nAlyTM+aPzam/U/xYhGO3uMaxwBUhvAX10putP17XLwhyXpg1CYvRfUjP/m1h0ZLb0+LAdnO9gVL+xux+rEOfGsOjvwAw3XdRdG3uDPN0NkAKFXxErVioKKs4C9oj5Slyn32nd7HbWeZiakLkBTY8sHBOGdI6FCmqRPkkD35WGP9zHwYv/cUbsqS1N+Du0ZREhawFLjOdFJKZ3AL2G8SjoA5I4ue8XIhe314w3Ga0HcEqI+nKzC7siD7FtpHg8jhwegXih+uX5xJv+yvP7OIOxgdrUwRQ2rKQcA1dhPF0vf6xHnpJmlajkkHDjmVCMNM5kzu/xNkCYChIbNPBtbMciwFlSvmMybe1Bnq0kL/NOuHHVxrld+EKYgtLTjlTHU4QavFlnFfSz0ssYBAIWflkwAUFDL9eum9CHeAPWMt/iKuwVIxhu0GvoyZHmFWd7RD+9ys8Q5AcNZ68bWSUUl6P1NHwIgTuYbCHif7n7Msjc1iaEejIAq+N2dQ0pLruetYNPh82VvScjOVT/knlH2XJda0NMlMiuK+jlseRnlqYOzj2Ob+FxOYcIBUEZtjS6SZWEqDvr1CmmkyiJce0c70X6NCHcAtc0m39uIOW9HwgCKP5DkBlEkXhDAaKat2oOkdhRwJNp6v/mf/KHblOR6ysgaErtRrAhQQFJUN3gckJ9ZRE3h3cvLdXdjraYKJP6NlKx5n13l3fp4RVdAIfUnLKXGPZ1WClItXcDoDD7wggJwEmxd0IYh0GW8Tq6VWRQvf+cE6E+wKzSaXRQaNTq8aYk+QFiUEmE X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: f924e6d2-47ff-46c4-3dae-08d9347dd6c0 X-MS-Exchange-CrossTenant-AuthSource: DB8PR04MB6795.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2021 06:28:52.9857 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Gg/ZhsNMAJuIhNcs0f4eat+w8okybEEW3NY74aArM61Blode7JktvQOGDD9l8Fk1cHlK0CDuiZ9Uxx3Cp25BAA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR04MB5883 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Frieder Schrempf reported a TX throuthput issue [1], it happens quite often that the measured bandwidth in TX direction drops from its expected/nominal value to something like ~50% (for 100M) or ~67% (for 1G) connections. [1] https://lore.kernel.org/linux-arm-kernel/421cc86c-b66f-b372-32f7-21e59f9a98bc@kontron.de/ The issue becomes clear after digging into it, Net core would select queues when transmitting packets. Since FEC have not impletemented ndo_select_queue callback yet, so it will call netdev_pick_tx to select queues randomly. For i.MX6SX ENET IP with AVB support, driver default enables this feature. According to the setting of QOS/RCMRn/DMAnCFG registers, AVB configured to Credit-based scheme, 50% bandwidth of each queue 1&2. With below tests let me think more: 1) With FEC_QUIRK_HAS_AVB quirk, can reproduce TX bandwidth fluctuations issue. 2) Without FEC_QUIRK_HAS_AVB quirk, can't reproduce TX bandwidth fluctuations issue. The related difference with or w/o FEC_QUIRK_HAS_AVB quirk is that, whether we program FTYPE field of TxBD or not. As I describe above, AVB feature is enabled by default. With FEC_QUIRK_HAS_AVB quirk, frames in queue 0 marked as non-AVB, and frames in queue 1&2 marked as AVB Class A&B. It's unreasonable if frames in queue 1&2 are not required to be time-sensitive. So when Net core select tx queues ramdomly, Credit-based scheme would work and lead to TX bandwidth fluctuated. On the other hand, w/o FEC_QUIRK_HAS_AVB quirk, frames in queue 1&2 are all marked as non-AVB, so Credit-based scheme would not work. Till now, how can we fix this TX throughput issue? Yes, please remove FEC_QUIRK_HAS_AVB quirk if you suffer it from time-nonsensitive networking. However, this quirk is used to indicate i.MX6SX, other setting depends on it. So this patch adds a new quirk FEC_QUIRK_HAS_MULTI_QUEUES to represent i.MX6SX, it is safe for us remove FEC_QUIRK_HAS_AVB quirk now. FEC_QUIRK_HAS_AVB quirk is set by default in the driver, and users may not know much about driver details, they would waste effort to find the root cause, that is not we want. The following patch is a implementation to fix it and users don't need to modify the driver. Tested-by: Frieder Schrempf Reported-by: Frieder Schrempf Signed-off-by: Joakim Zhang --- drivers/net/ethernet/freescale/fec.h | 5 +++++ drivers/net/ethernet/freescale/fec_main.c | 11 ++++++----- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/freescale/fec.h b/drivers/net/ethernet/freescale/fec.h index 0602d5d5d2ee..2e002e4b4b4a 100644 --- a/drivers/net/ethernet/freescale/fec.h +++ b/drivers/net/ethernet/freescale/fec.h @@ -467,6 +467,11 @@ struct bufdesc_ex { */ #define FEC_QUIRK_NO_HARD_RESET (1 << 18) +/* i.MX6SX ENET IP supports multiple queues (3 queues), use this quirk to + * represents this ENET IP. + */ +#define FEC_QUIRK_HAS_MULTI_QUEUES (1 << 19) + struct bufdesc_prop { int qid; /* Address of Rx and Tx buffers */ diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index ad82cffc6f3f..98cd38379275 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -122,7 +122,7 @@ static const struct fec_devinfo fec_imx6x_info = { FEC_QUIRK_HAS_VLAN | FEC_QUIRK_HAS_AVB | FEC_QUIRK_ERR007885 | FEC_QUIRK_BUG_CAPTURE | FEC_QUIRK_HAS_RACC | FEC_QUIRK_HAS_COALESCE | - FEC_QUIRK_CLEAR_SETUP_MII, + FEC_QUIRK_CLEAR_SETUP_MII | FEC_QUIRK_HAS_MULTI_QUEUES, }; static const struct fec_devinfo fec_imx6ul_info = { @@ -421,6 +421,7 @@ fec_enet_txq_submit_frag_skb(struct fec_enet_priv_tx_q *txq, estatus |= FEC_TX_BD_FTYPE(txq->bd.qid); if (skb->ip_summed == CHECKSUM_PARTIAL) estatus |= BD_ENET_TX_PINS | BD_ENET_TX_IINS; + ebdp->cbd_bdu = 0; ebdp->cbd_esc = cpu_to_fec32(estatus); } @@ -954,7 +955,7 @@ fec_restart(struct net_device *ndev) * For i.MX6SX SOC, enet use AXI bus, we use disable MAC * instead of reset MAC itself. */ - if (fep->quirks & FEC_QUIRK_HAS_AVB || + if (fep->quirks & FEC_QUIRK_HAS_MULTI_QUEUES || ((fep->quirks & FEC_QUIRK_NO_HARD_RESET) && fep->link)) { writel(0, fep->hwp + FEC_ECNTRL); } else { @@ -1165,7 +1166,7 @@ fec_stop(struct net_device *ndev) * instead of reset MAC itself. */ if (!(fep->wol_flag & FEC_WOL_FLAG_SLEEP_ON)) { - if (fep->quirks & FEC_QUIRK_HAS_AVB) { + if (fep->quirks & FEC_QUIRK_HAS_MULTI_QUEUES) { writel(0, fep->hwp + FEC_ECNTRL); } else { writel(1, fep->hwp + FEC_ECNTRL); @@ -2570,7 +2571,7 @@ static void fec_enet_itr_coal_set(struct net_device *ndev) writel(tx_itr, fep->hwp + FEC_TXIC0); writel(rx_itr, fep->hwp + FEC_RXIC0); - if (fep->quirks & FEC_QUIRK_HAS_AVB) { + if (fep->quirks & FEC_QUIRK_HAS_MULTI_QUEUES) { writel(tx_itr, fep->hwp + FEC_TXIC1); writel(rx_itr, fep->hwp + FEC_RXIC1); writel(tx_itr, fep->hwp + FEC_TXIC2); @@ -3371,7 +3372,7 @@ static int fec_enet_init(struct net_device *ndev) fep->csum_flags |= FLAG_RX_CSUM_ENABLED; } - if (fep->quirks & FEC_QUIRK_HAS_AVB) { + if (fep->quirks & FEC_QUIRK_HAS_MULTI_QUEUES) { fep->tx_align = 0; fep->rx_align = 0x3f; } From patchwork Mon Jun 21 06:27:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joakim Zhang X-Patchwork-Id: 464913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 006C3C48BE5 for ; Mon, 21 Jun 2021 06:29:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBCA161153 for ; Mon, 21 Jun 2021 06:29:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230186AbhFUGbN (ORCPT ); Mon, 21 Jun 2021 02:31:13 -0400 Received: from mail-eopbgr140040.outbound.protection.outlook.com ([40.107.14.40]:33280 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S229918AbhFUGbL (ORCPT ); Mon, 21 Jun 2021 02:31:11 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M8ghlJKlsXfzqPbHr9FOe4IcTBiqNfVsMZw4wFleMCTr5dVCCufDYnk+bct19+vUbmUYwjENSWvyQ+t6drC3eGRBR0duDA10kPoV/DBsz8WcnoYpYCvzjt//JgBBZyIepD1IMy8dxShkIPS597/aY8hLDrvKHrpBYciMaTCD30mm6uRD1Sk9cz5GuO3tSEZ4E0xLuqS7zcl3wp8x7sUIe7TmL2cbY5GTRQj1lxndbxGvR5v2rL666u+eDHNz1VexCSExiJQ82CV8yD3OvIwMUVUJswgYAtaHzb+0xcp7xeS5xMXZvmvm5qupRBiiQjdJbuZbhgr23cgNMutWhaB5vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ij/c4BMHC5yA75SWzXud2WwbvgL1bwuIbO3imqmEwIw=; b=fxukCvVVRRyVBNY0Sia8tG/fRmnYh81JqtaAC/ARJbTq7ol9HCzL6FTljTMgGwk1wIoYhpnhc1zJY3ZeOzaMsfMYkprIJlEx+Em19VCb1CeDAN6qltgkR8nVWyCoNUiigvrSXwqkcwncZ8TqDSKlHwgnIIgQH0bEo0YnAbNLSSpq0ok+HZzrgDEU3OldjohHruDPl5QtdMUBh2vyPQEkwKa14va8RN2KbzwYijQqydKmEUSy9W0PO9e8DuZ+BA72LmYFUbJpt4t3bsPu7OGmyZPHvecHhsUTF4vMCLACwdN6qEtCpxvmC3UdfHRS9x3wEE8X0rRY1F8yRhZB0gOdCg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ij/c4BMHC5yA75SWzXud2WwbvgL1bwuIbO3imqmEwIw=; b=LCOLu3WZka23k8fz+jnGOm5VrmRCNBXWPWbkXk5Lf8sKnpVhj9q2DWyKP9V4DNpHAQyHl+ZQLaGEnG8lQ2GcSkpimUrWpSseN6AxRoWaPhuFchNyleJ/Frvx2rhR5di5XzJ6i2gsaF4H8SNZh5WxtLl++ds8+lJgQnYl6RyZnZI= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=nxp.com; Received: from DB8PR04MB6795.eurprd04.prod.outlook.com (2603:10a6:10:fa::15) by DB8PR04MB5883.eurprd04.prod.outlook.com (2603:10a6:10:b1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.21; Mon, 21 Jun 2021 06:28:55 +0000 Received: from DB8PR04MB6795.eurprd04.prod.outlook.com ([fe80::f5e8:4886:3923:e92e]) by DB8PR04MB6795.eurprd04.prod.outlook.com ([fe80::f5e8:4886:3923:e92e%8]) with mapi id 15.20.4242.023; Mon, 21 Jun 2021 06:28:55 +0000 From: Joakim Zhang To: davem@davemloft.net, kuba@kernel.org, frieder.schrempf@kontron.de, andrew@lunn.ch Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-imx@nxp.com Subject: [PATCH V3 2/2] net: fec: add ndo_select_queue to fix TX bandwidth fluctuations Date: Mon, 21 Jun 2021 14:27:37 +0800 Message-Id: <20210621062737.16896-3-qiangqing.zhang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210621062737.16896-1-qiangqing.zhang@nxp.com> References: <20210621062737.16896-1-qiangqing.zhang@nxp.com> X-Originating-IP: [119.31.174.71] X-ClientProxiedBy: SGAP274CA0002.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b6::14) To DB8PR04MB6795.eurprd04.prod.outlook.com (2603:10a6:10:fa::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (119.31.174.71) by SGAP274CA0002.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.15 via Frontend Transport; Mon, 21 Jun 2021 06:28:53 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 558cf420-4429-44a4-6649-08d9347dd85c X-MS-TrafficTypeDiagnostic: DB8PR04MB5883: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eet8/KF5J6sIvt7quWs7IeljrgjLVBUoobMutEMiLrRh/Gycn+MpGjYtTQt7DwJA2jMCfGRw1Zp0wSdpXLdscfIC3SNMCKzsD64y1melK41tReL55Kqab244wOTLkbG81941HAwaFEQn8iFt+OO8Sci6bM6ol1JajJvosym0VXnVbI4YJ0MCf8C1ARkq64FKAnx3hO9l9Pm0NqZvHSdc7YejkVpTQDhch/U+UbXxmZfL0djZOT0BbtDNfmDaOgoqLFagh+FaoOBZdRCrhpTZw7+41yWdtLgqRtJ4Oa6+NBjVzerloOz86mB7HGcPQwIg8GDHp2uihg1RR2mlkYwCwcxo9oUioPGNhx8H2MPAX3xzkkJRkcnyQOSZqL9HnrS2MA8uK+75Lrf9OUaGpIfY2iUcPUT9Ykmr6GL1UucwJEta8rhy02B0MIHSTJpwq/0hMfGJb2Zb9TbCy8aPCWlNNSUQsBBtvazQ17FrX0My4LZ7ZTPAHFYK/BszdIDrB4tGTSGsh4H+vKIRgmIJyjvTwITpwfID2gBI8lGMk6x5vqoBm8Ituy0GB4dJc6TZjFCJTNBinWgiJawNZ1o6RUmhVzLLErirneJJwR3hJQ1r7KZLAniU6Ol6dO7pmHhixwa3WgF3iunjEZ9Dok09WEHk1GurYB+BztvcHx6I93RtkgOTSP2AitLUbsW5iGgrJi4U X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB8PR04MB6795.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(376002)(136003)(346002)(39860400002)(66946007)(66556008)(66476007)(36756003)(6512007)(956004)(2616005)(478600001)(4326008)(86362001)(83380400001)(16526019)(6486002)(186003)(6666004)(38350700002)(316002)(5660300002)(8676002)(2906002)(8936002)(6506007)(38100700002)(26005)(1076003)(52116002)(69590400013); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nf6kSvJS0CXEMKIztblVNH1Bkn1KrWW+w2AuutHM4Hp/jgXv/NJnJki1FYGunixcVbJqZQ3pzZl/9tSZzR95N9vF1oR5Hs+IasZZtYL4ybDK/9bgHtyJv8PgbrYY4RItzjkvT4U78MKCoGQOoZ2nXLDRLaUW48+MDFsmkZeyhe3oWNSQg8cdCQRMyiUhtG3RNeARLhRPq4L5Qq6LcFoflH/URj8bGZ+TsLVAihQt7FRY5Wd1pCm7cnKCzjK8d0o592rUqiBSMBXbUqRpB44e61gm1FtRFre0+tpNG4XQmSTgPSIEj4EBHJfATTL5z+EzdcaM4VzoAkViNp1oG2e3mFnPyCfkmsD0/3hsiTULRV8e/BgIJYly3S5Sc/9fHBI5r188ly98FxJ9VcMqI4/JUiSLwuSTIhjheur7DpH8K1GFLrRY/ZTDCLjghq5K84DaWautTNBt8nsN2yvhmV4+j6sEy20kqASux8XgSLWI8pudtQ8JfvctNpv2++DwWYCv+qhsnyoi/aadmPeZB8skO2PUyAGCoVtuL2NeW+bMhDC3oyfONuuSfM1Fl9MUm1NA5NVGAS+1qSROZpQsncJqKvwtk1bYtUtq9YNg2yX/hn2BBbVdVASgxaSyygg7ToTdSoH236Jol7vkNXyNBskaLn+BEsBTJ6hBQdIw1bCP97zD/4MmOe6nizCsmChtAvc5UvyWiBqV3/hReBI00uHOk+pVrbrC89dGYmORXS2a1i83zCnX3sYf+jHWKJtqmYiZ5KPIAvSonELODAqp/gPmvE83hB6E+W0nKxOiHQqsxfajk1GhD4dEDkDOiC57qVt1eiWYu0dnS2WtzKwy4scG7rTlXJrE9njK8A3ZvaFjh5yeIOT9EjDMUYQOGl26mAj76XRqASQ15yRLBvV7EKAU+3MhX1pAb1A1naM8dvO/pFzavkMocrjxnS1xnjcvAt1z2l7iSN7DcMjC+3pcA4ZzyMiLjnVOD/kabfWocszCt+UpjE+dkfCIolm+sdHQCMlRyfs99tATfsu1RmX9kqBkTxY8rTRakeTmWjpphr7Nw97ghDmgCwbR/YBY9q4FyLzLp8bTdv0iTYAI6JAYLfAit78PHgEWYRej8xochQHUu2oFTEdmkOJ+rAYk21Oi19JBc+maCoCyuKScoCUR9tFO66EqXfE/I6GtRS+p2Xa1QUiYHB9ZtxPIPpGK6B0VS5+rQ9HVbW8SsIW3DHAzXWh9cskBzLCZAkYue0ZuV7l/KhXYQe3S3iZIsQq92phK8m180ps9LwThsDNu5YsT6dI2ckfs6ZnzghGSe3PkCoy54O64iCfEmudB1zNrPV60UvqP X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 558cf420-4429-44a4-6649-08d9347dd85c X-MS-Exchange-CrossTenant-AuthSource: DB8PR04MB6795.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2021 06:28:55.6629 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XniXPoeqxZ9aEeJqNVkidCv6u9fIKjqK/BzU9QAMoB83xZBZc5llwXRHnUTrdxflwmUitzEwO/XcmPLz7ibJ7w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR04MB5883 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Fugang Duan As we know that AVB is enabled by default, and the ENET IP design is queue 0 for best effort, queue 1&2 for AVB Class A&B. Bandwidth of each queue 1&2 set in driver is 50%, TX bandwidth fluctuated when selecting tx queues randomly with FEC_QUIRK_HAS_AVB quirk available. This patch adds ndo_select_queue callback to select queues for transmitting to fix this issue. It will always return queue 0 if this is not a vlan packet, and return queue 1 or 2 based on priority of vlan packet. You may complain that in fact we only use single queue for trasmitting if we are not targeted to VLAN. Yes, but seems we have no choice, since AVB is enabled when the driver probed, we can't switch this feature dynamicly. After compare multiple queues to single queue, TX throughput almost no improvement. One way we can implemet is to configure the driver to multiple queues with Round-robin scheme by default. Then add ndo_setup_tc callback to enable/disable AVB feature for users. Unfortunately, ENET AVB IP seems not follow the standard 802.1Qav spec. We only can program DMAnCFG[IDLE_SLOPE] field to calculate bandwidth fraction. And idle slope is restricted to certain valus (a total of 19). It's far away from CBS QDisc implemented in Linux TC framework. If you strongly suggest to do this, I think we only can support limited numbers of bandwidth and reject others, but it's really urgly and wried. With this patch, VLAN tagged packets route to queue 0/1/2 based on vlan priority; VLAN untagged packets route to queue 0. Tested-by: Frieder Schrempf Reported-by: Frieder Schrempf Signed-off-by: Fugang Duan Signed-off-by: Joakim Zhang Reported-by: kernel test robot --- drivers/net/ethernet/freescale/fec_main.c | 32 +++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index 98cd38379275..8aea707a65a7 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -76,6 +76,8 @@ static void fec_enet_itr_coal_init(struct net_device *ndev); #define DRIVER_NAME "fec" +static const u16 fec_enet_vlan_pri_to_queue[8] = {0, 0, 1, 1, 1, 2, 2, 2}; + /* Pause frame feild and FIFO threshold */ #define FEC_ENET_FCE (1 << 5) #define FEC_ENET_RSEM_V 0x84 @@ -3240,10 +3242,40 @@ static int fec_set_features(struct net_device *netdev, return 0; } +static u16 fec_enet_get_raw_vlan_tci(struct sk_buff *skb) +{ + struct vlan_ethhdr *vhdr; + unsigned short vlan_TCI = 0; + + if (skb->protocol == htons(ETH_P_ALL)) { + vhdr = (struct vlan_ethhdr *)(skb->data); + vlan_TCI = ntohs(vhdr->h_vlan_TCI); + } + + return vlan_TCI; +} + +static u16 fec_enet_select_queue(struct net_device *ndev, struct sk_buff *skb, + struct net_device *sb_dev) +{ + struct fec_enet_private *fep = netdev_priv(ndev); + u16 vlan_tag; + + if (!(fep->quirks & FEC_QUIRK_HAS_AVB)) + return netdev_pick_tx(ndev, skb, NULL); + + vlan_tag = fec_enet_get_raw_vlan_tci(skb); + if (!vlan_tag) + return vlan_tag; + + return fec_enet_vlan_pri_to_queue[vlan_tag >> 13]; +} + static const struct net_device_ops fec_netdev_ops = { .ndo_open = fec_enet_open, .ndo_stop = fec_enet_close, .ndo_start_xmit = fec_enet_start_xmit, + .ndo_select_queue = fec_enet_select_queue, .ndo_set_rx_mode = set_multicast_list, .ndo_validate_addr = eth_validate_addr, .ndo_tx_timeout = fec_timeout,