From patchwork Wed Jun 9 13:47:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matteo Croce X-Patchwork-Id: 457404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96056C48BCF for ; Wed, 9 Jun 2021 13:48:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7DD0A61364 for ; Wed, 9 Jun 2021 13:48:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236379AbhFINuV (ORCPT ); Wed, 9 Jun 2021 09:50:21 -0400 Received: from mail-wm1-f41.google.com ([209.85.128.41]:38423 "EHLO mail-wm1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232911AbhFINuQ (ORCPT ); Wed, 9 Jun 2021 09:50:16 -0400 Received: by mail-wm1-f41.google.com with SMTP id t4-20020a1c77040000b029019d22d84ebdso4333110wmi.3; Wed, 09 Jun 2021 06:48:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ESskJJKWWlduLfjCtKwaFwkDwGwCfUguoy2FpDTie+U=; b=ktisJstPHmRLQbwP+cBMho7+TeT7LzqBXyVxdRx3tllWah03ZfzuSZfTSZsiMAp7wB CoEHLcHx+B1SOaZVG+t5LE2ucZKHpiUA6vfYhmR0RuX2+JdtQ7u5jjgVZkCvnQF7ZJ5a GbvmZoCo+6g/LC2ocIKd0pKhctR9YooHeLZpPIf+XosovcqCgE1RcVQGIz0pPXSfrBfw 5eoF9lmAuPSOLaMCZcYhlU6hWbkOXwBsLcvL7h5Q4gRJBBbzP4IaT4wPHYKLf8ShoxiI hnXazIPUZ4jsOH/SKbEi+Py5NPFZ0Yr6uG+55MfJ8VQ81IGLf8ud3mCeiZJ3tHiVFZlg XjDA== X-Gm-Message-State: AOAM533tSwUmi8IGtik4eKH1d14UAFZrKM8BbBl/iLnUWSpBBmngFQOd kDKU3sUZZFbPa2n8v4jzOFsBtzmuzbKksQ== X-Google-Smtp-Source: ABdhPJzUFqClOwwnV3jDQgDgg8y0vsvrmryRlJh82cCs1Yqy4hfuoAr6jQ8xXMvcRpFt1KdxqvrMIw== X-Received: by 2002:a05:600c:1c22:: with SMTP id j34mr10239192wms.166.1623246492759; Wed, 09 Jun 2021 06:48:12 -0700 (PDT) Received: from msft-t490s.teknoraver.net (net-37-119-128-179.cust.vodafonedsl.it. [37.119.128.179]) by smtp.gmail.com with ESMTPSA id o5sm13882351wrw.65.2021.06.09.06.48.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 06:48:12 -0700 (PDT) From: Matteo Croce To: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Marcin Wojtas , Russell King , "David S. Miller" , Jakub Kicinski , Sven Auhagen Subject: [PATCH net-next 1/2] mvpp2: prefetch right address Date: Wed, 9 Jun 2021 15:47:13 +0200 Message-Id: <20210609134714.13715-2-mcroce@linux.microsoft.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609134714.13715-1-mcroce@linux.microsoft.com> References: <20210609134714.13715-1-mcroce@linux.microsoft.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Matteo Croce In the RX buffer, the received data starts after a headroom used to align the IP header and to allow prepending headers efficiently. The prefetch() should take this into account, and prefetch from the very start of the received data. We can see that ether_addr_equal_64bits(), which is the first function to access the data, drops from the top of the perf top output. prefetch(data): Overhead Shared Object Symbol 11.64% [kernel] [k] eth_type_trans prefetch(data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM): Overhead Shared Object Symbol 13.42% [kernel] [k] build_skb 10.35% [mvpp2] [k] mvpp2_rx 9.35% [kernel] [k] __netif_receive_skb_core 8.24% [kernel] [k] kmem_cache_free 7.97% [kernel] [k] dev_gro_receive 7.68% [kernel] [k] page_pool_put_page 7.32% [kernel] [k] kmem_cache_alloc 7.09% [mvpp2] [k] mvpp2_bm_pool_put 3.36% [kernel] [k] eth_type_trans Also, move the eth_type_trans() call a bit down, to give the RAM more time to prefetch the data. Signed-off-by: Matteo Croce --- drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index 5663c1b21870..07d8f3e31b52 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -3938,7 +3938,7 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, goto err_drop_frame; /* Prefetch header */ - prefetch(data); + prefetch(data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM); if (bm_pool->frag_size > PAGE_SIZE) frag_size = 0; @@ -4008,8 +4008,8 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, skb_reserve(skb, MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM); skb_put(skb, rx_bytes); - skb->protocol = eth_type_trans(skb, dev); mvpp2_rx_csum(port, rx_status, skb); + skb->protocol = eth_type_trans(skb, dev); napi_gro_receive(napi, skb); continue; From patchwork Wed Jun 9 13:47:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matteo Croce X-Patchwork-Id: 458433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D240FC48BCF for ; Wed, 9 Jun 2021 13:48:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BDF7C61364 for ; Wed, 9 Jun 2021 13:48:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236624AbhFINuO (ORCPT ); Wed, 9 Jun 2021 09:50:14 -0400 Received: from mail-wr1-f51.google.com ([209.85.221.51]:45611 "EHLO mail-wr1-f51.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232911AbhFINuJ (ORCPT ); Wed, 9 Jun 2021 09:50:09 -0400 Received: by mail-wr1-f51.google.com with SMTP id z8so25531988wrp.12; Wed, 09 Jun 2021 06:48:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ob6PnWhKav7jMFqFY+b+WTV4eaWghvwiq60OTVa4u2A=; b=GYHSeZPo34jeBLHh/sCUIdxO1hQZh36/JnIyZDl0LES2nd23mJwjBw3GSlGkBfOQJx sR8vGgFXr+jt+aMIR3dvvAzZep2RdTIGtGjLZvB3GlJvlqUzG+i/DqPfqKxVlRzKa1uk J36X2QQ6rlpPy7AplDQ2dosl9aMrEG09mPl7Q+XmVpFkqbKpo7wMxl92lPHZh8FtmEy4 dGp+t+CIoXh6d23izgxdotfQbERpnyTbftiFvEbGT2bMUlPmHghsM4A+/kpVvznSG+sN aF1TRHZbn0luDb5XzVQ8ZDLG4VdQLFrGjJR1DT/UBDcFLBwbpjMbu7kF/uwYtgJv8RC8 nZSA== X-Gm-Message-State: AOAM5301qvP4bTOAQxGu0WjirDJp039bqUg1hk8u7UdQPR/0JcdETMxT yGVYEChX2yKpqdLfGYCm8uRPsMBkht+EXg== X-Google-Smtp-Source: ABdhPJyaQfaop0D//baox/2aMwkJjAdH0+kRl4J2HwWcdWYJxkqS9EQfqvwqrw4W6zuMhvoxcFMRZw== X-Received: by 2002:adf:f78d:: with SMTP id q13mr27990870wrp.191.1623246493659; Wed, 09 Jun 2021 06:48:13 -0700 (PDT) Received: from msft-t490s.teknoraver.net (net-37-119-128-179.cust.vodafonedsl.it. [37.119.128.179]) by smtp.gmail.com with ESMTPSA id o5sm13882351wrw.65.2021.06.09.06.48.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 06:48:13 -0700 (PDT) From: Matteo Croce To: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Marcin Wojtas , Russell King , "David S. Miller" , Jakub Kicinski , Sven Auhagen Subject: [PATCH net-next 2/2] mvpp2: prefetch page Date: Wed, 9 Jun 2021 15:47:14 +0200 Message-Id: <20210609134714.13715-3-mcroce@linux.microsoft.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609134714.13715-1-mcroce@linux.microsoft.com> References: <20210609134714.13715-1-mcroce@linux.microsoft.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Matteo Croce Most of the time during the RX is caused by the compound_head() call done at the end of the RX loop: │ build_skb(): [...] │ static inline struct page *compound_head(struct page *page) │ { │ unsigned long head = READ_ONCE(page->compound_head); 65.23 │ ldr x2, [x1, #8] Prefetch the page struct as soon as possible, to speedup the RX path noticeabily by a ~3-4% packet rate in a drop test. │ build_skb(): [...] │ static inline struct page *compound_head(struct page *page) │ { │ unsigned long head = READ_ONCE(page->compound_head); 17.92 │ ldr x2, [x1, #8] Signed-off-by: Matteo Croce --- drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index 07d8f3e31b52..9bca8c8f9f8d 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -3900,15 +3900,19 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, phys_addr_t phys_addr; u32 rx_status, timestamp; int pool, rx_bytes, err, ret; + struct page *page; void *data; + phys_addr = mvpp2_rxdesc_cookie_get(port, rx_desc); + data = (void *)phys_to_virt(phys_addr); + page = virt_to_page(data); + prefetch(page); + rx_done++; rx_status = mvpp2_rxdesc_status_get(port, rx_desc); rx_bytes = mvpp2_rxdesc_size_get(port, rx_desc); rx_bytes -= MVPP2_MH_SIZE; dma_addr = mvpp2_rxdesc_dma_addr_get(port, rx_desc); - phys_addr = mvpp2_rxdesc_cookie_get(port, rx_desc); - data = (void *)phys_to_virt(phys_addr); pool = (rx_status & MVPP2_RXD_BM_POOL_ID_MASK) >> MVPP2_RXD_BM_POOL_ID_OFFS; @@ -3997,7 +4001,7 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, } if (pp) - skb_mark_for_recycle(skb, virt_to_page(data), pp); + skb_mark_for_recycle(skb, page, pp); else dma_unmap_single_attrs(dev->dev.parent, dma_addr, bm_pool->buf_size, DMA_FROM_DEVICE,