From patchwork Fri Mar 12 19:46:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 399391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C562C433E9 for ; Fri, 12 Mar 2021 19:47:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F3EED64FC0 for ; Fri, 12 Mar 2021 19:47:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234456AbhCLTqg (ORCPT ); Fri, 12 Mar 2021 14:46:36 -0500 Received: from mail2.protonmail.ch ([185.70.40.22]:13533 "EHLO mail2.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234395AbhCLTq1 (ORCPT ); Fri, 12 Mar 2021 14:46:27 -0500 Date: Fri, 12 Mar 2021 19:46:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615578386; bh=ftpGwrO8lUj/+nf3ZEMhVVW9gOAMLofs/x8sHMcxcZI=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=YLIpjyXbRHOJR+M8H9IrEyKtNVJTJ/BxjwfScOG+Z8sNglh2C8R/wlJjmCeldzlN+ +DsMxE+PBwQfH9rhFHUZlcFEDXQWrT5kXTRTD5SMn3SEBdyUEIGP6oz89Ah3IvDJ7d 4GRiakvF9sQJSIhszF15g5vdaa3LoWqitEwLC1RePxuJ7q8d2HxixhvingNzPj8nQS ShDOGRWcx7JgqHoYxunGftQqinuPE9puiSahF54zQLZz/FcmYsDETKc3MMl6rFI6bq x6xggMbk5vshrAx6fmf5N/B4y/WbW+RpO27B6atmvOzYynnv76CDrk5ZkPkc68Xb49 yRY4pBdkR50nA== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Jonathan Lemon , Alexander Lobakin , Eric Dumazet , Willem de Bruijn , Kevin Hao , Pablo Neira Ayuso , Jakub Sitnicki , Marco Elver , Dexuan Cui , Vladimir Oltean , Ariel Levkovich , Wang Qing , Davide Caratti , Guillaume Nault , Eran Ben Elisha , Mauro Carvalho Chehab , Kirill Tkhai , Bartosz Golaszewski , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 1/6] flow_dissector: constify bpf_flow_dissector's data pointers Message-ID: <20210312194538.337504-2-alobakin@pm.me> In-Reply-To: <20210312194538.337504-1-alobakin@pm.me> References: <20210312194538.337504-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org BPF Flow dissection programs are read-only and don't touch input buffers. Mark @data and @data_end in struct bpf_flow_dissector as const in preparation for global input constifying. Signed-off-by: Alexander Lobakin --- include/net/flow_dissector.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.30.2 diff --git a/include/net/flow_dissector.h b/include/net/flow_dissector.h index cc10b10dc3a1..bf00e71816ed 100644 --- a/include/net/flow_dissector.h +++ b/include/net/flow_dissector.h @@ -368,8 +368,8 @@ static inline void *skb_flow_dissector_target(struct flow_dissector *flow_dissec struct bpf_flow_dissector { struct bpf_flow_keys *flow_keys; const struct sk_buff *skb; - void *data; - void *data_end; + const void *data; + const void *data_end; }; static inline void From patchwork Fri Mar 12 19:46:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 399390 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A615C4332D for ; Fri, 12 Mar 2021 19:47:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D0CC564F84 for ; Fri, 12 Mar 2021 19:47:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234565AbhCLTrR (ORCPT ); Fri, 12 Mar 2021 14:47:17 -0500 Received: from mail1.protonmail.ch ([185.70.40.18]:19338 "EHLO mail1.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234499AbhCLTqw (ORCPT ); Fri, 12 Mar 2021 14:46:52 -0500 Date: Fri, 12 Mar 2021 19:46:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615578410; bh=X3TuYcTaZPUU1Xz+TlNF44PidtjG5KU7YmET3XhmCM4=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=eMm2VsMlvn6slWnURKOEXNXqAP4812HwOHfFtntPxqoBrbcEPVuaUzU2gqSpKinQJ taTSblo/uuAw4Wlgry6aHr8bsNEoWYzsX+ASI+HkNVjFyCSB+vhPou858Te09LOZNL tNqJ11fNReVQV5P08iMJYc8xl4+c/M0DZsqcRZ0FNS3k/qPPRfRg/0V9bif2Eki76r NlUMkLDkjgO9Y5dkIMPUf+KlTDsOI0yIjlXX7UciI+BRRMzIG1gJ/t10YQRWrgVa6I FO2sEZC+1tnx6ZH6cTtQyhXn4lVJeo9QUg74mVth6i/T9JOhmBS5r75nHoyLPG93QK qGn2saS8EYtCQ== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Jonathan Lemon , Alexander Lobakin , Eric Dumazet , Willem de Bruijn , Kevin Hao , Pablo Neira Ayuso , Jakub Sitnicki , Marco Elver , Dexuan Cui , Vladimir Oltean , Ariel Levkovich , Wang Qing , Davide Caratti , Guillaume Nault , Eran Ben Elisha , Mauro Carvalho Chehab , Kirill Tkhai , Bartosz Golaszewski , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup Message-ID: <20210312194538.337504-5-alobakin@pm.me> In-Reply-To: <20210312194538.337504-1-alobakin@pm.me> References: <20210312194538.337504-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Caught by the text editor. Fix it separately from the actual changes. Signed-off-by: Alexander Lobakin --- include/linux/etherdevice.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.30.2 diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h index 2e5debc0373c..bcb2f81baafb 100644 --- a/include/linux/etherdevice.h +++ b/include/linux/etherdevice.h @@ -11,7 +11,7 @@ * Authors: Ross Biro * Fred N. van Kempen, * - * Relocated to include/linux where it belongs by Alan Cox + * Relocated to include/linux where it belongs by Alan Cox * */ #ifndef _LINUX_ETHERDEVICE_H From patchwork Fri Mar 12 19:46:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 399387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE047C433E6 for ; Fri, 12 Mar 2021 20:01:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB92D64F8D for ; Fri, 12 Mar 2021 20:01:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234567AbhCLUAf (ORCPT ); Fri, 12 Mar 2021 15:00:35 -0500 Received: from mail-40136.protonmail.ch ([185.70.40.136]:62182 "EHLO mail-40136.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234517AbhCLTrD (ORCPT ); Fri, 12 Mar 2021 14:47:03 -0500 Date: Fri, 12 Mar 2021 19:46:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615578421; bh=OtoaEa4C9A8M8IjnX8ICb3j9vVq3aPxG0RyT+eIX9OU=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=V85xkCPVeVQhvYy7zZb5ATiY7ybI8IaO1ADBBYQH3uO/HtG021dPYP9SF81OIwtiQ 2oOyTTEEgpLjTxUzUcPpRzPPkLYBlRyrcTnF0d1gjg8uxa8TDdXL94S367o7aePwy2 svwpdW7jzhiFQTudYqjxyjEj4Z0DWrtuJIrZM0cdyGjVBIu120xJaq0pPwnBsU/l5X lmHSCJHppAY6v4k0MkB60ShTWnTp+XbuIrw7nK18k6GZ91p0fSX1/qcLVqEfJ9Wrb0 kRUGz2BlJe9BZYsc/oVuQHtmoTJHsVl1KQ1yFp5EgDsftLl/Vc2M7o49F0VG/VFJFC SA/RO78NuKUZA== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Jonathan Lemon , Alexander Lobakin , Eric Dumazet , Willem de Bruijn , Kevin Hao , Pablo Neira Ayuso , Jakub Sitnicki , Marco Elver , Dexuan Cui , Vladimir Oltean , Ariel Levkovich , Wang Qing , Davide Caratti , Guillaume Nault , Eran Ben Elisha , Mauro Carvalho Chehab , Kirill Tkhai , Bartosz Golaszewski , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 5/6] ethernet: constify eth_get_headlen()'s @data argument Message-ID: <20210312194538.337504-6-alobakin@pm.me> In-Reply-To: <20210312194538.337504-1-alobakin@pm.me> References: <20210312194538.337504-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org It's used only for flow dissection, which now takes constant data pointers. Signed-off-by: Alexander Lobakin --- include/linux/etherdevice.h | 2 +- net/ethernet/eth.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.30.2 diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h index bcb2f81baafb..330345b1be54 100644 --- a/include/linux/etherdevice.h +++ b/include/linux/etherdevice.h @@ -29,7 +29,7 @@ struct device; int eth_platform_get_mac_address(struct device *dev, u8 *mac_addr); unsigned char *arch_get_platform_mac_address(void); int nvmem_get_mac_address(struct device *dev, void *addrbuf); -u32 eth_get_headlen(const struct net_device *dev, void *data, unsigned int len); +u32 eth_get_headlen(const struct net_device *dev, const void *data, u32 len); __be16 eth_type_trans(struct sk_buff *skb, struct net_device *dev); extern const struct header_ops eth_header_ops; diff --git a/net/ethernet/eth.c b/net/ethernet/eth.c index 4106373180c6..e01cf766d2c5 100644 --- a/net/ethernet/eth.c +++ b/net/ethernet/eth.c @@ -122,7 +122,7 @@ EXPORT_SYMBOL(eth_header); * Make a best effort attempt to pull the length for all of the headers for * a given frame in a linear buffer. */ -u32 eth_get_headlen(const struct net_device *dev, void *data, unsigned int len) +u32 eth_get_headlen(const struct net_device *dev, const void *data, u32 len) { const unsigned int flags = FLOW_DISSECTOR_F_PARSE_1ST_FRAG; const struct ethhdr *eth = (const struct ethhdr *)data; From patchwork Fri Mar 12 19:47:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 399389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D9B7C433E0 for ; Fri, 12 Mar 2021 19:48:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7DBD64F83 for ; Fri, 12 Mar 2021 19:48:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234501AbhCLTrm (ORCPT ); Fri, 12 Mar 2021 14:47:42 -0500 Received: from mail-40131.protonmail.ch ([185.70.40.131]:49061 "EHLO mail-40131.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234486AbhCLTrN (ORCPT ); Fri, 12 Mar 2021 14:47:13 -0500 Date: Fri, 12 Mar 2021 19:47:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615578431; bh=0WXUDRS6l3uzo7FR/B86Gske9XTm/nKNi8/gZiVufio=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=W0fw0vAzGEdIe1NRTwFI0Zc+jgpfNGER5XVeToKKqjERriU/eabdNJjFcYaPzIYvR sgsHppSchZl7I1AchfFUvTTIHfHiQn6wcnBeP+CKBgsdVAPF/raBbAKhLA0Rd/x3uU TcM7ZMb+BjPva1FjJYCBHlK9NXuEl44vhYJ1GfasZIITzCKPNlW18uvDcvV5BsWGz2 R6FHupHuS4DYidzdmyScY9++8sttJFJWdffJ0skHDWhN1dgUfY7saizL7brVRZF0sP oUQCymviXI5/PNSi+Crt42y1nYhSAccdY+hbPhzVKl3UzxAydlQ8qF2tk0XFRAq85H 8/P633Zz1TQtg== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Jonathan Lemon , Alexander Lobakin , Eric Dumazet , Willem de Bruijn , Kevin Hao , Pablo Neira Ayuso , Jakub Sitnicki , Marco Elver , Dexuan Cui , Vladimir Oltean , Ariel Levkovich , Wang Qing , Davide Caratti , Guillaume Nault , Eran Ben Elisha , Mauro Carvalho Chehab , Kirill Tkhai , Bartosz Golaszewski , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 6/6] skbuff: micro-optimize {, __}skb_header_pointer() Message-ID: <20210312194538.337504-7-alobakin@pm.me> In-Reply-To: <20210312194538.337504-1-alobakin@pm.me> References: <20210312194538.337504-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org {,__}skb_header_pointer() helpers exist mainly for preventing accesses-beyond-end of the linear data. In the vast majorify of cases, they bail out on the first condition. All code going after is mostly a fallback. Mark the most common branch as 'likely' one to move it in-line. Also, skb_copy_bits() can return negative values only when the input arguments are invalid, e.g. offset is greater than skb->len. It can be safely marked as 'unlikely' branch, assuming that hotpath code provides sane input to not fail here. These two bump the throughput with a single Flow Dissector pass on every packet (e.g. with RPS or driver that uses eth_get_headlen()) on 20 Mbps per flow/core. Signed-off-by: Alexander Lobakin --- include/linux/skbuff.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) -- 2.30.2 diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 7873f24c0ae5..71f4d609819e 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3680,11 +3680,10 @@ static inline void * __must_check __skb_header_pointer(const struct sk_buff *skb, int offset, int len, const void *data, int hlen, void *buffer) { - if (hlen - offset >= len) + if (likely(hlen - offset >= len)) return (void *)data + offset; - if (!skb || - skb_copy_bits(skb, offset, buffer, len) < 0) + if (!skb || unlikely(skb_copy_bits(skb, offset, buffer, len) < 0)) return NULL; return buffer;