From patchwork Fri Feb 21 17:44:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 868873 Received: from mta-65-226.siemens.flowmailer.net (mta-65-226.siemens.flowmailer.net [185.136.65.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0635A250BE2 for ; Fri, 21 Feb 2025 17:44:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.136.65.226 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740159879; cv=none; b=pYB4psgs51bmXpd6CJjVRst2T5zIOEbzFiFAxEu6Qc7IMKbBd2d0lsMlruQzx7Dagz9EXxBEplBtIuQ+RFaaCbPwnFlGbWrRaYLuQ1vtNJlku8bO+7tljdmv6r5SmROee37kcb3ZnzBSYB+UH3/f7JIsg8tx1KK1KOqeSi2tYLI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740159879; c=relaxed/simple; bh=/CjSY5t2hAmWsRakO82xJ2VQmD+iQWuE+ehvrsRELRQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V8ywymtUOu07kfsScGvOWYeKsLXYvB+tuCnLAU//gKcf1yEH/EySbFWmJK0ay/tSllBv+qQjx4Eo6lNXYbe4vqLEMf39TovCTxyWKiUJUIYj1kjJTb79X5YdvWcgSNKd2YWgvqKF9Mh8CrFzsStLMJJU2q6XSk4K7H6GFR27C8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=siemens.com; spf=pass smtp.mailfrom=rts-flowmailer.siemens.com; dkim=pass (2048-bit key) header.d=siemens.com header.i=jan.kiszka@siemens.com header.b=aFrg1IZC; arc=none smtp.client-ip=185.136.65.226 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=siemens.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rts-flowmailer.siemens.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=siemens.com header.i=jan.kiszka@siemens.com header.b="aFrg1IZC" Received: by mta-65-226.siemens.flowmailer.net with ESMTPSA id 202502211744282274a47580795f54dc for ; Fri, 21 Feb 2025 18:44:28 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=jan.kiszka@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=27cs8I+6qCV9+Ed2Jt7ukHjW8TQgn0i2uzLEi23vu54=; b=aFrg1IZCqzKUB7DntC0sREi0HzJWzR+p93h9v8jWeKdad2g7ZXksPLmdiFZeyjNNpztdwg nAnwZRJUTtslQw6bmYmKkodEBIeYlC0RNbeYrAHXKRE89gsWfHsofpGkaKGuUYh19U0MC+hV /p20MW24NYFirBoEsOr2KNZUXoiRcXOlBL8mXpceUwMeqcw3/hoFO4rGwr1h1pXCBcMC4yWq soQEEXJbqLU2vGaSsnkTMvMS3F0X6VznGXLSbfv8Q0AEwgw56p5+a9vSfyO6MKhaDocp/SxC C9Qjx3LHNkmn1MmDd0cmw6Oh83AHNktax/nixRkLf4THjNIWrtp/8D/g==; From: Jan Kiszka To: "Luis Claudio R . Goncalves" , stable-rt@vger.kernel.org, Pavel Machek Cc: linux-kernel@vger.kernel.org, linux-rt-users , Clark Williams , Steven Rostedt , Joseph Salisbury , Florian Bezdeka , Felix Moessbauer , Andreas Ziegler Subject: [5.10-rt][PATCH 1/2] u64_stats: Introduce u64_stats_set() Date: Fri, 21 Feb 2025 18:44:25 +0100 Message-ID: In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-rt-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-294854:519-21489:flowmailer From: Ahmed S. Darwish [ Upstream commit f2efdb17928924c9c935c136dea764a081032006 ] Allow to directly set a u64_stats_t value which is used to provide an init function which sets it directly to zero intead of memset() the value. Add u64_stats_set() to the u64_stats API. [bigeasy: commit message. ] Signed-off-by: Ahmed S. Darwish Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: David S. Miller Signed-off-by: Jan Kiszka --- include/linux/u64_stats_sync.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/include/linux/u64_stats_sync.h b/include/linux/u64_stats_sync.h index f23192d555285..af6fc08d2fcac 100644 --- a/include/linux/u64_stats_sync.h +++ b/include/linux/u64_stats_sync.h @@ -83,6 +83,11 @@ static inline u64 u64_stats_read(const u64_stats_t *p) return local64_read(&p->v); } +static inline void u64_stats_set(u64_stats_t *p, u64 val) +{ + local64_set(&p->v, val); +} + static inline void u64_stats_add(u64_stats_t *p, unsigned long val) { local64_add(val, &p->v); @@ -104,6 +109,11 @@ static inline u64 u64_stats_read(const u64_stats_t *p) return p->v; } +static inline void u64_stats_set(u64_stats_t *p, u64 val) +{ + p->v = val; +} + static inline void u64_stats_add(u64_stats_t *p, unsigned long val) { p->v += val; From patchwork Fri Feb 21 17:44:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 867352 Received: from mta-64-226.siemens.flowmailer.net (mta-64-226.siemens.flowmailer.net [185.136.64.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19BC6250BE6 for ; Fri, 21 Feb 2025 17:44:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.136.64.226 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740159880; cv=none; b=l4hS6gR6kjmbEgdu4iyX9tAQb22bopFCrRMf3fespnCvRlca4gCfExqK+F0Qbg+aD3YbvXqH1uhHE5nvU+8LsDA+GIlp3bhzYDr2sEnuEieoTJwRjto3p44CDA5Naislj1YmZAz1mtr3i43lwjmqYy6cPJOkXLraIF6icNXrSKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740159880; c=relaxed/simple; bh=V4PGUwG5vJKhMkA07GY2KCfo07Y0e4cWOF0kQlVop2E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gdVvN0/OCAllWGgzSamSxu9WcMlqn3vM+vLcf7qjRp/0ATgYAh8xvJq4fYweqWW+V2HbztWrGGiXekt6quyVJwVZ4kk/ivtOsQZD71mGdaWyFnGtmXjQBVd3h4nCUFWAp8eL8a0u8JwN0bhc2VUeM3vGqby5lwK42VlH2KE23VE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=siemens.com; spf=pass smtp.mailfrom=rts-flowmailer.siemens.com; dkim=pass (2048-bit key) header.d=siemens.com header.i=jan.kiszka@siemens.com header.b=Q6TWxiQW; arc=none smtp.client-ip=185.136.64.226 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=siemens.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rts-flowmailer.siemens.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=siemens.com header.i=jan.kiszka@siemens.com header.b="Q6TWxiQW" Received: by mta-64-226.siemens.flowmailer.net with ESMTPSA id 202502211744292260dbc0b1d9d074c9 for ; Fri, 21 Feb 2025 18:44:29 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=jan.kiszka@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=AzPvl9mIXsb0UCgAsgejP02nf5bWd33xa25QZ7WkoAk=; b=Q6TWxiQW2rbppakzGvNvXRoQk+LKnRyLpwpf26iVKk+vREcf00F11tW2gffCmSdgXYiZtn KS2QBnnGu2nmXpF/tgBtZGCRUJGyhoynLnRhZQe//kRKDmqr2eZR8XmhXmjGtd3Nttz4y4wl wboNzpB7V+U/DpyTXJsBD5ul5utbd8pC7OZfJm3RDtsQjztoSFEFASn4G8YVCo8M744h2P5/ 58DxAcCZ9/bjPhpmpd0voOnGVR6ZRGkuvO681XXYIBQdXjUeqoIUsakt7Y5QQRatK+mh6vlY BRmnMqaojbSLRANZBqlSIhHvQS/jRoAxiRh8Ww4InRQXlqnDgX/5y4tg==; From: Jan Kiszka To: "Luis Claudio R . Goncalves" , stable-rt@vger.kernel.org, Pavel Machek Cc: linux-kernel@vger.kernel.org, linux-rt-users , Clark Williams , Steven Rostedt , Joseph Salisbury , Florian Bezdeka , Felix Moessbauer , Andreas Ziegler , Eric Dumazet Subject: [5.10-rt][PATCH 2/2] netfilter: nft_counter: Use u64_stats_t for statistic. Date: Fri, 21 Feb 2025 18:44:26 +0100 Message-ID: <4b7e265f787b5f3519aa59ed202c6f41b2e4fcd0.1740159866.git.jan.kiszka@siemens.com> In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-rt-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-294854:519-21489:flowmailer From: Sebastian Andrzej Siewior [ Upstream commit 4a1d3acd6ea86075e77fcc1188c3fc372833ba73 ] The nft_counter uses two s64 counters for statistics. Those two are protected by a seqcount to ensure that the 64bit variable is always properly seen during updates even on 32bit architectures where the store is performed by two writes. A side effect is that the two counter (bytes and packet) are written and read together in the same window. This can be replaced with u64_stats_t. write_seqcount_begin()/ end() is replaced with u64_stats_update_begin()/ end() and behaves the same way as with seqcount_t on 32bit architectures. Additionally there is a preempt_disable on PREEMPT_RT to ensure that a reader does not preempt a writer. On 64bit architectures the macros are removed and the reads happen without any retries. This also means that the reader can observe one counter (bytes) from before the update and the other counter (packets) but that is okay since there is no requirement to have both counter from the same update window. Convert the statistic to u64_stats_t. There is one optimisation: nft_counter_do_init() and nft_counter_clone() allocate a new per-CPU counter and assign a value to it. During this assignment preemption is disabled which is not needed because the counter is not yet exposed to the system so there can not be another writer or reader. Therefore disabling preemption is omitted and raw_cpu_ptr() is used to obtain a pointer to a counter for the assignment. Cc: Eric Dumazet Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Pablo Neira Ayuso Signed-off-by: Jan Kiszka --- net/netfilter/nft_counter.c | 78 +++++++++++++++++++------------------ 1 file changed, 40 insertions(+), 38 deletions(-) diff --git a/net/netfilter/nft_counter.c b/net/netfilter/nft_counter.c index ea102b9a168b8..79508b5f71d17 100644 --- a/net/netfilter/nft_counter.c +++ b/net/netfilter/nft_counter.c @@ -8,13 +8,18 @@ #include #include #include -#include +#include #include #include #include #include struct nft_counter { + u64_stats_t bytes; + u64_stats_t packets; +}; + +struct nft_counter_tot { s64 bytes; s64 packets; }; @@ -23,25 +28,24 @@ struct nft_counter_percpu_priv { struct nft_counter __percpu *counter; }; -static DEFINE_PER_CPU(seqcount_t, nft_counter_seq); +static DEFINE_PER_CPU(struct u64_stats_sync, nft_counter_sync); static inline void nft_counter_do_eval(struct nft_counter_percpu_priv *priv, struct nft_regs *regs, const struct nft_pktinfo *pkt) { + struct u64_stats_sync *nft_sync; struct nft_counter *this_cpu; - seqcount_t *myseq; local_bh_disable(); this_cpu = this_cpu_ptr(priv->counter); - myseq = this_cpu_ptr(&nft_counter_seq); - - write_seqcount_begin(myseq); + nft_sync = this_cpu_ptr(&nft_counter_sync); - this_cpu->bytes += pkt->skb->len; - this_cpu->packets++; + u64_stats_update_begin(nft_sync); + u64_stats_add(&this_cpu->bytes, pkt->skb->len); + u64_stats_inc(&this_cpu->packets); + u64_stats_update_end(nft_sync); - write_seqcount_end(myseq); local_bh_enable(); } @@ -64,17 +68,16 @@ static int nft_counter_do_init(const struct nlattr * const tb[], if (cpu_stats == NULL) return -ENOMEM; - preempt_disable(); - this_cpu = this_cpu_ptr(cpu_stats); + this_cpu = raw_cpu_ptr(cpu_stats); if (tb[NFTA_COUNTER_PACKETS]) { - this_cpu->packets = - be64_to_cpu(nla_get_be64(tb[NFTA_COUNTER_PACKETS])); + u64_stats_set(&this_cpu->packets, + be64_to_cpu(nla_get_be64(tb[NFTA_COUNTER_PACKETS]))); } if (tb[NFTA_COUNTER_BYTES]) { - this_cpu->bytes = - be64_to_cpu(nla_get_be64(tb[NFTA_COUNTER_BYTES])); + u64_stats_set(&this_cpu->bytes, + be64_to_cpu(nla_get_be64(tb[NFTA_COUNTER_BYTES]))); } - preempt_enable(); + priv->counter = cpu_stats; return 0; } @@ -102,40 +105,41 @@ static void nft_counter_obj_destroy(const struct nft_ctx *ctx, } static void nft_counter_reset(struct nft_counter_percpu_priv *priv, - struct nft_counter *total) + struct nft_counter_tot *total) { + struct u64_stats_sync *nft_sync; struct nft_counter *this_cpu; - seqcount_t *myseq; local_bh_disable(); this_cpu = this_cpu_ptr(priv->counter); - myseq = this_cpu_ptr(&nft_counter_seq); + nft_sync = this_cpu_ptr(&nft_counter_sync); + + u64_stats_update_begin(nft_sync); + u64_stats_add(&this_cpu->packets, -total->packets); + u64_stats_add(&this_cpu->bytes, -total->bytes); + u64_stats_update_end(nft_sync); - write_seqcount_begin(myseq); - this_cpu->packets -= total->packets; - this_cpu->bytes -= total->bytes; - write_seqcount_end(myseq); local_bh_enable(); } static void nft_counter_fetch(struct nft_counter_percpu_priv *priv, - struct nft_counter *total) + struct nft_counter_tot *total) { struct nft_counter *this_cpu; - const seqcount_t *myseq; u64 bytes, packets; unsigned int seq; int cpu; memset(total, 0, sizeof(*total)); for_each_possible_cpu(cpu) { - myseq = per_cpu_ptr(&nft_counter_seq, cpu); + struct u64_stats_sync *nft_sync = per_cpu_ptr(&nft_counter_sync, cpu); + this_cpu = per_cpu_ptr(priv->counter, cpu); do { - seq = read_seqcount_begin(myseq); - bytes = this_cpu->bytes; - packets = this_cpu->packets; - } while (read_seqcount_retry(myseq, seq)); + seq = u64_stats_fetch_begin(nft_sync); + bytes = u64_stats_read(&this_cpu->bytes); + packets = u64_stats_read(&this_cpu->packets); + } while (u64_stats_fetch_retry(nft_sync, seq)); total->bytes += bytes; total->packets += packets; @@ -146,7 +150,7 @@ static int nft_counter_do_dump(struct sk_buff *skb, struct nft_counter_percpu_priv *priv, bool reset) { - struct nft_counter total; + struct nft_counter_tot total; nft_counter_fetch(priv, &total); @@ -235,7 +239,7 @@ static int nft_counter_clone(struct nft_expr *dst, const struct nft_expr *src, g struct nft_counter_percpu_priv *priv_clone = nft_expr_priv(dst); struct nft_counter __percpu *cpu_stats; struct nft_counter *this_cpu; - struct nft_counter total; + struct nft_counter_tot total; nft_counter_fetch(priv, &total); @@ -243,11 +247,9 @@ static int nft_counter_clone(struct nft_expr *dst, const struct nft_expr *src, g if (cpu_stats == NULL) return -ENOMEM; - preempt_disable(); - this_cpu = this_cpu_ptr(cpu_stats); - this_cpu->packets = total.packets; - this_cpu->bytes = total.bytes; - preempt_enable(); + this_cpu = raw_cpu_ptr(cpu_stats); + u64_stats_set(&this_cpu->packets, total.packets); + u64_stats_set(&this_cpu->bytes, total.bytes); priv_clone->counter = cpu_stats; return 0; @@ -279,7 +281,7 @@ static int __init nft_counter_module_init(void) int cpu, err; for_each_possible_cpu(cpu) - seqcount_init(per_cpu_ptr(&nft_counter_seq, cpu)); + u64_stats_init(per_cpu_ptr(&nft_counter_sync, cpu)); err = nft_register_obj(&nft_counter_obj_type); if (err < 0)