From patchwork Tue Aug 11 22:04:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Fastabend X-Patchwork-Id: 262603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C846DC433E3 for ; Tue, 11 Aug 2020 22:05:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9BF27206C3 for ; Tue, 11 Aug 2020 22:05:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Eq2sz+WB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726472AbgHKWFJ (ORCPT ); Tue, 11 Aug 2020 18:05:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726023AbgHKWFJ (ORCPT ); Tue, 11 Aug 2020 18:05:09 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02DF1C06174A; Tue, 11 Aug 2020 15:05:09 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id m34so7342052pgl.11; Tue, 11 Aug 2020 15:05:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Sy6hc4d69FHZj4Y8QtKCdUHMRPl7YPruS+rAB2afZRo=; b=Eq2sz+WBGSHzPNrVGYA3C+oHprPywi9CQcFXbf0Q2aRAW0FbwEYE8F2DiWpLXwif// SJgBmvV7Z0hicg02GyUI+KazVFi2ZfdPh7Uvix0tlUNTyo5aYVucuIxbrVF5UMZRkT5v R36zADZW9Bywabol1DNiJcJuVWsImoBrxq2r7B+ESVmUwDvXvbDTcdx9Tlgd4jznH9Au gJNaEjxdrfoFidYxQtMTUMPds/D2ucLKvOYOi8rMiOXXASYygCEVfOYFruGfk9wmunRb E1IV2Z0dVNR+wQQsX71EaRdNvuaFtXNRDEo0hzt6D+08T77mINq0Bzx47swPdkgibHXT Ij8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=Sy6hc4d69FHZj4Y8QtKCdUHMRPl7YPruS+rAB2afZRo=; b=X576Cz60CtiZ7Hcsk6QhCwLn5nCFDqXPTqvec+pJIplmA2zb/6ydwEeBl4WvsvXrtt Ynq5dKT/4cwdcKigcFMXZmCUXQZi23aiEdc0xqB78epBLAhXmFz5hj2oJ3F9ceyAa+uC 7hYs5KM/IPkv3lvgTVj9DdGww52t8zUKBkonKObCGWDzOzUvdsYuA9jFk0/XovZGcPec 5x9JI3JPCoGOZRJm6TYPlcGLm2bANiS10H46qmWIAWJ4LoFNt6PVOW2FEhPNFBVZdxIO gcY5pNIF/gxxvHMiGq7ZZczlfHvlT9x9ziBCA2X/1fuOLLKhCMhAZr8WvD+2v0U2rzth 37Jw== X-Gm-Message-State: AOAM530bhKM5rvJtxtjcSzEbd8R8KEuY4f+pIfun3idCloj736nLQr/X DwnHYagvLtV2y1ZLnIGKs1Y= X-Google-Smtp-Source: ABdhPJzp45X2g7VwTUHsZd/OvJ9yHjTGie5kcJmbHRzwH3xDfB83wKLy365wI/KZvHFfZDwbpUE8DA== X-Received: by 2002:a63:6c0a:: with SMTP id h10mr2479996pgc.11.1597183508597; Tue, 11 Aug 2020 15:05:08 -0700 (PDT) Received: from [127.0.1.1] ([184.63.162.180]) by smtp.gmail.com with ESMTPSA id w23sm145711pgj.5.2020.08.11.15.05.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Aug 2020 15:05:07 -0700 (PDT) Subject: [bpf PATCH v3 2/5] bpf: sock_ops sk access may stomp registers when dst_reg = src_reg From: John Fastabend To: songliubraving@fb.com, kafai@fb.com, daniel@iogearbox.net, ast@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, john.fastabend@gmail.com Date: Tue, 11 Aug 2020 15:04:56 -0700 Message-ID: <159718349653.4728.6559437186853473612.stgit@john-Precision-5820-Tower> In-Reply-To: <159718333343.4728.9389284976477402193.stgit@john-Precision-5820-Tower> References: <159718333343.4728.9389284976477402193.stgit@john-Precision-5820-Tower> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Similar to patch ("bpf: sock_ops ctx access may stomp registers") if the src_reg = dst_reg when reading the sk field of a sock_ops struct we generate xlated code, 53: (61) r9 = *(u32 *)(r9 +28) 54: (15) if r9 == 0x0 goto pc+3 56: (79) r9 = *(u64 *)(r9 +0) This stomps on the r9 reg to do the sk_fullsock check and then when reading the skops->sk field instead of the sk pointer we get the sk_fullsock. To fix use similar pattern noted in the previous fix and use the temp field to save/restore a register used to do sk_fullsock check. After the fix the generated xlated code reads, 52: (7b) *(u64 *)(r9 +32) = r8 53: (61) r8 = *(u32 *)(r9 +28) 54: (15) if r9 == 0x0 goto pc+3 55: (79) r8 = *(u64 *)(r9 +32) 56: (79) r9 = *(u64 *)(r9 +0) 57: (05) goto pc+1 58: (79) r8 = *(u64 *)(r9 +32) Here r9 register was in-use so r8 is chosen as the temporary register. In line 52 r8 is saved in temp variable and at line 54 restored in case fullsock != 0. Finally we handle fullsock == 0 case by restoring at line 58. This adds a new macro SOCK_OPS_GET_SK it is almost possible to merge this with SOCK_OPS_GET_FIELD, but I found the extra branch logic a bit more confusing than just adding a new macro despite a bit of duplicating code. Fixes: 1314ef561102e ("bpf: export bpf_sock for BPF_PROG_TYPE_SOCK_OPS prog type") Acked-by: Song Liu Acked-by: Martin KaFai Lau Signed-off-by: John Fastabend --- net/core/filter.c | 49 ++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 38 insertions(+), 11 deletions(-) diff --git a/net/core/filter.c b/net/core/filter.c index 1baeeff..b2df520 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -8358,6 +8358,43 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, } \ } while (0) +#define SOCK_OPS_GET_SK() \ + do { \ + int fullsock_reg = si->dst_reg, reg = BPF_REG_9, jmp = 1; \ + if (si->dst_reg == reg || si->src_reg == reg) \ + reg--; \ + if (si->dst_reg == reg || si->src_reg == reg) \ + reg--; \ + if (si->dst_reg == si->src_reg) { \ + *insn++ = BPF_STX_MEM(BPF_DW, si->src_reg, reg, \ + offsetof(struct bpf_sock_ops_kern, \ + temp)); \ + fullsock_reg = reg; \ + jmp += 2; \ + } \ + *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ + struct bpf_sock_ops_kern, \ + is_fullsock), \ + fullsock_reg, si->src_reg, \ + offsetof(struct bpf_sock_ops_kern, \ + is_fullsock)); \ + *insn++ = BPF_JMP_IMM(BPF_JEQ, fullsock_reg, 0, jmp); \ + if (si->dst_reg == si->src_reg) \ + *insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg, \ + offsetof(struct bpf_sock_ops_kern, \ + temp)); \ + *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ + struct bpf_sock_ops_kern, sk),\ + si->dst_reg, si->src_reg, \ + offsetof(struct bpf_sock_ops_kern, sk));\ + if (si->dst_reg == si->src_reg) { \ + *insn++ = BPF_JMP_A(1); \ + *insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg, \ + offsetof(struct bpf_sock_ops_kern, \ + temp)); \ + } \ + } while (0) + #define SOCK_OPS_GET_TCP_SOCK_FIELD(FIELD) \ SOCK_OPS_GET_FIELD(FIELD, FIELD, struct tcp_sock) @@ -8642,17 +8679,7 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, SOCK_OPS_GET_TCP_SOCK_FIELD(bytes_acked); break; case offsetof(struct bpf_sock_ops, sk): - *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( - struct bpf_sock_ops_kern, - is_fullsock), - si->dst_reg, si->src_reg, - offsetof(struct bpf_sock_ops_kern, - is_fullsock)); - *insn++ = BPF_JMP_IMM(BPF_JEQ, si->dst_reg, 0, 1); - *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( - struct bpf_sock_ops_kern, sk), - si->dst_reg, si->src_reg, - offsetof(struct bpf_sock_ops_kern, sk)); + SOCK_OPS_GET_SK(); break; } return insn - insn_buf; From patchwork Tue Aug 11 22:05:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Fastabend X-Patchwork-Id: 262602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3810C433E0 for ; Tue, 11 Aug 2020 22:05:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D1128206C3 for ; Tue, 11 Aug 2020 22:05:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Glazygkw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726424AbgHKWFx (ORCPT ); Tue, 11 Aug 2020 18:05:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726023AbgHKWFr (ORCPT ); Tue, 11 Aug 2020 18:05:47 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9FB1C06174A; Tue, 11 Aug 2020 15:05:47 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id r4so187174pls.2; Tue, 11 Aug 2020 15:05:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Jv34c1zVCW1Zn5DLAPBWJEA8tPhSD/upIhUsB5wLwsc=; b=Glazygkwli4I0GeR7e0TwA3Io1YsUv0TYNj4H3nsJnhtQMCYZuFW/H+//TJwp31rZQ p9vEvYOy3WpDsvCVdJCk9K7BBlJL+sblJP/AXU8VJiQe1e97i9O2oeGYCccEBnx2Eb9d 0i9qttnPmUx95Ysq9saKVD6MykgBHkuJYKvZmWVpqqbc8ay6LOR3CS++DzXwGv0MU9/W grMhmYluABPR+wQGx3xOv2woo0dbPyd/nqJk1G6q1IwtCDPc9f8gkfQrRbGt2VYoV8jO gLneTMWSLA2Z+gkjJK/XAi+4eo46w2A7E0Sp1rkz2iFf/UsnDnhz8jgo9S2Dtxfwsyuh oQ5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=Jv34c1zVCW1Zn5DLAPBWJEA8tPhSD/upIhUsB5wLwsc=; b=f9nXymBg5abPd8CuQZCfs0ce2qn92kCBlBYLRua/1PvLGScQ0dYywQ3rZhzEUjJOeF Dd/3/l7fWAKaGjsmlCMcO18CwmUJOhfsMGfSYIcrcl+jhiHUMCBiNBTpAE4ILJLBOQyG kxucOVJ5lZlQ9iIRQMA/+IJhjYpucec7Y6Xxs0ySEduDzWs1p3RNiRD/TpFlTX7beyhH v+3HVDCJI5Y3cbv4Xzd6npJfMCD8esk8wdMqwihbX+EGG54iJ67o1+fi7jNPBWMGlLPn 2aYxfmptjj86WvfurE9+jSSoBl8jYfxmaW6KfF4YUJsLdP+iq0HjRBx1bi20wxlWsEVk Ob8A== X-Gm-Message-State: AOAM5326BNbQ/+vfPHJk+mxbsmPoVJKlhaDx8IVtyaT9MN3ybSsotMX0 pIcD/jEnUnPbUOZRgbFlAEU= X-Google-Smtp-Source: ABdhPJwT7MP8ZcJ2f4sHtYqnqRWYKweUzqtXdXsDQNwIv5plaQD3q9WJ3RTWQtdKDIm2PA6kVVSOfA== X-Received: by 2002:a17:90a:d252:: with SMTP id o18mr3075776pjw.146.1597183547305; Tue, 11 Aug 2020 15:05:47 -0700 (PDT) Received: from [127.0.1.1] ([184.63.162.180]) by smtp.gmail.com with ESMTPSA id mr21sm3633652pjb.57.2020.08.11.15.05.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Aug 2020 15:05:46 -0700 (PDT) Subject: [bpf PATCH v3 4/5] bpf, selftests: Add tests for sock_ops load with r9, r8.r7 registers From: John Fastabend To: songliubraving@fb.com, kafai@fb.com, daniel@iogearbox.net, ast@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, john.fastabend@gmail.com Date: Tue, 11 Aug 2020 15:05:33 -0700 Message-ID: <159718353345.4728.8805043614257933227.stgit@john-Precision-5820-Tower> In-Reply-To: <159718333343.4728.9389284976477402193.stgit@john-Precision-5820-Tower> References: <159718333343.4728.9389284976477402193.stgit@john-Precision-5820-Tower> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Loads in sock_ops case when using high registers requires extra logic to ensure the correct temporary value is used. We need to ensure the temp register does not use either the src_reg or dst_reg. Lets add an asm test to force the logic is triggered. The xlated code is here, 30: (7b) *(u64 *)(r9 +32) = r7 31: (61) r7 = *(u32 *)(r9 +28) 32: (15) if r7 == 0x0 goto pc+2 33: (79) r7 = *(u64 *)(r9 +0) 34: (63) *(u32 *)(r7 +916) = r8 35: (79) r7 = *(u64 *)(r9 +32) Notice r9 and r8 are not used for temp registers and r7 is chosen. Acked-by: Song Liu Acked-by: Martin KaFai Lau Signed-off-by: John Fastabend --- .../testing/selftests/bpf/progs/test_tcpbpf_kern.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c index f8b13682..6420b61 100644 --- a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c +++ b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c @@ -75,6 +75,13 @@ int bpf_testcb(struct bpf_sock_ops *skops) : [skops] "r"(skops) :); + asm volatile ( + "r9 = %[skops];\n" + "r8 = *(u32 *)(r9 +164);\n" + "*(u32 *)(r9 +164) = r8;\n" + :: [skops] "r"(skops) + : "r9", "r8"); + op = (int) skops->op; update_event_map(op);