From patchwork Mon Jul 5 15:38:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52F89C07E9C for ; Mon, 5 Jul 2021 15:38:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3149561261 for ; Mon, 5 Jul 2021 15:38:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234087AbhGEPl1 (ORCPT ); Mon, 5 Jul 2021 11:41:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234028AbhGEPlT (ORCPT ); Mon, 5 Jul 2021 11:41:19 -0400 Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9DC1C061768 for ; Mon, 5 Jul 2021 08:38:39 -0700 (PDT) Received: by mail-pl1-x634.google.com with SMTP id h6so9307000plf.11 for ; Mon, 05 Jul 2021 08:38:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wSBI/a1YQn+fTxpONrrL4XFvQob9I0YMvY/ZJ5LMK3E=; b=f5+9SbWD/jRiNMaaB2WfXHDjTQ2flSxuQlYkTNKNBY/kuRJfYeuy+MoIlBKm8vmJvW iuuw8RLEy2vWuy/hPohfE2w8RLnczkOTVd2IOvdaYtJB6DvYAS0MEea8lyekBZl6YZMR +zSXZI1kmWygmTbors7EKK7+LdBzExRc+0PD7b3Vb3n04azoUatzPjvUugEiMW5P2i9p O5DeKEdvWDAx3HwF+yAD7fZaKZtMVEJ2DwVlY0YcQwyK8wR5PsLyaZGEZMzaCspgjArl FBt2iX63mR5jlDh4J1ZjtVFpWmGo+TrpiNoGXVUZ+iL92rAoGTiEijKYQhXaaOCxc3G+ 5JKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wSBI/a1YQn+fTxpONrrL4XFvQob9I0YMvY/ZJ5LMK3E=; b=r46rv4H2dzpk7bV+6MxNnE4C5mryoKLyiCdL2/cH6UlpZutKqz4XRSen6X5Y7CWcsu erZ//R2S0WWETWDurTmAG6Gw8SyZuPpQ5cqd662jr7fUdgI8gEIsx1qqxT1WRXoqTL9D 6ShARiXP3t3o8zSFLyHfkx+NspmtqgGCDcYMFnruKLB92M/b9uUhBKdc36H7gPy6dmKw 4Ni+iliZkM5mcuqi7MzA2OfgszLru6HFIL/R+aQEGHrhlg1YRhK8+c9VfUN58jpR38vY cpPnRxkv3h+OMyu5PMM5APn1AdgoAUt6KZbKlSpmaPcEpTKSoueH37DtKR3uXTACEeTq ktDQ== X-Gm-Message-State: AOAM53116H3i7hQWwJWGSV6sSckJlVY/EF0MkpEJD8BkKCzUXzuz+zbl 1IGcD7Z93jekqcVreU+QORA= X-Google-Smtp-Source: ABdhPJyKzrJtJlTJeDdbnrA6jkSUOr7Gr4Hy9TszNesvCuP6GZlQR0FuKluTwHc+NzJA13U3P7Av+g== X-Received: by 2002:a17:902:fe10:b029:127:6549:fe98 with SMTP id g16-20020a170902fe10b02901276549fe98mr12973708plj.25.1625499519314; Mon, 05 Jul 2021 08:38:39 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.38.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:38:38 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 1/9] bonding: fix suspicious RCU usage in bond_ipsec_add_sa() Date: Mon, 5 Jul 2021 15:38:06 +0000 Message-Id: <20210705153814.11453-2-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To dereference bond->curr_active_slave, it uses rcu_dereference(). But it and the caller doesn't acquire RCU so a warning occurs. So add rcu_read_lock(). Test commands: ip link add dummy0 type dummy ip link add bond0 type bond ip link set dummy0 master bond0 ip link set dummy0 up ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi 0x07 \ mode transport \ reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel \ src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload \ dev bond0 dir in Splat looks like: ============================= WARNING: suspicious RCU usage 5.13.0-rc3+ #1168 Not tainted ----------------------------- drivers/net/bonding/bond_main.c:411 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by ip/684: #0: ffffffff9a2757c0 (&net->xfrm.xfrm_cfg_mutex){+.+.}-{3:3}, at: xfrm_netlink_rcv+0x59/0x80 [xfrm_user] 55.191733][ T684] stack backtrace: CPU: 0 PID: 684 Comm: ip Not tainted 5.13.0-rc3+ #1168 Call Trace: dump_stack+0xa4/0xe5 bond_ipsec_add_sa+0x18c/0x1f0 [bonding] xfrm_dev_state_add+0x2a9/0x770 ? memcpy+0x38/0x60 xfrm_add_sa+0x2278/0x3b10 [xfrm_user] ? xfrm_get_policy+0xaa0/0xaa0 [xfrm_user] ? register_lock_class+0x1750/0x1750 xfrm_user_rcv_msg+0x331/0x660 [xfrm_user] ? rcu_read_lock_sched_held+0x91/0xc0 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? find_held_lock+0x3a/0x1c0 ? mutex_lock_io_nested+0x1210/0x1210 ? sched_clock_cpu+0x18/0x170 netlink_rcv_skb+0x121/0x350 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? netlink_ack+0x9d0/0x9d0 ? netlink_deliver_tap+0x17c/0xa50 xfrm_netlink_rcv+0x68/0x80 [xfrm_user] netlink_unicast+0x41c/0x610 ? netlink_attachskb+0x710/0x710 netlink_sendmsg+0x6b9/0xb70 [ ... ] Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- v1 -> v2: - no change drivers/net/bonding/bond_main.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 03b1a93d7fea..fd7b7f894917 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -403,10 +403,12 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) struct net_device *bond_dev = xs->xso.dev; struct bonding *bond; struct slave *slave; + int err; if (!bond_dev) return -EINVAL; + rcu_read_lock(); bond = netdev_priv(bond_dev); slave = rcu_dereference(bond->curr_active_slave); xs->xso.real_dev = slave->dev; @@ -415,10 +417,13 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) if (!(slave->dev->xfrmdev_ops && slave->dev->xfrmdev_ops->xdo_dev_state_add)) { slave_warn(bond_dev, slave->dev, "Slave does not support ipsec offload\n"); + rcu_read_unlock(); return -EINVAL; } - return slave->dev->xfrmdev_ops->xdo_dev_state_add(xs); + err = slave->dev->xfrmdev_ops->xdo_dev_state_add(xs); + rcu_read_unlock(); + return err; } /** From patchwork Mon Jul 5 15:38:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EB14C07E9B for ; Mon, 5 Jul 2021 15:38:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E2C3A61261 for ; Mon, 5 Jul 2021 15:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234081AbhGEPl3 (ORCPT ); Mon, 5 Jul 2021 11:41:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234030AbhGEPlU (ORCPT ); Mon, 5 Jul 2021 11:41:20 -0400 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A9C5C061760 for ; Mon, 5 Jul 2021 08:38:43 -0700 (PDT) Received: by mail-pf1-x42c.google.com with SMTP id y4so16942918pfi.9 for ; Mon, 05 Jul 2021 08:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IDYrvr7e+y4qJuJVWK50prV8u1HescEo/y+ADLAQaf8=; b=PonyqAIJXsmvJy35H2sDTQRsrDwQCSlBCJi+EOo8Ltnrfb6+X76OdmlF/Eh3YpmVhy 2U6tApjeLwvFsNcYmnfRILK/feoCbyMr+Ud/QVU4oGWRjFlDS4UXMpNq4Nfw8K8asSXn iaXPPXpgUIP3RvrGxRU2KCkV5xBCcfh9PnIlRn2S0Qniffkey8LOPWat94pN+P+QHoxi EWsmFgGkcc3iaFc7W9CD7oSfG4VxWw7AJXnqgQUHuVMUALha47FLSTQLmkqsu9hUdnB5 Amv78Foy6R3Pm0tjgzfDVd3og2MDEJCisC6lp+1PDBn/zzZ9VGY1ovDLw2XFIGjrT2Po A9dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IDYrvr7e+y4qJuJVWK50prV8u1HescEo/y+ADLAQaf8=; b=dO0mzueWjfn7P+dmESeP2gY0wV4SdPdh3kyYb5MLW8viuNOX+4bqUZq4NGDR9ktqGz Mp+fOdxj61P3GwvGUpMImEgrq+w2QaSQ/9InGECaYEi52Q1f/rZUajv5xKt62AuO4F8n mqKCmmkjXCOSJPIGl52/ar79OJ8scvZaOAdYFlwsnsVBurXcDKF78llkLdYKLqvEemSq BaaFPlpSC19oTvYaYtoIqK2lcvr14jGiQvE0qYKVGpR9BApOP22/l+ManBRVqmpeVnNq RhV80dXGZuJJ7zDHUXf4Msxe5gvBaVh1xkrosK3yzN9WcgZQvOOpc+9uVtcEc0kYE/Q1 7NWg== X-Gm-Message-State: AOAM5311KDUtaOJceCCxMrkUzGN0UkOxByGfrO5eiAOR6H8RsUqSjqfm c6mRR1s9bfwfoieE+xbfFZwReJ2uh2ukLg== X-Google-Smtp-Source: ABdhPJw7++r/Zn8/TX309ovlM6NTQJlHLO7FSsTCHNi+lufAwJMwT/1u5ZU6SOMImWcTbhH+13ZGzw== X-Received: by 2002:a63:fc02:: with SMTP id j2mr9700113pgi.235.1625499522715; Mon, 05 Jul 2021 08:38:42 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.38.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:38:42 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 2/9] bonding: fix null dereference in bond_ipsec_add_sa() Date: Mon, 5 Jul 2021 15:38:07 +0000 Message-Id: <20210705153814.11453-3-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org If bond doesn't have real device, bond->curr_active_slave is null. But bond_ipsec_add_sa() dereferences bond->curr_active_slave without null checking. So, null-ptr-deref would occur. Test commands: ip link add bond0 type bond ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi \ 0x07 mode transport reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel src 14.0.0.52/24 \ dst 14.0.0.70/24 proto tcp offload dev bond0 dir in Splat looks like: KASAN: null-ptr-deref in range [0x0000000000000000-0x0000000000000007] CPU: 4 PID: 680 Comm: ip Not tainted 5.13.0-rc3+ #1168 RIP: 0010:bond_ipsec_add_sa+0xc4/0x2e0 [bonding] Code: 85 21 02 00 00 4d 8b a6 48 0c 00 00 e8 75 58 44 ce 85 c0 0f 85 14 01 00 00 48 b8 00 00 00 00 00 fc ff df 4c 89 e2 48 c1 ea 03 <80> 3c 02 00 0f 85 fc 01 00 00 48 8d bb e0 02 00 00 4d 8b 2c 24 48 RSP: 0018:ffff88810946f508 EFLAGS: 00010246 RAX: dffffc0000000000 RBX: ffff88810b4e8040 RCX: 0000000000000001 RDX: 0000000000000000 RSI: ffffffff8fe34280 RDI: ffff888115abe100 RBP: ffff88810946f528 R08: 0000000000000003 R09: fffffbfff2287e11 R10: 0000000000000001 R11: ffff888115abe0c8 R12: 0000000000000000 R13: ffffffffc0aea9a0 R14: ffff88800d7d2000 R15: ffff88810b4e8330 FS: 00007efc5552e680(0000) GS:ffff888119c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055c2530dbf40 CR3: 0000000103056004 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: xfrm_dev_state_add+0x2a9/0x770 ? memcpy+0x38/0x60 xfrm_add_sa+0x2278/0x3b10 [xfrm_user] ? xfrm_get_policy+0xaa0/0xaa0 [xfrm_user] ? register_lock_class+0x1750/0x1750 xfrm_user_rcv_msg+0x331/0x660 [xfrm_user] ? rcu_read_lock_sched_held+0x91/0xc0 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? find_held_lock+0x3a/0x1c0 ? mutex_lock_io_nested+0x1210/0x1210 ? sched_clock_cpu+0x18/0x170 netlink_rcv_skb+0x121/0x350 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? netlink_ack+0x9d0/0x9d0 ? netlink_deliver_tap+0x17c/0xa50 xfrm_netlink_rcv+0x68/0x80 [xfrm_user] netlink_unicast+0x41c/0x610 ? netlink_attachskb+0x710/0x710 netlink_sendmsg+0x6b9/0xb70 [ ...] Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- v1 -> v2: - no change drivers/net/bonding/bond_main.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index fd7b7f894917..e1009e169d42 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -411,6 +411,11 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) rcu_read_lock(); bond = netdev_priv(bond_dev); slave = rcu_dereference(bond->curr_active_slave); + if (!slave) { + rcu_read_unlock(); + return -ENODEV; + } + xs->xso.real_dev = slave->dev; bond->xs = xs; From patchwork Mon Jul 5 15:38:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470746 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CC49C07E9E for ; Mon, 5 Jul 2021 15:38:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 36FC461176 for ; Mon, 5 Jul 2021 15:38:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234110AbhGEPl2 (ORCPT ); Mon, 5 Jul 2021 11:41:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233874AbhGEPlX (ORCPT ); Mon, 5 Jul 2021 11:41:23 -0400 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FA4FC06175F for ; Mon, 5 Jul 2021 08:38:46 -0700 (PDT) Received: by mail-pf1-x436.google.com with SMTP id f20so9064447pfa.1 for ; Mon, 05 Jul 2021 08:38:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lMSSdoWiTyyXbHxY8RuHT5ZkkeUWhPJEXNlMOqEppmY=; b=JjH+MLdJgwevILRyRtDlrWI/UyJQKA69mnaa43RRVamwhiB7ngebuZHkPh3Yqlr1x7 A6PM561okVOKamdcq2mGn21UUz8uWhg3005qns54kQGpUWwRAO2K3CaKfkVBF9/rhEzX NqAPjZ6b9eNHNdhxlnvmzAZUpKcRSc6H8mO2uNSXIhRrpffue6SIHLINweXNr08lXqHt zzn34udUYb/J0f4Dg8SEV0LZeMantEOi0v5sCnL8EpNMWoG1QFenVXjiG17J5sVOysGy apoRP2QMACzezgo3D4JL+TwCz75f/N0Pb8flCdfIYttI4SC2u2xkHB+roBoDUGZajlha rxmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lMSSdoWiTyyXbHxY8RuHT5ZkkeUWhPJEXNlMOqEppmY=; b=bSx3YGKP3DD3nQVNjSThCHx/EW+SIzvvRrIYvpoKvSjUeQwuE9ICCcLliOTqn+kGNQ 6Gtc58GnOo+ALYVHEI3KLGrx3966NEzGi2Gxz+YuwQUq0JeQhSSeZoIfazibAC+1tKQY 0sjkf3S4GeQ6FISgPvxxXRnub/kIg95aILO0iFiyz33nE0aN1WUNIP16AGn3kOq7HUwU yn4iGfohEHkyXCPZzE2DZnUurZ/v3dys4rOR5wXurkznTCzkQiSZLAYrFeUkkj9lJkIZ peps/1IP0Hb79I23jrGVlWezuNFsOWvoBKTww2LpEF7vNcMMmoVTLuMjN13H4eMHjx+T JIeQ== X-Gm-Message-State: AOAM531TkR9WIemRMXxW/bEpfvhzWZcQ8bj1rEKCcoQQufp6gGcrx9nE CX8S/FgBPWgqDqmuGpX4V+Q= X-Google-Smtp-Source: ABdhPJygmUHiM0ZWSgSUTOVaBrR29PMSQNxWjAf/APb0Xr+u82oIRuZhde1kOmrX4w8pRwULVBPjbA== X-Received: by 2002:aa7:86c4:0:b029:316:98ff:30c with SMTP id h4-20020aa786c40000b029031698ff030cmr15617765pfo.66.1625499526135; Mon, 05 Jul 2021 08:38:46 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.38.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:38:45 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 3/9] net: netdevsim: use xso.real_dev instead of xso.dev in callback functions of struct xfrmdev_ops Date: Mon, 5 Jul 2021 15:38:08 +0000 Message-Id: <20210705153814.11453-4-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org There are two pointers in struct xfrm_state_offload, *dev, *real_dev. These are used in callback functions of struct xfrmdev_ops. The *dev points whether bonding interface or real interface. If bonding ipsec offload is used, it points bonding interface If not, it points real interface. And real_dev always points real interface. So, netdevsim should always use real_dev instead of dev. Of course, real_dev always not be null. Test commands: ip netns add A ip netns exec A bash modprobe netdevsim echo "1 1" > /sys/bus/netdevsim/new_device ip link add bond0 type bond mode active-backup ip link set eth0 master bond0 ip link set eth0 up ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi 0x07 mode \ transport reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel src 14.0.0.52/24 \ dst 14.0.0.70/24 proto tcp offload dev bond0 dir in Splat looks like: BUG: spinlock bad magic on CPU#5, kworker/5:1/53 lock: 0xffff8881068c2cc8, .magic: 11121314, .owner: /-1, .owner_cpu: -235736076 CPU: 5 PID: 53 Comm: kworker/5:1 Not tainted 5.13.0-rc3+ #1168 Workqueue: events linkwatch_event Call Trace: dump_stack+0xa4/0xe5 do_raw_spin_lock+0x20b/0x270 ? rwlock_bug.part.1+0x90/0x90 _raw_spin_lock_nested+0x5f/0x70 bond_get_stats+0xe4/0x4c0 [bonding] ? rcu_read_lock_sched_held+0xc0/0xc0 ? bond_neigh_init+0x2c0/0x2c0 [bonding] ? dev_get_alias+0xe2/0x190 ? dev_get_port_parent_id+0x14a/0x360 ? rtnl_unregister+0x190/0x190 ? dev_get_phys_port_name+0xa0/0xa0 ? memset+0x1f/0x40 ? memcpy+0x38/0x60 ? rtnl_phys_switch_id_fill+0x91/0x100 dev_get_stats+0x8c/0x270 rtnl_fill_stats+0x44/0xbe0 ? nla_put+0xbe/0x140 rtnl_fill_ifinfo+0x1054/0x3ad0 [ ... ] Fixes: 272c2330adc9 ("xfrm: bail early on slave pass over skb") Signed-off-by: Taehee Yoo --- v1 -> v2: - no change drivers/net/netdevsim/ipsec.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/net/netdevsim/ipsec.c b/drivers/net/netdevsim/ipsec.c index 3811f1bde84e..b80ed2ffd45e 100644 --- a/drivers/net/netdevsim/ipsec.c +++ b/drivers/net/netdevsim/ipsec.c @@ -85,7 +85,7 @@ static int nsim_ipsec_parse_proto_keys(struct xfrm_state *xs, u32 *mykey, u32 *mysalt) { const char aes_gcm_name[] = "rfc4106(gcm(aes))"; - struct net_device *dev = xs->xso.dev; + struct net_device *dev = xs->xso.real_dev; unsigned char *key_data; char *alg_name = NULL; int key_len; @@ -134,7 +134,7 @@ static int nsim_ipsec_add_sa(struct xfrm_state *xs) u16 sa_idx; int ret; - dev = xs->xso.dev; + dev = xs->xso.real_dev; ns = netdev_priv(dev); ipsec = &ns->ipsec; @@ -194,7 +194,7 @@ static int nsim_ipsec_add_sa(struct xfrm_state *xs) static void nsim_ipsec_del_sa(struct xfrm_state *xs) { - struct netdevsim *ns = netdev_priv(xs->xso.dev); + struct netdevsim *ns = netdev_priv(xs->xso.real_dev); struct nsim_ipsec *ipsec = &ns->ipsec; u16 sa_idx; @@ -211,7 +211,7 @@ static void nsim_ipsec_del_sa(struct xfrm_state *xs) static bool nsim_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) { - struct netdevsim *ns = netdev_priv(xs->xso.dev); + struct netdevsim *ns = netdev_priv(xs->xso.real_dev); struct nsim_ipsec *ipsec = &ns->ipsec; ipsec->ok++; From patchwork Mon Jul 5 15:38:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E20D0C07E99 for ; Mon, 5 Jul 2021 15:38:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C35A261964 for ; Mon, 5 Jul 2021 15:38:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233311AbhGEPld (ORCPT ); Mon, 5 Jul 2021 11:41:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234065AbhGEPl1 (ORCPT ); Mon, 5 Jul 2021 11:41:27 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3F0FC061574 for ; Mon, 5 Jul 2021 08:38:49 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id kt19so11936733pjb.2 for ; Mon, 05 Jul 2021 08:38:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=yxNLjQak/ls2CeE+u1aPtfhRBK1qIZzO+wmLkN9GXjY=; b=kBEP6I9m//osXCubvNq7x3wBAW7PqhdDStGMw7WvHaHki0m6PSncVkz2Z3484n0G+t AAAS8gC6Wz8ajBzLcjt5wLsClxJJo4OeP1v6ds3lea/y6zD2bk9sV6ZkRBwKwr0kX3s3 IslfSi1zfcmsYjQ6h1k9SdiNNfbNp+G2C1slQ8iTw3EeKxrPRJ4sbip3/9MM1n9tDJl9 ktjNzxnLV/mBMuyiBYpdN2Rl+0253jnpl1KhmDHe6PtiFJOfrpoutGtFbM0CLnePDK0h kjx5AnyFY8xsEVuCfcuSi3C4OWNDDK+ncQUmSvXZwOvXYXzRltv+YoQ9zAM3L8DEBL/j nykQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=yxNLjQak/ls2CeE+u1aPtfhRBK1qIZzO+wmLkN9GXjY=; b=qXVAHIGIUEE2W77UGD4K1WyBezIPh6pIDKUPeIAHYEhiCJXybD4+evT8dRizHikDel mJteXVVK8DbbTENXvtp/EvWLQzrDoC4qilEBIEu0MuvQpe6Ral+DnraFotLCVEXR9goU 68BQYTxlNVWe7MpeAt3PhISSYNssd/n9RCAg6ipq+BREok2haOW12mhTgTkGkK4roIAf FvSYQ+pIjLsloYCu8/fZM7EvU3Hk3R+3Kg4crz30gXg1U1aDZ6+y84yxEXqKtDgU2k1o 8WZgK9fOyjWw71JcXFTXX6dYkCd5A8y7lZ2MniTQALtdXufKTCflOcDSIvtuxQInE0We bF/A== X-Gm-Message-State: AOAM531q5QNqFhXOdRGKAZSWE+XyA1zF9R6Iq97werMjPvriIxKhMtda vUgQQJjGEE5BE42TPnE8tJ0= X-Google-Smtp-Source: ABdhPJxFWdmCNt3dcpuTqgtXWVH+DK8JNSVxXjzOOea8rbEJ0N537agocVw5rY/O5uGDnNjXtAmXMw== X-Received: by 2002:a17:90a:5892:: with SMTP id j18mr16259929pji.18.1625499529515; Mon, 05 Jul 2021 08:38:49 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.38.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:38:49 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 4/9] ixgbevf: use xso.real_dev instead of xso.dev in callback functions of struct xfrmdev_ops Date: Mon, 5 Jul 2021 15:38:09 +0000 Message-Id: <20210705153814.11453-5-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org There are two pointers in struct xfrm_state_offload, *dev, *real_dev. These are used in callback functions of struct xfrmdev_ops. The *dev points whether bonding interface or real interface. If bonding ipsec offload is used, it points bonding interface If not, it points real interface. And real_dev always points real interface. So, ixgbevf should always use real_dev instead of dev. Of course, real_dev always not be null. Test commands: ip link add bond0 type bond #eth0 is ixgbevf interface ip link set eth0 master bond0 ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi 0x07 mode \ transport reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel src 14.0.0.52/24 \ dst 14.0.0.70/24 proto tcp offload dev bond0 dir in Splat looks like: KASAN: null-ptr-deref in range [0x0000000000000000-0x0000000000000007] CPU: 6 PID: 688 Comm: ip Not tainted 5.13.0-rc3+ #1168 RIP: 0010:ixgbevf_ipsec_find_empty_idx+0x28/0x1b0 [ixgbevf] Code: 00 00 0f 1f 44 00 00 55 53 48 89 fb 48 83 ec 08 40 84 f6 0f 84 9c 00 00 00 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 <0f> b6 04 02 84 c0 74 08 3c 01 0f 8e 4c 01 00 00 66 81 3b 00 04 0f RSP: 0018:ffff8880089af390 EFLAGS: 00010246 RAX: dffffc0000000000 RBX: 0000000000000000 RCX: 0000000000000001 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000 RBP: ffff8880089af4f8 R08: 0000000000000003 R09: fffffbfff4287e11 R10: 0000000000000001 R11: ffff888005de8908 R12: 0000000000000000 R13: ffff88810936a000 R14: ffff88810936a000 R15: ffff888004d78040 FS: 00007fdf9883a680(0000) GS:ffff88811a400000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055bc14adbf40 CR3: 000000000b87c005 CR4: 00000000003706e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: ixgbevf_ipsec_add_sa+0x1bf/0x9c0 [ixgbevf] ? rcu_read_lock_sched_held+0x91/0xc0 ? ixgbevf_ipsec_parse_proto_keys.isra.9+0x280/0x280 [ixgbevf] ? lock_acquire+0x191/0x720 ? bond_ipsec_add_sa+0x48/0x350 [bonding] ? lockdep_hardirqs_on_prepare+0x3e0/0x3e0 ? rcu_read_lock_held+0x91/0xa0 ? rcu_read_lock_sched_held+0xc0/0xc0 bond_ipsec_add_sa+0x193/0x350 [bonding] xfrm_dev_state_add+0x2a9/0x770 ? memcpy+0x38/0x60 xfrm_add_sa+0x2278/0x3b10 [xfrm_user] ? xfrm_get_policy+0xaa0/0xaa0 [xfrm_user] ? register_lock_class+0x1750/0x1750 xfrm_user_rcv_msg+0x331/0x660 [xfrm_user] ? rcu_read_lock_sched_held+0x91/0xc0 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? find_held_lock+0x3a/0x1c0 ? mutex_lock_io_nested+0x1210/0x1210 ? sched_clock_cpu+0x18/0x170 netlink_rcv_skb+0x121/0x350 [ ... ] Fixes: 272c2330adc9 ("xfrm: bail early on slave pass over skb") Signed-off-by: Taehee Yoo --- - no change drivers/net/ethernet/intel/ixgbevf/ipsec.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbevf/ipsec.c b/drivers/net/ethernet/intel/ixgbevf/ipsec.c index caaea2c920a6..e3e4676af9e4 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ipsec.c +++ b/drivers/net/ethernet/intel/ixgbevf/ipsec.c @@ -211,7 +211,7 @@ struct xfrm_state *ixgbevf_ipsec_find_rx_state(struct ixgbevf_ipsec *ipsec, static int ixgbevf_ipsec_parse_proto_keys(struct xfrm_state *xs, u32 *mykey, u32 *mysalt) { - struct net_device *dev = xs->xso.dev; + struct net_device *dev = xs->xso.real_dev; unsigned char *key_data; char *alg_name = NULL; int key_len; @@ -260,12 +260,15 @@ static int ixgbevf_ipsec_parse_proto_keys(struct xfrm_state *xs, **/ static int ixgbevf_ipsec_add_sa(struct xfrm_state *xs) { - struct net_device *dev = xs->xso.dev; - struct ixgbevf_adapter *adapter = netdev_priv(dev); - struct ixgbevf_ipsec *ipsec = adapter->ipsec; + struct net_device *dev = xs->xso.real_dev; + struct ixgbevf_adapter *adapter; + struct ixgbevf_ipsec *ipsec; u16 sa_idx; int ret; + adapter = netdev_priv(dev); + ipsec = adapter->ipsec; + if (xs->id.proto != IPPROTO_ESP && xs->id.proto != IPPROTO_AH) { netdev_err(dev, "Unsupported protocol 0x%04x for IPsec offload\n", xs->id.proto); @@ -383,11 +386,14 @@ static int ixgbevf_ipsec_add_sa(struct xfrm_state *xs) **/ static void ixgbevf_ipsec_del_sa(struct xfrm_state *xs) { - struct net_device *dev = xs->xso.dev; - struct ixgbevf_adapter *adapter = netdev_priv(dev); - struct ixgbevf_ipsec *ipsec = adapter->ipsec; + struct net_device *dev = xs->xso.real_dev; + struct ixgbevf_adapter *adapter; + struct ixgbevf_ipsec *ipsec; u16 sa_idx; + adapter = netdev_priv(dev); + ipsec = adapter->ipsec; + if (xs->xso.flags & XFRM_OFFLOAD_INBOUND) { sa_idx = xs->xso.offload_handle - IXGBE_IPSEC_BASE_RX_INDEX; From patchwork Mon Jul 5 15:38:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 267C7C07E9B for ; Mon, 5 Jul 2021 15:38:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E98A610A6 for ; Mon, 5 Jul 2021 15:38:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233427AbhGEPle (ORCPT ); Mon, 5 Jul 2021 11:41:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234105AbhGEPlb (ORCPT ); Mon, 5 Jul 2021 11:41:31 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59CEEC061574 for ; Mon, 5 Jul 2021 08:38:53 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id d12so18627276pgd.9 for ; Mon, 05 Jul 2021 08:38:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4W4d/nkYx4Mla7iCAH4eZcYB0hVDv7++rO1p/nGxERs=; b=VBRqrepJ8KVNthnbx1Zn4/KdB5aOeOA2TwDYboexi6pzsfdviosQucxsUahfRb26p2 APXwC7LgwLALACoGkG/uWn1I5hNeM/pz5s44nZs0fZz6e229gahoAEHCapxDRhLgK1g+ nj0ilousARx03UpKAiQs0bVrPTX/3xwhLIZ0sXqn5R3DLr3G5in5Tvpmll5It2L4tds2 S1sDGupBTd+phAHgaZ3J8fMoNmPIzGm/9u/GMZWxFuzobNjFMAIn2OnXB1XTQwLAZDkI qikG3D6xLyuCZTd3cGBwz0eSaGqfQrvoa5eAX5iiujXDF28wzg2yTc9fUGYf9RjfLTKL b0WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4W4d/nkYx4Mla7iCAH4eZcYB0hVDv7++rO1p/nGxERs=; b=K1IIMF8oo53d3s8YpFRsOi2H3ZxtzE5iSSQoXnlXL6NbhjgxRASHJ8GKZ38iVHooLM xlrPuxpu9IaTlBZ0ieGupViUGiHxZeCla+fs5muPFfwhhUDYODD4Hk+r/m0sF9wleDK7 FiTotmgcDRuGVv1CrbjebpvF7spqLgseWgVzPpN4uETHEdcS9zC7oEpXKyhL8rOYnO2B opkQEzk0TD7KgJNETnZRBmSULEVt3KfhniZDRd1EHkC9ew6W+XwMgm06BRXl1WY6rUiI PUr9Uh2S+M/UCE5nLE/7SsRnhFU2OkspOD5We2xyx3X+dpv1kV2pyutm9k36jMKaCorg 4XXQ== X-Gm-Message-State: AOAM531h8xL0GvqqSin8b47/5qA9if/xZBYGX9YW3CoMVZeBMzVOa1Y6 zG3JITZpTLT7eBNLqELagpw= X-Google-Smtp-Source: ABdhPJycaLgSodeOf23RDbXCbdIrPuZ8fzWq1/Th2MBB9l2ZZYdFvz6abQkjyMrWBTQDqTAYAFvIlQ== X-Received: by 2002:aa7:820d:0:b029:2f1:d22d:f21d with SMTP id k13-20020aa7820d0000b02902f1d22df21dmr15742463pfi.7.1625499532930; Mon, 05 Jul 2021 08:38:52 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.38.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:38:52 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 5/9] bonding: fix suspicious RCU usage in bond_ipsec_del_sa() Date: Mon, 5 Jul 2021 15:38:10 +0000 Message-Id: <20210705153814.11453-6-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To dereference bond->curr_active_slave, it uses rcu_dereference(). But it and the caller doesn't acquire RCU so a warning occurs. So add rcu_read_lock(). Test commands: ip netns add A ip netns exec A bash modprobe netdevsim echo "1 1" > /sys/bus/netdevsim/new_device ip link add bond0 type bond ip link set eth0 master bond0 ip link set eth0 up ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi 0x07 mode \ transport reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel src 14.0.0.52/24 \ dst 14.0.0.70/24 proto tcp offload dev bond0 dir in ip x s f Splat looks like: ============================= WARNING: suspicious RCU usage 5.13.0-rc3+ #1168 Not tainted ----------------------------- drivers/net/bonding/bond_main.c:448 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ip/705: #0: ffff888106701780 (&net->xfrm.xfrm_cfg_mutex){+.+.}-{3:3}, at: xfrm_netlink_rcv+0x59/0x80 [xfrm_user] #1: ffff8880075b0098 (&x->lock){+.-.}-{2:2}, at: xfrm_state_delete+0x16/0x30 stack backtrace: CPU: 6 PID: 705 Comm: ip Not tainted 5.13.0-rc3+ #1168 Call Trace: dump_stack+0xa4/0xe5 bond_ipsec_del_sa+0x16a/0x1c0 [bonding] __xfrm_state_delete+0x51f/0x730 xfrm_state_delete+0x1e/0x30 xfrm_state_flush+0x22f/0x390 xfrm_flush_sa+0xd8/0x260 [xfrm_user] ? xfrm_flush_policy+0x290/0x290 [xfrm_user] xfrm_user_rcv_msg+0x331/0x660 [xfrm_user] ? rcu_read_lock_sched_held+0x91/0xc0 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? find_held_lock+0x3a/0x1c0 ? mutex_lock_io_nested+0x1210/0x1210 ? sched_clock_cpu+0x18/0x170 netlink_rcv_skb+0x121/0x350 [ ... ] Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- v1 -> v2: - no change drivers/net/bonding/bond_main.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index e1009e169d42..7659e1fab19e 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -444,21 +444,24 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) if (!bond_dev) return; + rcu_read_lock(); bond = netdev_priv(bond_dev); slave = rcu_dereference(bond->curr_active_slave); if (!slave) - return; + goto out; xs->xso.real_dev = slave->dev; if (!(slave->dev->xfrmdev_ops && slave->dev->xfrmdev_ops->xdo_dev_state_delete)) { slave_warn(bond_dev, slave->dev, "%s: no slave xdo_dev_state_delete\n", __func__); - return; + goto out; } slave->dev->xfrmdev_ops->xdo_dev_state_delete(xs); +out: + rcu_read_unlock(); } /** From patchwork Mon Jul 5 15:38:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E45F0C07E99 for ; Mon, 5 Jul 2021 15:39:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CE52A61261 for ; Mon, 5 Jul 2021 15:39:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232809AbhGEPlg (ORCPT ); Mon, 5 Jul 2021 11:41:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233362AbhGEPld (ORCPT ); Mon, 5 Jul 2021 11:41:33 -0400 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D662BC061574 for ; Mon, 5 Jul 2021 08:38:56 -0700 (PDT) Received: by mail-pg1-x529.google.com with SMTP id t9so18625365pgn.4 for ; Mon, 05 Jul 2021 08:38:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=T4dhU4vNUYUMAfVdjNq0yBk3ipIhsnfg40hCltrxSNQ=; b=g9AjZANKI1igS+w4HK69J8zGtHKAt9N7mTpeFAJqN+JqKAViqng1Wuszbq45xLglOP 86C4nLehp/oIPsUZgMcljPJVgna782qZdhZnyTry4J0O6sKRpGZ9371rPjtcjQEAfs/K GhEJGBS/yIdPT2xlE5eEASu9pyMwNfAksleShLn5/9BwICgRvRJLFP1Tzpig0fzNiNAG H2lhF/K47LLgqVN+DsGvRJLdbphgLXhH6YWc++7/qLRGC4BgcV1wYOKMS3wKsyc/m4gi zI2ablOF37ALQzaFnPpEK/R2PS9O9gCn8e9AgEzZCNnecF+fs3yKN7F1pyFXTr2J2F0J 6GPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=T4dhU4vNUYUMAfVdjNq0yBk3ipIhsnfg40hCltrxSNQ=; b=TFqDgUpxvYqG/Auq9LsMMChTT6yUM/5HcIvZKROYzQ2yhoOZR8ysR5xMKByafMl8gt K2ucxUsrmEMPItcg8GNpVw/WGH1vRU/gwRzL21+FrmNG9JkCtZpnjrKrIzNpWzAaTtUK TyOxCCPG/UiKsamdV5gIqFxYoCtJQyfRj+SeFi/AwyswNmKOr/a19VEL2tPQ5j7anF1X tuz7oXiUNkT5V+Kc70iCifr7jHylVOgAEgFfTcsMG6KrMEOjiOKw//0E6RGoeZQ0i8F3 JNdu/5CvrLrrIqVVTL/jLkNHn7oy7uXYjR1PbEV3RkGc7arhbXagoNqhY/d2gb9S5fqC 8C2Q== X-Gm-Message-State: AOAM530KdafZQ4ybjCsmAszGdZSZOCIGkUHZySWD8jme5J9zmBAq/uWA hmK+CQHhH0sRsMmY1fB1siM= X-Google-Smtp-Source: ABdhPJzv6bT6qCiLVBujxoVeW+4yFJCwanlBmfDWu3/3/Gv7xnM61NnFSiRXhgivVfvYtA7IZ0SiLA== X-Received: by 2002:a62:8309:0:b029:312:d19:ac8b with SMTP id h9-20020a6283090000b02903120d19ac8bmr15771559pfe.63.1625499536445; Mon, 05 Jul 2021 08:38:56 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.38.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:38:55 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 6/9] bonding: disallow setting nested bonding + ipsec offload Date: Mon, 5 Jul 2021 15:38:11 +0000 Message-Id: <20210705153814.11453-7-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bonding interface can be nested and it supports ipsec offload. So, it allows setting the nested bonding + ipsec scenario. But code does not support this scenario. So, it should be disallowed. interface graph: bond2 | bond1 | eth0 The nested bonding + ipsec offload may not a real usecase. So, disallowing this scenario is fine. Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- v1 -> v2: - no change drivers/net/bonding/bond_main.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 7659e1fab19e..f268e67cb2f0 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -419,8 +419,9 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) xs->xso.real_dev = slave->dev; bond->xs = xs; - if (!(slave->dev->xfrmdev_ops - && slave->dev->xfrmdev_ops->xdo_dev_state_add)) { + if (!slave->dev->xfrmdev_ops || + !slave->dev->xfrmdev_ops->xdo_dev_state_add || + netif_is_bond_master(slave->dev)) { slave_warn(bond_dev, slave->dev, "Slave does not support ipsec offload\n"); rcu_read_unlock(); return -EINVAL; @@ -453,8 +454,9 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) xs->xso.real_dev = slave->dev; - if (!(slave->dev->xfrmdev_ops - && slave->dev->xfrmdev_ops->xdo_dev_state_delete)) { + if (!slave->dev->xfrmdev_ops || + !slave->dev->xfrmdev_ops->xdo_dev_state_delete || + netif_is_bond_master(slave->dev)) { slave_warn(bond_dev, slave->dev, "%s: no slave xdo_dev_state_delete\n", __func__); goto out; } @@ -479,8 +481,9 @@ static bool bond_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) return true; - if (!(slave_dev->xfrmdev_ops - && slave_dev->xfrmdev_ops->xdo_dev_offload_ok)) { + if (!slave_dev->xfrmdev_ops || + !slave_dev->xfrmdev_ops->xdo_dev_offload_ok || + netif_is_bond_master(slave_dev)) { slave_warn(bond_dev, slave_dev, "%s: no slave xdo_dev_offload_ok\n", __func__); return false; } From patchwork Mon Jul 5 15:38:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A3D4C07E99 for ; Mon, 5 Jul 2021 15:39:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 36FE761103 for ; Mon, 5 Jul 2021 15:39:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232584AbhGEPlk (ORCPT ); Mon, 5 Jul 2021 11:41:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232782AbhGEPlh (ORCPT ); Mon, 5 Jul 2021 11:41:37 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5220CC061762 for ; Mon, 5 Jul 2021 08:39:00 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id ie21so10186687pjb.0 for ; Mon, 05 Jul 2021 08:39:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jwrsrWFAGe1w2gwfoKFl8stTQyPRTQnsymr8hWUX8J4=; b=SwDZsThxhNBm+q6T3YwnaXxN+NYTuf8m+v0GiZmabPgBJIjurHtJtCle8EDMM1NA+b lF3qIIK/BKMLGiRqTpmzILTCa1AYSkbEk7m/kiql5IJy/LcCyVCjZt59Kc8sTL6f1y9Z Hz00KFQQPz7TCkdOdpCCNXfn2L3uEvk8Le2Dhr25dT06kpz4BFLX63It3VRue7jvdfY3 80FZmvDVoJsP8yiQ1HS1hLPFVl0OCeRKCBeSiM6/BXQSB7+V+EH/6HFe166WHY98o3NR z84AbK2nNoJYwHnSgyeT6MPyG00jrgNgmUCyYQLT4mKjVVAZ2Tqlx0GIhgx6g2vp4HmX 87Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jwrsrWFAGe1w2gwfoKFl8stTQyPRTQnsymr8hWUX8J4=; b=T5tmYLkX9203z/IExzBKQRAqNVBDvfjYwM9v2f4BIlBTul1Q41AEYhbE5b0Hq6LfLX ONCfE1DeU8DUoYxppNGR4jZStYOG8vgoj0HPveIbjBxDpXz3TwONvV68a1XKFvXhZ0gx 7/fmzAF29p9BPjiFoLbZXXCX5jfecduxchU2iyZWlt4sHLG4wCd99hHpDqV0YccaAWCc g1IHwWsN4ZwdT2W83jTf4HmDVwMJ/Yry5HTYzIO1VeSfy9YhkVDpbNDsoTnwroCDOppi LIo6Ra88xmkrD4zCMQvN5J9+4dxut4J1Ix2sycVoRmyTpA80FaLkp9C2PYdI4jYZEGua LuDQ== X-Gm-Message-State: AOAM531p9b5xsSe3Lv0Gq8B0woKzypj8sxeb6LWuTDMR3sV7r3yi79w7 hEIQ97DuCF0jys7JwpMjkks= X-Google-Smtp-Source: ABdhPJzrNGyRgc9Hnx4FSWY0dTvcNwwbNmy5uJF5MkfrAb1Gm1ejCSZNkAUG0vAmi+RJsXxrlnG2cQ== X-Received: by 2002:a17:90b:3506:: with SMTP id ls6mr15380359pjb.25.1625499539866; Mon, 05 Jul 2021 08:38:59 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.38.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:38:59 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 7/9] bonding: Add struct bond_ipesc to manage SA Date: Mon, 5 Jul 2021 15:38:12 +0000 Message-Id: <20210705153814.11453-8-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bonding has been supporting ipsec offload. When SA is added, bonding just passes SA to its own active real interface. But it doesn't manage SA. So, when events(add/del real interface, active real interface change, etc) occur, bonding can't handle that well because It doesn't manage SA. So some problems(panic, UAF, refcnt leak)occur. In order to make it stable, it should manage SA. That's the reason why struct bond_ipsec is added. When a new SA is added to bonding interface, it is stored in the bond_ipsec list. And the SA is passed to a current active real interface. If events occur, it uses bond_ipsec data to handle these events. bond->ipsec_list is protected by bond->ipsec_lock. If a current active real interface is changed, the following logic works. 1. delete all SAs from old active real interface 2. Add all SAs to the new active real interface. 3. If a new active real interface doesn't support ipsec offload or SA's option, it sets real_dev to NULL. Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- - Do not print unncessary warning in bond_ipsec_add_sa_all() - Add comment for ipsec_lock drivers/net/bonding/bond_main.c | 139 +++++++++++++++++++++++++++----- include/net/bonding.h | 9 ++- 2 files changed, 127 insertions(+), 21 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index f268e67cb2f0..9c44ec92eb72 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -401,6 +401,7 @@ static int bond_vlan_rx_kill_vid(struct net_device *bond_dev, static int bond_ipsec_add_sa(struct xfrm_state *xs) { struct net_device *bond_dev = xs->xso.dev; + struct bond_ipsec *ipsec; struct bonding *bond; struct slave *slave; int err; @@ -416,9 +417,6 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) return -ENODEV; } - xs->xso.real_dev = slave->dev; - bond->xs = xs; - if (!slave->dev->xfrmdev_ops || !slave->dev->xfrmdev_ops->xdo_dev_state_add || netif_is_bond_master(slave->dev)) { @@ -427,11 +425,63 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) return -EINVAL; } + ipsec = kmalloc(sizeof(*ipsec), GFP_ATOMIC); + if (!ipsec) { + rcu_read_unlock(); + return -ENOMEM; + } + xs->xso.real_dev = slave->dev; + err = slave->dev->xfrmdev_ops->xdo_dev_state_add(xs); + if (!err) { + ipsec->xs = xs; + INIT_LIST_HEAD(&ipsec->list); + spin_lock_bh(&bond->ipsec_lock); + list_add(&ipsec->list, &bond->ipsec_list); + spin_unlock_bh(&bond->ipsec_lock); + } else { + kfree(ipsec); + } rcu_read_unlock(); return err; } +static void bond_ipsec_add_sa_all(struct bonding *bond) +{ + struct net_device *bond_dev = bond->dev; + struct bond_ipsec *ipsec; + struct slave *slave; + + rcu_read_lock(); + slave = rcu_dereference(bond->curr_active_slave); + if (!slave) + goto out; + + if (!slave->dev->xfrmdev_ops || + !slave->dev->xfrmdev_ops->xdo_dev_state_add || + netif_is_bond_master(slave->dev)) { + spin_lock_bh(&bond->ipsec_lock); + if (!list_empty(&bond->ipsec_list)) + slave_warn(bond_dev, slave->dev, + "%s: no slave xdo_dev_state_add\n", + __func__); + spin_unlock_bh(&bond->ipsec_lock); + goto out; + } + + spin_lock_bh(&bond->ipsec_lock); + list_for_each_entry(ipsec, &bond->ipsec_list, list) { + ipsec->xs->xso.real_dev = slave->dev; + if (slave->dev->xfrmdev_ops->xdo_dev_state_add(ipsec->xs)) { + slave_warn(bond_dev, slave->dev, "%s: failed to add SA\n", __func__); + ipsec->xs->xso.real_dev = NULL; + } + } + spin_unlock_bh(&bond->ipsec_lock); +out: + rcu_read_unlock(); +} + /** * bond_ipsec_del_sa - clear out this specific SA * @xs: pointer to transformer state struct @@ -439,6 +489,7 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) static void bond_ipsec_del_sa(struct xfrm_state *xs) { struct net_device *bond_dev = xs->xso.dev; + struct bond_ipsec *ipsec; struct bonding *bond; struct slave *slave; @@ -452,7 +503,10 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) if (!slave) goto out; - xs->xso.real_dev = slave->dev; + if (!xs->xso.real_dev) + goto out; + + WARN_ON(xs->xso.real_dev != slave->dev); if (!slave->dev->xfrmdev_ops || !slave->dev->xfrmdev_ops->xdo_dev_state_delete || @@ -463,6 +517,48 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) slave->dev->xfrmdev_ops->xdo_dev_state_delete(xs); out: + spin_lock_bh(&bond->ipsec_lock); + list_for_each_entry(ipsec, &bond->ipsec_list, list) { + if (ipsec->xs == xs) { + list_del(&ipsec->list); + kfree(ipsec); + break; + } + } + spin_unlock_bh(&bond->ipsec_lock); + rcu_read_unlock(); +} + +static void bond_ipsec_del_sa_all(struct bonding *bond) +{ + struct net_device *bond_dev = bond->dev; + struct bond_ipsec *ipsec; + struct slave *slave; + + rcu_read_lock(); + slave = rcu_dereference(bond->curr_active_slave); + if (!slave) { + rcu_read_unlock(); + return; + } + + spin_lock_bh(&bond->ipsec_lock); + list_for_each_entry(ipsec, &bond->ipsec_list, list) { + if (!ipsec->xs->xso.real_dev) + continue; + + if (!slave->dev->xfrmdev_ops || + !slave->dev->xfrmdev_ops->xdo_dev_state_delete || + netif_is_bond_master(slave->dev)) { + slave_warn(bond_dev, slave->dev, + "%s: no slave xdo_dev_state_delete\n", + __func__); + } else { + slave->dev->xfrmdev_ops->xdo_dev_state_delete(ipsec->xs); + } + ipsec->xs->xso.real_dev = NULL; + } + spin_unlock_bh(&bond->ipsec_lock); rcu_read_unlock(); } @@ -474,22 +570,27 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) static bool bond_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) { struct net_device *bond_dev = xs->xso.dev; - struct bonding *bond = netdev_priv(bond_dev); - struct slave *curr_active = rcu_dereference(bond->curr_active_slave); - struct net_device *slave_dev = curr_active->dev; + struct net_device *real_dev; + struct slave *curr_active; + struct bonding *bond; + + bond = netdev_priv(bond_dev); + curr_active = rcu_dereference(bond->curr_active_slave); + real_dev = curr_active->dev; if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) return true; - if (!slave_dev->xfrmdev_ops || - !slave_dev->xfrmdev_ops->xdo_dev_offload_ok || - netif_is_bond_master(slave_dev)) { - slave_warn(bond_dev, slave_dev, "%s: no slave xdo_dev_offload_ok\n", __func__); + if (!xs->xso.real_dev) + return false; + + if (!real_dev->xfrmdev_ops || + !real_dev->xfrmdev_ops->xdo_dev_offload_ok || + netif_is_bond_master(real_dev)) { return false; } - xs->xso.real_dev = slave_dev; - return slave_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); + return real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); } static const struct xfrmdev_ops bond_xfrmdev_ops = { @@ -1006,8 +1107,7 @@ void bond_change_active_slave(struct bonding *bond, struct slave *new_active) return; #ifdef CONFIG_XFRM_OFFLOAD - if (old_active && bond->xs) - bond_ipsec_del_sa(bond->xs); + bond_ipsec_del_sa_all(bond); #endif /* CONFIG_XFRM_OFFLOAD */ if (new_active) { @@ -1083,10 +1183,7 @@ void bond_change_active_slave(struct bonding *bond, struct slave *new_active) } #ifdef CONFIG_XFRM_OFFLOAD - if (new_active && bond->xs) { - xfrm_dev_state_flush(dev_net(bond->dev), bond->dev, true); - bond_ipsec_add_sa(bond->xs); - } + bond_ipsec_add_sa_all(bond); #endif /* CONFIG_XFRM_OFFLOAD */ /* resend IGMP joins since active slave has changed or @@ -3343,6 +3440,7 @@ static int bond_master_netdev_event(unsigned long event, return bond_event_changename(event_bond); case NETDEV_UNREGISTER: bond_remove_proc_entry(event_bond); + xfrm_dev_state_flush(dev_net(bond_dev), bond_dev, true); break; case NETDEV_REGISTER: bond_create_proc_entry(event_bond); @@ -4906,7 +5004,8 @@ void bond_setup(struct net_device *bond_dev) #ifdef CONFIG_XFRM_OFFLOAD /* set up xfrm device ops (only supported in active-backup right now) */ bond_dev->xfrmdev_ops = &bond_xfrmdev_ops; - bond->xs = NULL; + INIT_LIST_HEAD(&bond->ipsec_list); + spin_lock_init(&bond->ipsec_lock); #endif /* CONFIG_XFRM_OFFLOAD */ /* don't acquire bond device's netif_tx_lock when transmitting */ diff --git a/include/net/bonding.h b/include/net/bonding.h index 019e998d944a..a02b19843819 100644 --- a/include/net/bonding.h +++ b/include/net/bonding.h @@ -201,6 +201,11 @@ struct bond_up_slave { */ #define BOND_LINK_NOCHANGE -1 +struct bond_ipsec { + struct list_head list; + struct xfrm_state *xs; +}; + /* * Here are the locking policies for the two bonding locks: * Get rcu_read_lock when reading or RTNL when writing slave list. @@ -249,7 +254,9 @@ struct bonding { #endif /* CONFIG_DEBUG_FS */ struct rtnl_link_stats64 bond_stats; #ifdef CONFIG_XFRM_OFFLOAD - struct xfrm_state *xs; + struct list_head ipsec_list; + /* protecting ipsec_list */ + spinlock_t ipsec_lock; #endif /* CONFIG_XFRM_OFFLOAD */ }; From patchwork Mon Jul 5 15:38:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BF15C07E9B for ; Mon, 5 Jul 2021 15:39:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0689E610A6 for ; Mon, 5 Jul 2021 15:39:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233342AbhGEPlm (ORCPT ); Mon, 5 Jul 2021 11:41:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232438AbhGEPlk (ORCPT ); Mon, 5 Jul 2021 11:41:40 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A12ADC061762 for ; Mon, 5 Jul 2021 08:39:03 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id s18so3544449pgg.8 for ; Mon, 05 Jul 2021 08:39:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z/GRneOwL1VpqYdTs9VsVkUYVJD8+xrWHwe0lfZhQeI=; b=UZAKV3CcpV0ZiK3Ivzk3HTYCveB+bhdQH0Y97ocRLi70jqdW2vPPtZ7+h+LImFqFAt qyOf4KZ+GwlKtXsAM6kavXJ++LRiDxkrUv0sf8gJC4+Yf/Z9vlshnzZZEo9yISQMvzO1 cqmtoXPmTRBiJpHzIoFkAB/3qWVy25OLEyQyhBulRhgfr/UQ/Lq0IyBXgbJdFv9zRmEV TYg1JNuAEBoAvPUNUnmuCuExCwIjaZXGKc7HOG0IsoOlHvHIkHAeZuynG/e1eOqiphSE rVkMRl1CyY3iPQ7gJCCCjWbeG2UEffu8zyThE7iNXnZnTECMzgsnqUIQFjc53y6Uyn6v ETcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z/GRneOwL1VpqYdTs9VsVkUYVJD8+xrWHwe0lfZhQeI=; b=BQAXsd65E8pkinwXhI4ZX3YvwwKc+fORxbdxZ6o7gV7hotmWwYujD0AF4LLcJrwc31 bp3oHmU8O7KMHKFRWvKLstHMBWLuT7k9d+gY4yZuYPaJke7hBqlOXCGg2a8i4WWHgZSZ Qdma26GV9XwgkjPuBjvcDu2RPpyuNPnBSnCJDOlYosFoZS74StjPGqBPlrEGZy2xqOYA DZ4PZ/ddTbhCPikNemgfnw2RUBDXkFZEHo22zD9MHtvg4kVBmbVO9uA+R6CmXN/CQCjd WfBYC+GPyfWfVvV1XAatRKp1yYSZ+ixp/rbP9Dazg7gEeBO9JNVQhchLdowt11/5NHm+ fOsQ== X-Gm-Message-State: AOAM530XVpGjwD1rNLForKrKsbBgqANl7Kd6Fq0P0ctCb0rIOBfgaDQ8 3DUQF4kDPO+wGzLR9oj4AfU= X-Google-Smtp-Source: ABdhPJxEsiHnOonwQLFFSka2dLVGQmWy5L0AYtJDHHFa9uYC0NWMf3vEvfgt001NDNmMY++w+qChCQ== X-Received: by 2002:a63:5156:: with SMTP id r22mr16651716pgl.264.1625499543237; Mon, 05 Jul 2021 08:39:03 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.39.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:39:02 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 8/9] bonding: fix suspicious RCU usage in bond_ipsec_offload_ok() Date: Mon, 5 Jul 2021 15:38:13 +0000 Message-Id: <20210705153814.11453-9-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To dereference bond->curr_active_slave, it uses rcu_dereference(). But it and the caller doesn't acquire RCU so a warning occurs. So add rcu_read_lock(). Splat looks like: WARNING: suspicious RCU usage 5.13.0-rc6+ #1179 Not tainted drivers/net/bonding/bond_main.c:571 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by ping/974: #0: ffff888109e7db70 (sk_lock-AF_INET){+.+.}-{0:0}, at: raw_sendmsg+0x1303/0x2cb0 stack backtrace: CPU: 2 PID: 974 Comm: ping Not tainted 5.13.0-rc6+ #1179 Call Trace: dump_stack+0xa4/0xe5 bond_ipsec_offload_ok+0x1f4/0x260 [bonding] xfrm_output+0x179/0x890 xfrm4_output+0xfa/0x410 ? __xfrm4_output+0x4b0/0x4b0 ? __ip_make_skb+0xecc/0x2030 ? xfrm4_udp_encap_rcv+0x800/0x800 ? ip_local_out+0x21/0x3a0 ip_send_skb+0x37/0xa0 raw_sendmsg+0x1bfd/0x2cb0 Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- - no change drivers/net/bonding/bond_main.c | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 9c44ec92eb72..a9cb06959320 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -573,24 +573,34 @@ static bool bond_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) struct net_device *real_dev; struct slave *curr_active; struct bonding *bond; + int err; bond = netdev_priv(bond_dev); + rcu_read_lock(); curr_active = rcu_dereference(bond->curr_active_slave); real_dev = curr_active->dev; - if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) - return true; + if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) { + err = true; + goto out; + } - if (!xs->xso.real_dev) - return false; + if (!xs->xso.real_dev) { + err = false; + goto out; + } if (!real_dev->xfrmdev_ops || !real_dev->xfrmdev_ops->xdo_dev_offload_ok || netif_is_bond_master(real_dev)) { - return false; + err = false; + goto out; } - return real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); + err = real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); +out: + rcu_read_unlock(); + return err; } static const struct xfrmdev_ops bond_xfrmdev_ops = { From patchwork Mon Jul 5 15:38:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 470396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D19AEC07E99 for ; Mon, 5 Jul 2021 15:39:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8FD461176 for ; Mon, 5 Jul 2021 15:39:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232440AbhGEPlu (ORCPT ); Mon, 5 Jul 2021 11:41:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233571AbhGEPlr (ORCPT ); Mon, 5 Jul 2021 11:41:47 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17123C061767 for ; Mon, 5 Jul 2021 08:39:07 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id h4so18620885pgp.5 for ; Mon, 05 Jul 2021 08:39:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4/o8vlqLZEYYy4rqyhVYly8UdixL87ijlGfpCwKp2EM=; b=Hi1CYnKaJD8B+s1p70zwuRhqyGMqkxTj02hYcpGZ8AUaVA3OeUxSzpu3Q3ODUsEKyT bTtilIy+9e1MeC/xB3CwvSy+isDwK1LN+FSlukpUUawJ9/h8vnD4PX22iIh4815c2wE8 dzvLA8g6SnwQ16Yw4XvjgciUKw5Ds+Efa09UAepBZzxFSyUplFat9e0j4zMdCP6Q1ojH oZVl/PA5UJSWDsfGTQfABms9lTSwNQwm33skw8nBsCMrPY8sCEIoYhReUsC0c+zv3F02 XI3raKQBEmr0f+H60tOpkdYLRaWWeDsCQqUmXU7FJqSN2MeF+wwpQ98GOdiM1vUIlaJ5 weoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4/o8vlqLZEYYy4rqyhVYly8UdixL87ijlGfpCwKp2EM=; b=m1CdOX4b1E7RiT3Wo5IieO3lOMqym3VToxscLy77RGAPtM7Otq+KLm8pBoIGw5GKuq HVhYocAxgFOB7jWfhqjFv42l8avPmh5/Ug6higEpk6MpL5C7tgSTEZAWIzifeIAPN5BF cNo0faOogw43gOzrvi1NuKB8FZWiKT2j/VrPXU42NW79VwHiqXRlf+H5z7V4Q8tbpkdn 2M6VuRMB4SCJ0xjiVXGuLDoP0w0ia6WjuOLpu2eZ9oZgD+dh4zal7Wkjez+7EJXsP/H6 mR2d895xSwM5HM8Mqjcp+glleWS7p5YcthsxnIEWNllLndTMNT5m+v1NLWD+uItg3AB7 FL8g== X-Gm-Message-State: AOAM531yAbukrsY+GjFHOYxi4IEDtuMEs3hpnLSKc8jwNypm6ox+kZDG Ph0RDWcizfvjnWMZFSqi2Ak= X-Google-Smtp-Source: ABdhPJxljA3EFgn011+V+dR3d8tS6FvrSWEV3mImCpX2tf4EaXQUMOuBx6QHATtpflJ0KsivCX/OTg== X-Received: by 2002:a63:4c19:: with SMTP id z25mr16231805pga.160.1625499546611; Mon, 05 Jul 2021 08:39:06 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id k10sm9310353pfp.63.2021.07.05.08.39.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 08:39:06 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, dsahern@kernel.org, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net v2 9/9] bonding: fix incorrect return value of bond_ipsec_offload_ok() Date: Mon, 5 Jul 2021 15:38:14 +0000 Message-Id: <20210705153814.11453-10-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705153814.11453-1-ap420073@gmail.com> References: <20210705153814.11453-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bond_ipsec_offload_ok() is called to check whether the interface supports ipsec offload or not. bonding interface support ipsec offload only in active-backup mode. So, if a bond interface is not in active-backup mode, it should return false but it returns true. Fixes: a3b658cfb664 ("bonding: allow xfrm offload setup post-module-load") Signed-off-by: Taehee Yoo --- v1 -> v2: - newly added drivers/net/bonding/bond_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index a9cb06959320..3bc6266ede0e 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -581,7 +581,7 @@ static bool bond_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) real_dev = curr_active->dev; if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) { - err = true; + err = false; goto out; }