From patchwork Fri Jul 2 14:26:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 469801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAD11C11F68 for ; Fri, 2 Jul 2021 14:27:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 95DE9613CB for ; Fri, 2 Jul 2021 14:27:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233076AbhGBO3z (ORCPT ); Fri, 2 Jul 2021 10:29:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233069AbhGBO3y (ORCPT ); Fri, 2 Jul 2021 10:29:54 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DEA12C061762 for ; Fri, 2 Jul 2021 07:27:21 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id b1so5673525pls.5 for ; Fri, 02 Jul 2021 07:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aDViKPmzC5DeXMqC6l6nHzYC2hj1+d4hlR2ybtliY5s=; b=QORkh6h0BQBCokm1UHRNF6w12HV57wAz0Hl+jHMIsEFOw4JeTQKLYy41lruI+t5XY8 HwcDmAmWBtxyNMxDp0jF6q0eZuRhaILy5qxo61bC+frlMUS4U1AmNkcpDXEmxXiltPFO 6S3vv4x0pxCLJ/WjhPiyLMw+rv2KKkHYiJvgB4wJaHLcnIYPyzi5ofV+T3Xsh+FwXFfu YDr8H1W1wsBBT3+4N0k461xVe36QXJCAbLEyIsRhFm4C6giAEwfP1aP1yCCrHlVzZbyh v8lu0ivpiLBsWrzQpy1PG45tJMrgRvruYKBDAXZLCL3t1AatZIY93fJLiymZbIIzi2LK lnog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aDViKPmzC5DeXMqC6l6nHzYC2hj1+d4hlR2ybtliY5s=; b=W3rr0AJ5laEqxodtUeDgYCzDHwor8NFZWLnBtj4J8OOJJgldWVv65l76NsV6ZllAxI 2ujboyifh1ZAu+/72FlKgJPA0U6eyDp1fhP/Nh3ejyLuePa6n3m9szonyCrzgDEjzfp4 8+q6uog6Ga6YzkPBZGzBryJDEJgMOTeHHZxwVfgNkWIoyuRwXkw+VJuIKOusphFAaIfh bh7fPe7h8Rmb7Y11czbhRhUCrTt4zAkjW/MZajkxcEaRpFHNYsdFV16q25r+/Ka8JDoE UfXEV/bmtVyny8LnjA4f1AzF9oi6YicDDy83w+N9iNPpMdc5EHJVx2QXkgt7kgxgxcKE /P9w== X-Gm-Message-State: AOAM5333UI08f+PN7cEc7aHKDOr+wInYZcRUyEcNlHuCgnZ15Gp0a5e0 iAMiOVWnku7Xv+HHLPKaFO4fvPAWq03M+g== X-Google-Smtp-Source: ABdhPJyAG+rZexoxXvp0A0QlNt0sS20WBJ2rd9TNU/iLdbnWdl82fPk9DBnu6UQomKVSRqK4CHS0Xw== X-Received: by 2002:a17:90a:8992:: with SMTP id v18mr61000pjn.109.1625236041175; Fri, 02 Jul 2021 07:27:21 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id nr12sm12683747pjb.1.2021.07.02.07.27.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jul 2021 07:27:20 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, netdev@vger.kernel.org, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net 1/8] bonding: fix suspicious RCU usage in bond_ipsec_add_sa() Date: Fri, 2 Jul 2021 14:26:41 +0000 Message-Id: <20210702142648.7677-2-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210702142648.7677-1-ap420073@gmail.com> References: <20210702142648.7677-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To dereference bond->curr_active_slave, it uses rcu_dereference(). But it and the caller doesn't acquire RCU so a warning occurs. So add rcu_read_lock(). Test commands: ip link add dummy0 type dummy ip link add bond0 type bond ip link set dummy0 master bond0 ip link set dummy0 up ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi 0x07 \ mode transport \ reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel \ src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload \ dev bond0 dir in Splat looks like: ============================= WARNING: suspicious RCU usage 5.13.0-rc3+ #1168 Not tainted ----------------------------- drivers/net/bonding/bond_main.c:411 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 1 lock held by ip/684: #0: ffffffff9a2757c0 (&net->xfrm.xfrm_cfg_mutex){+.+.}-{3:3}, at: xfrm_netlink_rcv+0x59/0x80 [xfrm_user] 55.191733][ T684] stack backtrace: CPU: 0 PID: 684 Comm: ip Not tainted 5.13.0-rc3+ #1168 Call Trace: dump_stack+0xa4/0xe5 bond_ipsec_add_sa+0x18c/0x1f0 [bonding] xfrm_dev_state_add+0x2a9/0x770 ? memcpy+0x38/0x60 xfrm_add_sa+0x2278/0x3b10 [xfrm_user] ? xfrm_get_policy+0xaa0/0xaa0 [xfrm_user] ? register_lock_class+0x1750/0x1750 xfrm_user_rcv_msg+0x331/0x660 [xfrm_user] ? rcu_read_lock_sched_held+0x91/0xc0 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? find_held_lock+0x3a/0x1c0 ? mutex_lock_io_nested+0x1210/0x1210 ? sched_clock_cpu+0x18/0x170 netlink_rcv_skb+0x121/0x350 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? netlink_ack+0x9d0/0x9d0 ? netlink_deliver_tap+0x17c/0xa50 xfrm_netlink_rcv+0x68/0x80 [xfrm_user] netlink_unicast+0x41c/0x610 ? netlink_attachskb+0x710/0x710 netlink_sendmsg+0x6b9/0xb70 [ ... ] Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- drivers/net/bonding/bond_main.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 03b1a93d7fea..fd7b7f894917 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -403,10 +403,12 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) struct net_device *bond_dev = xs->xso.dev; struct bonding *bond; struct slave *slave; + int err; if (!bond_dev) return -EINVAL; + rcu_read_lock(); bond = netdev_priv(bond_dev); slave = rcu_dereference(bond->curr_active_slave); xs->xso.real_dev = slave->dev; @@ -415,10 +417,13 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) if (!(slave->dev->xfrmdev_ops && slave->dev->xfrmdev_ops->xdo_dev_state_add)) { slave_warn(bond_dev, slave->dev, "Slave does not support ipsec offload\n"); + rcu_read_unlock(); return -EINVAL; } - return slave->dev->xfrmdev_ops->xdo_dev_state_add(xs); + err = slave->dev->xfrmdev_ops->xdo_dev_state_add(xs); + rcu_read_unlock(); + return err; } /** From patchwork Fri Jul 2 14:26:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 469800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9EBCC11F68 for ; Fri, 2 Jul 2021 14:27:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A0CE26142D for ; Fri, 2 Jul 2021 14:27:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233101AbhGBOaB (ORCPT ); Fri, 2 Jul 2021 10:30:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233069AbhGBOaA (ORCPT ); Fri, 2 Jul 2021 10:30:00 -0400 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D8D0C061762 for ; Fri, 2 Jul 2021 07:27:28 -0700 (PDT) Received: by mail-pf1-x42a.google.com with SMTP id g21so9049303pfc.11 for ; Fri, 02 Jul 2021 07:27:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Ae52QNnE0pHQHMrza2GfE9HuoTMbli8q1cxcOb1VCWs=; b=tQLJjd4T5G8PVo+StjSIf6mpCwfBtjO1/LESbFs6Cb6EuXm1jd3y4ezbteUI+sW2Hk BSKvd8I4FsuVMSXXhpes8W88oPV2ePaoRiWQovn2+ZzDGcASvFoGRE1hfR+rOyrNabmD 6hsWO7CIZG2nTmImpZcbuKAwREBU9/0Bl7q8xHbYN5dIl8qL9mzfWNQa7hzX72L4N570 OYVzHDypvdQTMTvnofDTXA4+MSjAItEx2CSFz/Ubf76kYZUGtzsKXgVy9+ZqaGFC/rIv 3/+q9hNoCKw5RZWlsWSoHIzA7xnJPcEBjnAVoslwF2zU9mc4PKorcwgxdFWcmlZtWP3m SstQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Ae52QNnE0pHQHMrza2GfE9HuoTMbli8q1cxcOb1VCWs=; b=PZr83q97tVZ9+VZKhCFgzS7EPuEJulo/ETlsApTgOiqNsJJB1tldHkaKDXGzbCqTrr dt0sLWpCHrOpWLmx8ye+yqMq4iL4gQT7dprEgapStXy6AzRwidDb/sEKKkCu/VBpPSE+ 9H2eO8ZAmUSnGz8n6TnJIt5old2IioZLGBqCBT703K1bNHEO3v3p4z5kcYym+oNIriZC /cqbyyzTSXSmyNQSjFpDnbS2JaqYFTeL2XalzZsIUnIP9zD2s5rredPaq8EAcRiBOU3N kSjf6uCJUFb7dmWLYeRPmyUJcZqu09IJLHNNDuwcF4dQEITozcWiTGULrVpBdFyZ8Csu ZpjQ== X-Gm-Message-State: AOAM530Zbjb+iNK3CcS5chLI4xNtYQvQ6q9lN6kr8+Q6IkxaZFLvSD/5 LWIcGi6wSGL1QyqeihGsAII= X-Google-Smtp-Source: ABdhPJyLsYnDdIe8Gy9RFoACQZY0DvExLuIY2ntl0HyABtJG1qsRHnI3CiqcqmW6o91cmOrVnnmPTA== X-Received: by 2002:a62:e50c:0:b029:2f9:b9b1:d44f with SMTP id n12-20020a62e50c0000b02902f9b9b1d44fmr5711054pff.42.1625236047986; Fri, 02 Jul 2021 07:27:27 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id nr12sm12683747pjb.1.2021.07.02.07.27.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jul 2021 07:27:27 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, netdev@vger.kernel.org, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net 3/8] net: netdevsim: use xso.real_dev instead of xso.dev in callback functions of struct xfrmdev_ops Date: Fri, 2 Jul 2021 14:26:43 +0000 Message-Id: <20210702142648.7677-4-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210702142648.7677-1-ap420073@gmail.com> References: <20210702142648.7677-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org There are two pointers in struct xfrm_state_offload, *dev, *real_dev. These are used in callback functions of struct xfrmdev_ops. The *dev points whether bonding interface or real interface. If bonding ipsec offload is used, it points bonding interface If not, it points real interface. And real_dev always points real interface. So, netdevsim should always use real_dev instead of dev. Of course, real_dev always not be null. Test commands: ip netns add A ip netns exec A bash modprobe netdevsim echo "1 1" > /sys/bus/netdevsim/new_device ip link add bond0 type bond mode active-backup ip link set eth0 master bond0 ip link set eth0 up ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi 0x07 mode \ transport reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel src 14.0.0.52/24 \ dst 14.0.0.70/24 proto tcp offload dev bond0 dir in Splat looks like: BUG: spinlock bad magic on CPU#5, kworker/5:1/53 lock: 0xffff8881068c2cc8, .magic: 11121314, .owner: /-1, .owner_cpu: -235736076 CPU: 5 PID: 53 Comm: kworker/5:1 Not tainted 5.13.0-rc3+ #1168 Workqueue: events linkwatch_event Call Trace: dump_stack+0xa4/0xe5 do_raw_spin_lock+0x20b/0x270 ? rwlock_bug.part.1+0x90/0x90 _raw_spin_lock_nested+0x5f/0x70 bond_get_stats+0xe4/0x4c0 [bonding] ? rcu_read_lock_sched_held+0xc0/0xc0 ? bond_neigh_init+0x2c0/0x2c0 [bonding] ? dev_get_alias+0xe2/0x190 ? dev_get_port_parent_id+0x14a/0x360 ? rtnl_unregister+0x190/0x190 ? dev_get_phys_port_name+0xa0/0xa0 ? memset+0x1f/0x40 ? memcpy+0x38/0x60 ? rtnl_phys_switch_id_fill+0x91/0x100 dev_get_stats+0x8c/0x270 rtnl_fill_stats+0x44/0xbe0 ? nla_put+0xbe/0x140 rtnl_fill_ifinfo+0x1054/0x3ad0 [ ... ] Fixes: 272c2330adc9 ("xfrm: bail early on slave pass over skb") Signed-off-by: Taehee Yoo --- drivers/net/netdevsim/ipsec.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/net/netdevsim/ipsec.c b/drivers/net/netdevsim/ipsec.c index 3811f1bde84e..b80ed2ffd45e 100644 --- a/drivers/net/netdevsim/ipsec.c +++ b/drivers/net/netdevsim/ipsec.c @@ -85,7 +85,7 @@ static int nsim_ipsec_parse_proto_keys(struct xfrm_state *xs, u32 *mykey, u32 *mysalt) { const char aes_gcm_name[] = "rfc4106(gcm(aes))"; - struct net_device *dev = xs->xso.dev; + struct net_device *dev = xs->xso.real_dev; unsigned char *key_data; char *alg_name = NULL; int key_len; @@ -134,7 +134,7 @@ static int nsim_ipsec_add_sa(struct xfrm_state *xs) u16 sa_idx; int ret; - dev = xs->xso.dev; + dev = xs->xso.real_dev; ns = netdev_priv(dev); ipsec = &ns->ipsec; @@ -194,7 +194,7 @@ static int nsim_ipsec_add_sa(struct xfrm_state *xs) static void nsim_ipsec_del_sa(struct xfrm_state *xs) { - struct netdevsim *ns = netdev_priv(xs->xso.dev); + struct netdevsim *ns = netdev_priv(xs->xso.real_dev); struct nsim_ipsec *ipsec = &ns->ipsec; u16 sa_idx; @@ -211,7 +211,7 @@ static void nsim_ipsec_del_sa(struct xfrm_state *xs) static bool nsim_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) { - struct netdevsim *ns = netdev_priv(xs->xso.dev); + struct netdevsim *ns = netdev_priv(xs->xso.real_dev); struct nsim_ipsec *ipsec = &ns->ipsec; ipsec->ok++; From patchwork Fri Jul 2 14:26:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 469799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99C81C11F68 for ; Fri, 2 Jul 2021 14:27:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E31161427 for ; Fri, 2 Jul 2021 14:27:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232979AbhGBOaH (ORCPT ); Fri, 2 Jul 2021 10:30:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233117AbhGBOaH (ORCPT ); Fri, 2 Jul 2021 10:30:07 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E0BCC061762 for ; Fri, 2 Jul 2021 07:27:35 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id a127so9055194pfa.10 for ; Fri, 02 Jul 2021 07:27:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mCsrNIIGzkQPNYdQ5/v9Q6VIXTZBuJVwXW0NnffdHXo=; b=iv2ZkdN4zWLPnbmQv2Ntr8MK2GQ6eBybcRjFwjlhI2Y+w5PRPtWf7yFpjzkypM3o9r 9IuG4EZ+FHx654KtLiH4TABP4jDWUPcSTZE2Hg5RvgQ1LX5OYTLOdY6T1tyXcly23qOd iMmQtY3JomcodHQzWFAnZGJLYJPhhgd4Y1r6HBph0ESA4dJalenaxKImUa++cMcbsc/s ENHWAJ9IDuPLsHILXouXEjrjcF406WdbtB0XPlJkBLxU670ZpFQU+ZpLV0QlL38PmQrI YA1KxzkW/9mgB8cPewQDOQGUa+KEwP7/m6pWKM6G0nt2CQIwMd/XtdqOpXqP8H1jPuM0 I8NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mCsrNIIGzkQPNYdQ5/v9Q6VIXTZBuJVwXW0NnffdHXo=; b=DNrOYdRTm10ZtEzSc8e3pc1fVVDX6eKuQUUqn40EFL1i/7L4sojNnIkvO946e7gUO/ cVxc1GkNN11nyYYnEFY/DvQrUL4mqFD3OSKph5mPg6zVvU6IRl7oKKmMVLvvNBgBmKjZ 4eaZxxEghC+1IRNOfX2CU+UdgVj5QiaPTJoGDA7BycgKhBXPF4213AVBmds4KAb7gdsQ cAYuGWtdIbQTkwB7Kdsj1uhDgGATiMr+b1831nXBewl+KNtZBCvx946Z/6clPy9W7Ttb u1tMNCCYCY1wqRDSKLsYnmMeTT+KjmVaSgytUNm5jhueL3rGZVGqmep0gEywodC2QT1D oD4A== X-Gm-Message-State: AOAM533nLEiV7OWiXMMY7G8nJ8dg20CAsCV2WYHpZLWUKiZHI+2zdhFB PNZU8rPePYksqAF4M8Vtv58= X-Google-Smtp-Source: ABdhPJzEvjiQ+VHYHX0kTuhnPEKsGDCvbSOQyGUTZgKdPWONZagGDgQeEMQ1sGDII5Z+noV3qAReJw== X-Received: by 2002:a63:1542:: with SMTP id 2mr271706pgv.329.1625236054590; Fri, 02 Jul 2021 07:27:34 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id nr12sm12683747pjb.1.2021.07.02.07.27.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jul 2021 07:27:34 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, netdev@vger.kernel.org, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net 5/8] bonding: fix suspicious RCU usage in bond_ipsec_del_sa() Date: Fri, 2 Jul 2021 14:26:45 +0000 Message-Id: <20210702142648.7677-6-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210702142648.7677-1-ap420073@gmail.com> References: <20210702142648.7677-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To dereference bond->curr_active_slave, it uses rcu_dereference(). But it and the caller doesn't acquire RCU so a warning occurs. So add rcu_read_lock(). Test commands: ip netns add A ip netns exec A bash modprobe netdevsim echo "1 1" > /sys/bus/netdevsim/new_device ip link add bond0 type bond ip link set eth0 master bond0 ip link set eth0 up ip link set bond0 up ip x s add proto esp dst 14.1.1.1 src 15.1.1.1 spi 0x07 mode \ transport reqid 0x07 replay-window 32 aead 'rfc4106(gcm(aes))' \ 0x44434241343332312423222114131211f4f3f2f1 128 sel src 14.0.0.52/24 \ dst 14.0.0.70/24 proto tcp offload dev bond0 dir in ip x s f Splat looks like: ============================= WARNING: suspicious RCU usage 5.13.0-rc3+ #1168 Not tainted ----------------------------- drivers/net/bonding/bond_main.c:448 suspicious rcu_dereference_check() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by ip/705: #0: ffff888106701780 (&net->xfrm.xfrm_cfg_mutex){+.+.}-{3:3}, at: xfrm_netlink_rcv+0x59/0x80 [xfrm_user] #1: ffff8880075b0098 (&x->lock){+.-.}-{2:2}, at: xfrm_state_delete+0x16/0x30 stack backtrace: CPU: 6 PID: 705 Comm: ip Not tainted 5.13.0-rc3+ #1168 Call Trace: dump_stack+0xa4/0xe5 bond_ipsec_del_sa+0x16a/0x1c0 [bonding] __xfrm_state_delete+0x51f/0x730 xfrm_state_delete+0x1e/0x30 xfrm_state_flush+0x22f/0x390 xfrm_flush_sa+0xd8/0x260 [xfrm_user] ? xfrm_flush_policy+0x290/0x290 [xfrm_user] xfrm_user_rcv_msg+0x331/0x660 [xfrm_user] ? rcu_read_lock_sched_held+0x91/0xc0 ? xfrm_user_state_lookup.constprop.39+0x320/0x320 [xfrm_user] ? find_held_lock+0x3a/0x1c0 ? mutex_lock_io_nested+0x1210/0x1210 ? sched_clock_cpu+0x18/0x170 netlink_rcv_skb+0x121/0x350 [ ... ] Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- drivers/net/bonding/bond_main.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index e1009e169d42..7659e1fab19e 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -444,21 +444,24 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) if (!bond_dev) return; + rcu_read_lock(); bond = netdev_priv(bond_dev); slave = rcu_dereference(bond->curr_active_slave); if (!slave) - return; + goto out; xs->xso.real_dev = slave->dev; if (!(slave->dev->xfrmdev_ops && slave->dev->xfrmdev_ops->xdo_dev_state_delete)) { slave_warn(bond_dev, slave->dev, "%s: no slave xdo_dev_state_delete\n", __func__); - return; + goto out; } slave->dev->xfrmdev_ops->xdo_dev_state_delete(xs); +out: + rcu_read_unlock(); } /** From patchwork Fri Jul 2 14:26:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 469798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C036AC11F68 for ; Fri, 2 Jul 2021 14:27:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A3FA9613CB for ; Fri, 2 Jul 2021 14:27:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233136AbhGBOaP (ORCPT ); Fri, 2 Jul 2021 10:30:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232850AbhGBOaO (ORCPT ); Fri, 2 Jul 2021 10:30:14 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0518C061762 for ; Fri, 2 Jul 2021 07:27:41 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id w22so5426991pff.5 for ; Fri, 02 Jul 2021 07:27:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nU7xv0r2pxdMfVRfM8DEYon7zSO5snWRMGGN+ysxbQ0=; b=Z4BZjkGczk1jPovG4BLPORqJ8MMHiqGyrJPJeVIVav2UBDKpQGs/HFe5OkB89TVxIH L4TG+WL5HVO9lTxCWvYsrEA0qzRX6Vjm653VmmAIeD97DNWJijK2DESvTiEjeGgVffrp oJvgDrMVrwQ7vEHMD8CK/BCuLA0+Vvd1fQnuGW+RDs8c84UsVOpKlQKXR1wnhS+xpBCX s5ncgwgI1z/pQiN0rFpusI6Gb7IB2WHL1GfX243T4dr+8A05U4JvVTXKkclT279Pcp0p /WWykMdg1GFCuVZIuYTmvMypaGd5d5iCYcIBeRdGqpReR+tshUtckCbIEMxoMRHXipjm n1Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nU7xv0r2pxdMfVRfM8DEYon7zSO5snWRMGGN+ysxbQ0=; b=VD8ILzS0GTkHO3XwspBdbQGKF4m/oNQ++WruAfLKKp5kmWaAIvpEeftPfguI05GmxF 5uXxjRlNrw6k3w5rZrOsg5GmUeD3AZZER2Diwj4g25Z+TAamrnbsg/N4tkgGrvHKlCKT 7wUuGZS41MDoHB8gwaZTk0tJlHOgudXo15kW6FaJYoLrGVmqEaRZ7DzAF5mQDIzlXfAN G9bxrbBeKmH5jv79v3oEzxqv2QMSFbUWPDC2W0//v8Vjh5du4xSwEf7kcnGXYZHaQplq +EbG4G2Yc+tNQMeLaVlE2kNY3n86LzxRmVQYMx/Xv24Dl/Ih7HQyPRFjbxKGQLmzF1qO CZzw== X-Gm-Message-State: AOAM532giTy3b5wrPoHUO2XiAwlWqtZG2N/y00ZrtQb8ExS9UJnUONjU W6FHtz7D8KqJF5JeeZqle5A= X-Google-Smtp-Source: ABdhPJwaRDMokHYVBe7wsAg9i82vD75fwa1p51GtEr/2Yh/aUbU25z09N2OlDitZqanSyVEPvneWcg== X-Received: by 2002:a63:5d65:: with SMTP id o37mr296156pgm.79.1625236061246; Fri, 02 Jul 2021 07:27:41 -0700 (PDT) Received: from localhost.localdomain ([49.173.165.50]) by smtp.gmail.com with ESMTPSA id nr12sm12683747pjb.1.2021.07.02.07.27.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jul 2021 07:27:40 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, kuba@kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, jarod@redhat.com, netdev@vger.kernel.org, intel-wired-lan@lists.osuosl.org Cc: ap420073@gmail.com Subject: [PATCH net 7/8] bonding: Add struct bond_ipesc to manage SA Date: Fri, 2 Jul 2021 14:26:47 +0000 Message-Id: <20210702142648.7677-8-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210702142648.7677-1-ap420073@gmail.com> References: <20210702142648.7677-1-ap420073@gmail.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org bonding has been supporting ipsec offload. When SA is added, bonding just passes SA to its own active real interface. But it doesn't manage SA. So, when events(add/del real interface, active real interface change, etc) occur, bonding can't handle that well because It doesn't manage SA. So some problems(panic, UAF, refcnt leak)occur. In order to make it stable, it should manage SA. That's the reason why struct bond_ipsec is added. When a new SA is added to bonding interface, it is stored in the bond_ipsec list. And the SA is passed to a current active real interface. If events occur, it uses bond_ipsec data to handle these events. bond->ipsec_list is protected by bond->ipsec_lock. If a current active real interface is changed, the following logic works. 1. delete all SAs from old active real interface 2. Add all SAs to the new active real interface. 3. If a new active real interface doesn't support ipsec offload or SA's option, it sets real_dev to NULL. Fixes: 18cb261afd7b ("bonding: support hardware encryption offload to slaves") Signed-off-by: Taehee Yoo --- drivers/net/bonding/bond_main.c | 134 +++++++++++++++++++++++++++----- include/net/bonding.h | 8 +- 2 files changed, 121 insertions(+), 21 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index f268e67cb2f0..d2d37efb61b6 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -401,6 +401,7 @@ static int bond_vlan_rx_kill_vid(struct net_device *bond_dev, static int bond_ipsec_add_sa(struct xfrm_state *xs) { struct net_device *bond_dev = xs->xso.dev; + struct bond_ipsec *ipsec; struct bonding *bond; struct slave *slave; int err; @@ -416,9 +417,6 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) return -ENODEV; } - xs->xso.real_dev = slave->dev; - bond->xs = xs; - if (!slave->dev->xfrmdev_ops || !slave->dev->xfrmdev_ops->xdo_dev_state_add || netif_is_bond_master(slave->dev)) { @@ -427,11 +425,58 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) return -EINVAL; } + ipsec = kmalloc(sizeof(*ipsec), GFP_ATOMIC); + if (!ipsec) { + rcu_read_unlock(); + return -ENOMEM; + } + xs->xso.real_dev = slave->dev; + err = slave->dev->xfrmdev_ops->xdo_dev_state_add(xs); + if (!err) { + ipsec->xs = xs; + INIT_LIST_HEAD(&ipsec->list); + spin_lock_bh(&bond->ipsec_lock); + list_add(&ipsec->list, &bond->ipsec_list); + spin_unlock_bh(&bond->ipsec_lock); + } else { + kfree(ipsec); + } rcu_read_unlock(); return err; } +static void bond_ipsec_add_sa_all(struct bonding *bond) +{ + struct net_device *bond_dev = bond->dev; + struct bond_ipsec *ipsec; + struct slave *slave; + + rcu_read_lock(); + slave = rcu_dereference(bond->curr_active_slave); + if (!slave) + goto out; + + if (!slave->dev->xfrmdev_ops || + !slave->dev->xfrmdev_ops->xdo_dev_state_add || + netif_is_bond_master(slave->dev)) { + slave_warn(bond_dev, slave->dev, "%s: no slave xdo_dev_state_add\n", __func__); + goto out; + } + + spin_lock_bh(&bond->ipsec_lock); + list_for_each_entry(ipsec, &bond->ipsec_list, list) { + ipsec->xs->xso.real_dev = slave->dev; + if (slave->dev->xfrmdev_ops->xdo_dev_state_add(ipsec->xs)) { + slave_warn(bond_dev, slave->dev, "%s: failed to add SA\n", __func__); + ipsec->xs->xso.real_dev = NULL; + } + } + spin_unlock_bh(&bond->ipsec_lock); +out: + rcu_read_unlock(); +} + /** * bond_ipsec_del_sa - clear out this specific SA * @xs: pointer to transformer state struct @@ -439,6 +484,7 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs) static void bond_ipsec_del_sa(struct xfrm_state *xs) { struct net_device *bond_dev = xs->xso.dev; + struct bond_ipsec *ipsec; struct bonding *bond; struct slave *slave; @@ -452,7 +498,10 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) if (!slave) goto out; - xs->xso.real_dev = slave->dev; + if (!xs->xso.real_dev) + goto out; + + WARN_ON(xs->xso.real_dev != slave->dev); if (!slave->dev->xfrmdev_ops || !slave->dev->xfrmdev_ops->xdo_dev_state_delete || @@ -463,6 +512,48 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) slave->dev->xfrmdev_ops->xdo_dev_state_delete(xs); out: + spin_lock_bh(&bond->ipsec_lock); + list_for_each_entry(ipsec, &bond->ipsec_list, list) { + if (ipsec->xs == xs) { + list_del(&ipsec->list); + kfree(ipsec); + break; + } + } + spin_unlock_bh(&bond->ipsec_lock); + rcu_read_unlock(); +} + +static void bond_ipsec_del_sa_all(struct bonding *bond) +{ + struct net_device *bond_dev = bond->dev; + struct bond_ipsec *ipsec; + struct slave *slave; + + rcu_read_lock(); + slave = rcu_dereference(bond->curr_active_slave); + if (!slave) { + rcu_read_unlock(); + return; + } + + spin_lock_bh(&bond->ipsec_lock); + list_for_each_entry(ipsec, &bond->ipsec_list, list) { + if (!ipsec->xs->xso.real_dev) + continue; + + if (!slave->dev->xfrmdev_ops || + !slave->dev->xfrmdev_ops->xdo_dev_state_delete || + netif_is_bond_master(slave->dev)) { + slave_warn(bond_dev, slave->dev, + "%s: no slave xdo_dev_state_delete\n", + __func__); + } else { + slave->dev->xfrmdev_ops->xdo_dev_state_delete(ipsec->xs); + } + ipsec->xs->xso.real_dev = NULL; + } + spin_unlock_bh(&bond->ipsec_lock); rcu_read_unlock(); } @@ -474,22 +565,27 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) static bool bond_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) { struct net_device *bond_dev = xs->xso.dev; - struct bonding *bond = netdev_priv(bond_dev); - struct slave *curr_active = rcu_dereference(bond->curr_active_slave); - struct net_device *slave_dev = curr_active->dev; + struct net_device *real_dev; + struct slave *curr_active; + struct bonding *bond; + + bond = netdev_priv(bond_dev); + curr_active = rcu_dereference(bond->curr_active_slave); + real_dev = curr_active->dev; if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) return true; - if (!slave_dev->xfrmdev_ops || - !slave_dev->xfrmdev_ops->xdo_dev_offload_ok || - netif_is_bond_master(slave_dev)) { - slave_warn(bond_dev, slave_dev, "%s: no slave xdo_dev_offload_ok\n", __func__); + if (!xs->xso.real_dev) + return false; + + if (!real_dev->xfrmdev_ops || + !real_dev->xfrmdev_ops->xdo_dev_offload_ok || + netif_is_bond_master(real_dev)) { return false; } - xs->xso.real_dev = slave_dev; - return slave_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); + return real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); } static const struct xfrmdev_ops bond_xfrmdev_ops = { @@ -1006,8 +1102,7 @@ void bond_change_active_slave(struct bonding *bond, struct slave *new_active) return; #ifdef CONFIG_XFRM_OFFLOAD - if (old_active && bond->xs) - bond_ipsec_del_sa(bond->xs); + bond_ipsec_del_sa_all(bond); #endif /* CONFIG_XFRM_OFFLOAD */ if (new_active) { @@ -1083,10 +1178,7 @@ void bond_change_active_slave(struct bonding *bond, struct slave *new_active) } #ifdef CONFIG_XFRM_OFFLOAD - if (new_active && bond->xs) { - xfrm_dev_state_flush(dev_net(bond->dev), bond->dev, true); - bond_ipsec_add_sa(bond->xs); - } + bond_ipsec_add_sa_all(bond); #endif /* CONFIG_XFRM_OFFLOAD */ /* resend IGMP joins since active slave has changed or @@ -3343,6 +3435,7 @@ static int bond_master_netdev_event(unsigned long event, return bond_event_changename(event_bond); case NETDEV_UNREGISTER: bond_remove_proc_entry(event_bond); + xfrm_dev_state_flush(dev_net(bond_dev), bond_dev, true); break; case NETDEV_REGISTER: bond_create_proc_entry(event_bond); @@ -4906,7 +4999,8 @@ void bond_setup(struct net_device *bond_dev) #ifdef CONFIG_XFRM_OFFLOAD /* set up xfrm device ops (only supported in active-backup right now) */ bond_dev->xfrmdev_ops = &bond_xfrmdev_ops; - bond->xs = NULL; + INIT_LIST_HEAD(&bond->ipsec_list); + spin_lock_init(&bond->ipsec_lock); #endif /* CONFIG_XFRM_OFFLOAD */ /* don't acquire bond device's netif_tx_lock when transmitting */ diff --git a/include/net/bonding.h b/include/net/bonding.h index 019e998d944a..ba03cf6165e4 100644 --- a/include/net/bonding.h +++ b/include/net/bonding.h @@ -201,6 +201,11 @@ struct bond_up_slave { */ #define BOND_LINK_NOCHANGE -1 +struct bond_ipsec { + struct list_head list; + struct xfrm_state *xs; +}; + /* * Here are the locking policies for the two bonding locks: * Get rcu_read_lock when reading or RTNL when writing slave list. @@ -249,7 +254,8 @@ struct bonding { #endif /* CONFIG_DEBUG_FS */ struct rtnl_link_stats64 bond_stats; #ifdef CONFIG_XFRM_OFFLOAD - struct xfrm_state *xs; + struct list_head ipsec_list; + spinlock_t ipsec_lock; #endif /* CONFIG_XFRM_OFFLOAD */ };