[5.4,009/123] ipvlan: add cond_resched_rcu() while processing muticast backlog

From: Mahesh Bandewar <maheshb@google.com>

From: Mahesh Bandewar <maheshb@google.com>

[ Upstream commit e18b353f102e371580f3f01dd47567a25acc3c1d ]

If there are substantial number of slaves created as simulated by
Syzbot, the backlog processing could take much longer and result
into the issue found in the Syzbot report.

INFO: rcu_sched detected stalls on CPUs/tasks:
        (detected by 1, t=10502 jiffies, g=5049, c=5048, q=752)
All QSes seen, last rcu_sched kthread activity 10502 (4294965563-4294955061), jiffies_till_next_fqs=1, root ->qsmask 0x0
syz-executor.1  R  running task on cpu   1  10984 11210   3866 0x30020008 179034491270
Call Trace:
 <IRQ>
 [<ffffffff81497163>] _sched_show_task kernel/sched/core.c:8063 [inline]
 [<ffffffff81497163>] _sched_show_task.cold+0x2fd/0x392 kernel/sched/core.c:8030
 [<ffffffff8146a91b>] sched_show_task+0xb/0x10 kernel/sched/core.c:8073
 [<ffffffff815c931b>] print_other_cpu_stall kernel/rcu/tree.c:1577 [inline]
 [<ffffffff815c931b>] check_cpu_stall kernel/rcu/tree.c:1695 [inline]
 [<ffffffff815c931b>] __rcu_pending kernel/rcu/tree.c:3478 [inline]
 [<ffffffff815c931b>] rcu_pending kernel/rcu/tree.c:3540 [inline]
 [<ffffffff815c931b>] rcu_check_callbacks.cold+0xbb4/0xc29 kernel/rcu/tree.c:2876
 [<ffffffff815e3962>] update_process_times+0x32/0x80 kernel/time/timer.c:1635
 [<ffffffff816164f0>] tick_sched_handle+0xa0/0x180 kernel/time/tick-sched.c:161
 [<ffffffff81616ae4>] tick_sched_timer+0x44/0x130 kernel/time/tick-sched.c:1193
 [<ffffffff815e75f7>] __run_hrtimer kernel/time/hrtimer.c:1393 [inline]
 [<ffffffff815e75f7>] __hrtimer_run_queues+0x307/0xd90 kernel/time/hrtimer.c:1455
 [<ffffffff815e90ea>] hrtimer_interrupt+0x2ea/0x730 kernel/time/hrtimer.c:1513
 [<ffffffff844050f4>] local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1031 [inline]
 [<ffffffff844050f4>] smp_apic_timer_interrupt+0x144/0x5e0 arch/x86/kernel/apic/apic.c:1056
 [<ffffffff84401cbe>] apic_timer_interrupt+0x8e/0xa0 arch/x86/entry/entry_64.S:778
RIP: 0010:do_raw_read_lock+0x22/0x80 kernel/locking/spinlock_debug.c:153
RSP: 0018:ffff8801dad07ab8 EFLAGS: 00000a02 ORIG_RAX: ffffffffffffff12
RAX: 0000000000000000 RBX: ffff8801c4135680 RCX: 0000000000000000
RDX: 1ffff10038826afe RSI: ffff88019d816bb8 RDI: ffff8801c41357f0
RBP: ffff8801dad07ac0 R08: 0000000000004b15 R09: 0000000000310273
R10: ffff88019d816bb8 R11: 0000000000000001 R12: ffff8801c41357e8
R13: 0000000000000000 R14: ffff8801cfb19850 R15: ffff8801cfb198b0
 [<ffffffff8101460e>] __raw_read_lock_bh include/linux/rwlock_api_smp.h:177 [inline]
 [<ffffffff8101460e>] _raw_read_lock_bh+0x3e/0x50 kernel/locking/spinlock.c:240
 [<ffffffff840d78ca>] ipv6_chk_mcast_addr+0x11a/0x6f0 net/ipv6/mcast.c:1006
 [<ffffffff84023439>] ip6_mc_input+0x319/0x8e0 net/ipv6/ip6_input.c:482
 [<ffffffff840211c8>] dst_input include/net/dst.h:449 [inline]
 [<ffffffff840211c8>] ip6_rcv_finish+0x408/0x610 net/ipv6/ip6_input.c:78
 [<ffffffff840214de>] NF_HOOK include/linux/netfilter.h:292 [inline]
 [<ffffffff840214de>] NF_HOOK include/linux/netfilter.h:286 [inline]
 [<ffffffff840214de>] ipv6_rcv+0x10e/0x420 net/ipv6/ip6_input.c:278
 [<ffffffff83a29efa>] __netif_receive_skb_one_core+0x12a/0x1f0 net/core/dev.c:5303
 [<ffffffff83a2a15c>] __netif_receive_skb+0x2c/0x1b0 net/core/dev.c:5417
 [<ffffffff83a2f536>] process_backlog+0x216/0x6c0 net/core/dev.c:6243
 [<ffffffff83a30d1b>] napi_poll net/core/dev.c:6680 [inline]
 [<ffffffff83a30d1b>] net_rx_action+0x47b/0xfb0 net/core/dev.c:6748
 [<ffffffff846002c8>] __do_softirq+0x2c8/0x99a kernel/softirq.c:317
 [<ffffffff813e656a>] invoke_softirq kernel/softirq.c:399 [inline]
 [<ffffffff813e656a>] irq_exit+0x16a/0x1a0 kernel/softirq.c:439
 [<ffffffff84405115>] exiting_irq arch/x86/include/asm/apic.h:561 [inline]
 [<ffffffff84405115>] smp_apic_timer_interrupt+0x165/0x5e0 arch/x86/kernel/apic/apic.c:1058
 [<ffffffff84401cbe>] apic_timer_interrupt+0x8e/0xa0 arch/x86/entry/entry_64.S:778
 </IRQ>
RIP: 0010:__sanitizer_cov_trace_pc+0x26/0x50 kernel/kcov.c:102
RSP: 0018:ffff880196033bd8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff12
RAX: ffff88019d8161c0 RBX: 00000000ffffffff RCX: ffffc90003501000
RDX: 0000000000000002 RSI: ffffffff816236d1 RDI: 0000000000000005
RBP: ffff880196033bd8 R08: ffff88019d8161c0 R09: 0000000000000000
R10: 1ffff10032c067f0 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000080 R14: 0000000000000000 R15: 0000000000000000
 [<ffffffff816236d1>] do_futex+0x151/0x1d50 kernel/futex.c:3548
 [<ffffffff816260f0>] C_SYSC_futex kernel/futex_compat.c:201 [inline]
 [<ffffffff816260f0>] compat_SyS_futex+0x270/0x3b0 kernel/futex_compat.c:175
 [<ffffffff8101da17>] do_syscall_32_irqs_on arch/x86/entry/common.c:353 [inline]
 [<ffffffff8101da17>] do_fast_syscall_32+0x357/0xe1c arch/x86/entry/common.c:415
 [<ffffffff84401a9b>] entry_SYSENTER_compat+0x8b/0x9d arch/x86/entry/entry_64_compat.S:139
RIP: 0023:0xf7f23c69
RSP: 002b:00000000f5d1f12c EFLAGS: 00000282 ORIG_RAX: 00000000000000f0
RAX: ffffffffffffffda RBX: 000000000816af88 RCX: 0000000000000080
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000816af8c
RBP: 00000000f5d1f228 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
rcu_sched kthread starved for 10502 jiffies! g5049 c5048 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=1
rcu_sched       R  running task on cpu   1  13048     8      2 0x90000000 179099587640
Call Trace:
 [<ffffffff8147321f>] context_switch+0x60f/0xa60 kernel/sched/core.c:3209
 [<ffffffff8100095a>] __schedule+0x5aa/0x1da0 kernel/sched/core.c:3934
 [<ffffffff810021df>] schedule+0x8f/0x1b0 kernel/sched/core.c:4011
 [<ffffffff8101116d>] schedule_timeout+0x50d/0xee0 kernel/time/timer.c:1803
 [<ffffffff815c13f1>] rcu_gp_kthread+0xda1/0x3b50 kernel/rcu/tree.c:2327
 [<ffffffff8144b318>] kthread+0x348/0x420 kernel/kthread.c:246
 [<ffffffff84400266>] ret_from_fork+0x56/0x70 arch/x86/entry/entry_64.S:393

Fixes: ba35f8588f47 (“ipvlan: Defer multicast / broadcast processing to a work-queue”)
Signed-off-by: Mahesh Bandewar <maheshb@google.com>
Reported-by: syzbot <syzkaller@googlegroups.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/net/ipvlan/ipvlan_core.c |    1 +
 1 file changed, 1 insertion(+)

Message ID	20200317103308.435913651@linuxfoundation.org
State	Superseded
Headers	show Return-Path: <SRS0=f8y2=5C=vger.kernel.org=stable-owner@kernel.org> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org> To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, stable@vger.kernel.org, Mahesh Bandewar <maheshb@google.com>, syzbot <syzkaller@googlegroups.com>, "David S. Miller" <davem@davemloft.net> Subject: [PATCH 5.4 009/123] ipvlan: add cond_resched_rcu() while processing muticast backlog Date: Tue, 17 Mar 2020 11:53:56 +0100 Message-Id: <20200317103308.435913651@linuxfoundation.org> In-Reply-To: <20200317103307.343627747@linuxfoundation.org> References: <20200317103307.343627747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: stable-owner@vger.kernel.org Precedence: bulk
Series	None \| expand [5.4,002/123] ALSA: hda/realtek - More constifications [5.4,003/123] ALSA: hda/realtek - Add Headset Mic supported for HP cPC [5.4,004/123] ALSA: hda/realtek - Fixed one of HP ALC671 platform Headset Mic supported [5.4,005/123] cgroup, netclassid: periodically release file_lock on classid updating [5.4,007/123] inet_diag: return classid for all socket types [5.4,009/123] ipvlan: add cond_resched_rcu() while processing muticast backlog [5.4,011/123] ipvlan: do not use cond_resched_rcu() in ipvlan_process_multicast() [5.4,012/123] ipvlan: dont deref eth hdr before checking its set [5.4,015/123] net: dsa: mv88e6xxx: fix lockup on warm boot [5.4,016/123] net: fec: validate the new settings in fec_enet_set_coalesce() [5.4,018/123] net/ipv6: use configured metric when add peer route [5.4,022/123] net/packet: tpacket_rcv: do not increment ring index on drop [5.4,023/123] net: phy: bcm63xx: fix OOPS due to missing driver name [5.4,025/123] net: systemport: fix index check to avoid an array out of bounds access [5.4,028/123] slip: make slhc_compress() more robust against malicious packets [5.4,031/123] bnxt_en: reinitialize IRQs when MTU is modified [5.4,032/123] bnxt_en: fix error handling when flashing from file [5.4,033/123] cgroup: memcg: net: do not associate sock with unrelated cgroup [5.4,034/123] net: memcg: late association of sock to memcg [5.4,035/123] net: memcg: fix lockdep splat in inet_csk_accept() [5.4,036/123] devlink: validate length of param values [5.4,039/123] nl802154: add missing attribute validation [5.4,042/123] macsec: add missing attribute validation for port [5.4,044/123] net: taprio: add missing attribute validation for txtime delay [5.4,045/123] team: add missing attribute validation for port ifindex [5.4,048/123] nfc: add missing attribute validation for SE API [5.4,052/123] net: phy: fix MDIO bus PM PHY resuming [5.4,053/123] net/ipv6: need update peer route when modify metric [5.4,056/123] net: dsa: Dont instantiate phylink for CPU/DSA ports unless needed [5.4,058/123] cgroup: cgroup_procs_next should increase position index [5.4,059/123] cgroup: Iterate tasks that did not finish do_exit() [5.4,062/123] virtio-blk: fix hw_queue stopped on arbitrary error [5.4,063/123] iommu/vt-d: quirk_ioat_snb_local_iommu: replace WARN_TAINT with pr_warn + add_taint [5.4,068/123] workqueue: dont use wq_select_unbound_cpu() for bound works [5.4,069/123] drm/amd/display: remove duplicated assignment to grph_obj_type [5.4,074/123] ktest: Add timeout for ssh sync testing [5.4,076/123] gfs2_atomic_open(): fix O_EXCL\|O_CREAT handling on cold dcache [5.4,079/123] ARC: define __ALIGN_STR and __ALIGN symbols for ARC [5.4,081/123] s390/dasd: fix data corruption for thin provisioned devices [5.4,083/123] blk-iocost: fix incorrect vtime comparison in iocg_is_idle() [5.4,085/123] macintosh: windfarm: fix MODINFO regression [5.4,087/123] efi: Fix a race and a buffer overflow while reading efivars via sysfs [5.4,088/123] efi: Add a sanity check to efivar_store_raw() [5.4,090/123] mt76: fix array overflow on receiving too many fragments for a packet [5.4,091/123] perf/amd/uncore: Replace manual sampling check with CAP_NO_INTERRUPT flag [5.4,092/123] x86/mce: Fix logic and comments around MSR_PPIN_CTL [5.4,095/123] iommu/vt-d: Fix RCU list debugging warnings [5.4,098/123] clk: imx8mn: Fix incorrect clock defines [5.4,099/123] pinctrl: meson-gxl: fix GPIOX sdio pins [5.4,101/123] virtio_ring: Fix mem leak with vring_new_virtqueue() [5.4,104/123] drm/i915/gvt: Fix unnecessary schedule timer when no vGPU exits [5.4,107/123] i2c: gpio: suppress error on probe defer [5.4,108/123] nl80211: add missing attribute validation for critical protocol indication [5.4,109/123] nl80211: add missing attribute validation for beacon report scanning [5.4,110/123] nl80211: add missing attribute validation for channel switch [5.4,111/123] perf bench futex-wake: Restore thread count default to online CPU count [5.4,113/123] netfilter: nft_payload: add missing attribute validation for payload csum flags [5.4,116/123] netfilter: nft_chain_nat: inet family is missing module ownership [5.4,117/123] iommu/vt-d: Fix the wrong printing in RHSA parsing [5.4,119/123] i2c: acpi: put device when verifying client fails [5.4,121/123] ipv6: restrict IPV6_ADDRFORM operation [5.4,123/123] net/smc: cancel event worker during device removal

[5.4,009/123] ipvlan: add cond_resched_rcu() while processing muticast backlog

Commit Message

Patch