[5.4,42/43] mm: memcg: fix memcg reclaim soft lockup

Message ID	20200925124729.884757815@linuxfoundation.org
State	Superseded
Headers	show Return-Path: <SRS0=ikAt=DC=vger.kernel.org=stable-owner@kernel.org> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org> To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, stable@vger.kernel.org, Michal Hocko <mhocko@suse.com>, Xunlei Pang <xlpang@linux.alibaba.com>, Andrew Morton <akpm@linux-foundation.org>, Chris Down <chris@chrisdown.name>, Johannes Weiner <hannes@cmpxchg.org>, Linus Torvalds <torvalds@linux-foundation.org>, Julius Hemanth Pitti <jpitti@cisco.com> Subject: [PATCH 5.4 42/43] mm: memcg: fix memcg reclaim soft lockup Date: Fri, 25 Sep 2020 14:48:54 +0200 Message-Id: <20200925124729.884757815@linuxfoundation.org> In-Reply-To: <20200925124723.575329814@linuxfoundation.org> References: <20200925124723.575329814@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	None \| expand [5.4,02/43] ibmvnic fix NULL tx_pools and rx_tools issue at do_reset [5.4,03/43] ibmvnic: add missing parenthesis in do_reset() [5.4,04/43] kprobes: fix kill kprobe which has been marked as gone [5.4,05/43] mm/thp: fix __split_huge_pmd_locked() for migration PMD [5.4,06/43] act_ife: load meta modules before tcf_idr_check_alloc() [5.4,07/43] bnxt_en: Avoid sending firmware messages when AER error is detected. [5.4,08/43] bnxt_en: Fix NULL ptr dereference crash in bnxt_fw_reset_task() [5.4,09/43] cxgb4: fix memory leak during module unload [5.4,10/43] cxgb4: Fix offset when clearing filter byte counters [5.4,11/43] geneve: add transport ports in route lookup for geneve [5.4,12/43] hdlc_ppp: add range checks in ppp_cp_parse_cr() [5.4,13/43] ip: fix tos reflection in ack and reset packets [5.4,14/43] ipv4: Initialize flowi4_multipath_hash in data path [5.4,15/43] ipv4: Update exception handling for multipath routes via same device [5.4,16/43] ipv6: avoid lockdep issue in fib6_del() [5.4,17/43] net: bridge: br_vlan_get_pvid_rcu() should dereference the VLAN group under RCU [5.4,18/43] net: DCB: Validate DCB_ATTR_DCB_BUFFER argument [5.4,19/43] net: dsa: rtl8366: Properly clear member config [5.4,20/43] net: Fix bridge enslavement failure [5.4,21/43] net: ipv6: fix kconfig dependency warning for IPV6_SEG6_HMAC [5.4,22/43] net/mlx5: Fix FTE cleanup [5.4,23/43] net: sch_generic: aviod concurrent reset and enqueue op for lockless qdisc [5.4,24/43] net: sctp: Fix IPv6 ancestor_size calc in sctp_copy_descendant [5.4,25/43] nfp: use correct define to return NONE fec [5.4,26/43] taprio: Fix allowing too small intervals [5.4,27/43] tipc: Fix memory leak in tipc_group_create_member() [5.4,28/43] tipc: fix shutdown() of connection oriented socket [5.4,29/43] tipc: use skb_unshare() instead in tipc_buf_append() [5.4,30/43] net/mlx5e: Enable adding peer miss rules only if merged eswitch is supported [5.4,31/43] net/mlx5e: TLS, Do not expose FPGA TLS counter if not supported [5.4,32/43] bnxt_en: return proper error codes in bnxt_show_temp [5.4,33/43] bnxt_en: Protect bnxt_set_eee() and bnxt_set_pauseparam() with mutex. [5.4,34/43] net: lantiq: Wake TX queue again [5.4,35/43] net: lantiq: use netif_tx_napi_add() for TX NAPI [5.4,36/43] net: lantiq: Use napi_complete_done() [5.4,37/43] net: lantiq: Disable IRQs only if NAPI gets scheduled [5.4,38/43] net: phy: Avoid NPD upon phy_detach() when driver is unbound [5.4,39/43] net: phy: Do not warn in phy_stop() on PHY_DOWN [5.4,40/43] net: qrtr: check skb_put_padto() return value [5.4,41/43] net: add __must_check to skb_put_padto() [5.4,42/43] mm: memcg: fix memcg reclaim soft lockup [5.4,43/43] iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE

Message ID

20200925124729.884757815@linuxfoundation.org

State

Superseded

Headers

From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
	Xunlei Pang <xlpang@linux.alibaba.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Chris Down <chris@chrisdown.name>, Johannes Weiner <hannes@cmpxchg.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Julius Hemanth Pitti <jpitti@cisco.com>
Subject: [PATCH 5.4 42/43] mm: memcg: fix memcg reclaim soft lockup
Date: Fri, 25 Sep 2020 14:48:54 +0200
Message-Id: <20200925124729.884757815@linuxfoundation.org>
In-Reply-To: <20200925124723.575329814@linuxfoundation.org>
References: <20200925124723.575329814@linuxfoundation.org>
User-Agent: quilt/0.66
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk

Series

None | expand

Commit Message

Greg KH Sept. 25, 2020, 12:48 p.m. UTC

From: Xunlei Pang <xlpang@linux.alibaba.com>

commit e3336cab2579012b1e72b5265adf98e2d6e244ad upstream.

We've met softlockup with "CONFIG_PREEMPT_NONE=y", when the target memcg
doesn't have any reclaimable memory.

It can be easily reproduced as below:

  watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204]
  CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12
  Call Trace:
    shrink_lruvec+0x49f/0x640
    shrink_node+0x2a6/0x6f0
    do_try_to_free_pages+0xe9/0x3e0
    try_to_free_mem_cgroup_pages+0xef/0x1f0
    try_charge+0x2c1/0x750
    mem_cgroup_charge+0xd7/0x240
    __add_to_page_cache_locked+0x2fd/0x370
    add_to_page_cache_lru+0x4a/0xc0
    pagecache_get_page+0x10b/0x2f0
    filemap_fault+0x661/0xad0
    ext4_filemap_fault+0x2c/0x40
    __do_fault+0x4d/0xf9
    handle_mm_fault+0x1080/0x1790

It only happens on our 1-vcpu instances, because there's no chance for
oom reaper to run to reclaim the to-be-killed process.

Add a cond_resched() at the upper shrink_node_memcgs() to solve this
issue, this will mean that we will get a scheduling point for each memcg
in the reclaimed hierarchy without any dependency on the reclaimable
memory in that memcg thus making it more predictable.

Suggested-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Xunlei Pang <xlpang@linux.alibaba.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Chris Down <chris@chrisdown.name>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Link: http://lkml.kernel.org/r/1598495549-67324-1-git-send-email-xlpang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Julius Hemanth Pitti <jpitti@cisco.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 mm/vmscan.c |    8 ++++++++
 1 file changed, 8 insertions(+)

--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2775,6 +2775,14 @@  static bool shrink_node(pg_data_t *pgdat
 			unsigned long reclaimed;
 			unsigned long scanned;
 
+			/*
+			 * This loop can become CPU-bound when target memcgs
+			 * aren't eligible for reclaim - either because they
+			 * don't have any reclaimable pages, or because their
+			 * memory is explicitly protected. Avoid soft lockups.
+			 */
+			cond_resched();
+
 			switch (mem_cgroup_protected(root, memcg)) {
 			case MEMCG_PROT_MIN:
 				/*

[5.4,42/43] mm: memcg: fix memcg reclaim soft lockup

Commit Message

Patch