From patchwork Fri Jun 19 12:25:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 224280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE783C433E1 for ; Fri, 19 Jun 2020 12:26:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8E452206C3 for ; Fri, 19 Jun 2020 12:26:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="Y1aWKKDh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726124AbgFSM0s (ORCPT ); Fri, 19 Jun 2020 08:26:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733063AbgFSMZ7 (ORCPT ); Fri, 19 Jun 2020 08:25:59 -0400 Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E00CFC06174E for ; Fri, 19 Jun 2020 05:25:58 -0700 (PDT) Received: by mail-qv1-xf44.google.com with SMTP id er17so4323677qvb.8 for ; Fri, 19 Jun 2020 05:25:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=Kw6dC+AwL0270Qh6aBSl6rxU82P9FZlXOVi7O4a23y8=; b=Y1aWKKDhKY+RSfaQQ3Jn8lOUjdX22c7tKA8B8H+ufdGf7smGVWXkoqIKR2dXKRnt5V i8OvMgAZrmiFbFZ9P9EYo6d1e8BehgSylM9tNrl0fBp3fTy2/xQg7mhGjZqNpApfyLaH shZ12aujLTihwnRs5dBgURcD1PcD10EeYEXstRMpyhNH61hBcJHa4NSKTjkef3TCM/lQ pZwTW53grP+gPoiujjS4Rojbz0QndWkCwmfM3n1FoiutkJvARq8FBfkUFxGgoETMmHNi WdlkXuxqmSkBv+3W2qC4F8aSY4lxm96gckYZaAdCDsbs5FvHwCsIKZO0oHpEd6Z/zy5r g4kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=Kw6dC+AwL0270Qh6aBSl6rxU82P9FZlXOVi7O4a23y8=; b=UFfKhUKmq0mKUSeO+UR/LWmsG1113RPYF/to1DTakxVNwUHnxBpTVd6jric/2UV0kk iGLqTGdxJtFI01m/i4J3DuTxxLtDMzEBseKnJs8iinJnnKwzAfXatDqUHlrqrTOwBiJx B9YbDuc9C83iFDacSXJhZM2VHL5kYbGcSkHQYZunRcwWdtpF4qDsa7nJCX95xjNfnIjG 5g0RzFE3tiRD+zEEsBY5qG2V0SxDqcMwDy1CddWd29AM4GNSWytwlNWrjjA57jzgs2/h 6xYJ7lsLgarH/mVWLJsPGonrGVZqFIp4hSfb1+NJlPaFIapf+Ys9d6cIjM2qU9Oe+iBI yaCg== X-Gm-Message-State: AOAM531YjoiHPITz4j8sJDAvtZUwEYLMOtnROEuxZDasSaOxba4C9sKT 1ZrFTu/v5MJs7pWaTYtzIHGiwBKVTA8= X-Google-Smtp-Source: ABdhPJwB+FtxBO06pXuncWvlXCrg11+xFqDCvaxZZZbzhQ/vwm37W8AnZvhxlSiOFjZ3WZWFU3GjRQ== X-Received: by 2002:a05:6214:1812:: with SMTP id o18mr8639970qvw.194.1592569557811; Fri, 19 Jun 2020 05:25:57 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id z77sm6519818qka.59.2020.06.19.05.25.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Jun 2020 05:25:56 -0700 (PDT) From: Pavel Tatashin To: stable@vger.kernel.org, akpm@linux-foundation.org, mhocko@suse.com, dan.j.williams@intel.com, shile.zhang@linux.alibaba.com, daniel.m.jordan@oracle.com, pasha.tatashin@soleen.com, ktkhai@virtuozzo.com, david@redhat.com, jmorris@namei.org, sashal@kernel.org, vbabka@suse.cz, gregkh@linuxfoundation.org, torvalds@linux-foundation.org Subject: [PATCH 1/3] mm/pagealloc.c: call touch_nmi_watchdog() on max order boundaries in deferred init Date: Fri, 19 Jun 2020 08:25:53 -0400 Message-Id: <20200619122555.372957-1-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Daniel Jordan From: Daniel Jordan commit 117003c32771df617acf66e140fbdbdeb0ac71f5 upstream. Patch series "initialize deferred pages with interrupts enabled", v4. Keep interrupts enabled during deferred page initialization in order to make code more modular and allow jiffies to update. Original approach, and discussion can be found here: http://lkml.kernel.org/r/20200311123848.118638-1-shile.zhang@linux.alibaba.com This patch (of 3): deferred_init_memmap() disables interrupts the entire time, so it calls touch_nmi_watchdog() periodically to avoid soft lockup splats. Soon it will run with interrupts enabled, at which point cond_resched() should be used instead. deferred_grow_zone() makes the same watchdog calls through code shared with deferred init but will continue to run with interrupts disabled, so it can't call cond_resched(). Pull the watchdog calls up to these two places to allow the first to be changed later, independently of the second. The frequency reduces from twice per pageblock (init and free) to once per max order block. Fixes: 3a2d7fa8a3d5 ("mm: disable interrupts while initializing deferred pages") Signed-off-by: Daniel Jordan Signed-off-by: Pavel Tatashin Signed-off-by: Andrew Morton Reviewed-by: David Hildenbrand Acked-by: Michal Hocko Acked-by: Vlastimil Babka Cc: Dan Williams Cc: Shile Zhang Cc: Kirill Tkhai Cc: James Morris Cc: Sasha Levin Cc: Yiqian Wei Cc: [4.17+] Link: http://lkml.kernel.org/r/20200403140952.17177-2-pasha.tatashin@soleen.com Signed-off-by: Linus Torvalds --- mm/page_alloc.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 13cc653122b7..f7130e4445d3 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1692,7 +1692,6 @@ static void __init deferred_free_pages(unsigned long pfn, } else if (!(pfn & nr_pgmask)) { deferred_free_range(pfn - nr_free, nr_free); nr_free = 1; - touch_nmi_watchdog(); } else { nr_free++; } @@ -1722,7 +1721,6 @@ static unsigned long __init deferred_init_pages(struct zone *zone, continue; } else if (!page || !(pfn & nr_pgmask)) { page = pfn_to_page(pfn); - touch_nmi_watchdog(); } else { page++; } @@ -1862,8 +1860,10 @@ static int __init deferred_init_memmap(void *data) * that we can avoid introducing any issues with the buddy * allocator. */ - while (spfn < epfn) + while (spfn < epfn) { nr_pages += deferred_init_maxorder(&i, zone, &spfn, &epfn); + touch_nmi_watchdog(); + } zone_empty: pgdat_resize_unlock(pgdat, &flags); @@ -1947,6 +1947,7 @@ deferred_grow_zone(struct zone *zone, unsigned int order) first_deferred_pfn = spfn; nr_pages += deferred_init_maxorder(&i, zone, &spfn, &epfn); + touch_nmi_watchdog(); /* We should only stop along section boundaries */ if ((first_deferred_pfn ^ spfn) < PAGES_PER_SECTION) From patchwork Fri Jun 19 12:25:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 224281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A5C7C433E0 for ; Fri, 19 Jun 2020 12:26:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 723BF206C3 for ; Fri, 19 Jun 2020 12:26:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="dmb4yE0A" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731860AbgFSM02 (ORCPT ); Fri, 19 Jun 2020 08:26:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733076AbgFSM0D (ORCPT ); Fri, 19 Jun 2020 08:26:03 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65E77C0613EF for ; Fri, 19 Jun 2020 05:26:02 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id d27so6993882qtg.4 for ; Fri, 19 Jun 2020 05:26:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Tp5TAaboXjZh4+uNgqbbixbFBgUJ2dwArVmWl3kTlS0=; b=dmb4yE0AzfFo5GF4ovr6s8HZJ5MoCArD/qBKhF30NOrIHg/6lQUmAEkX+Jl4C19Xlh e6lBcTvLUj8W94YO/xiChJHVmam4RhYCxWWd0gUe9cPaa9/D3/PgCVSulgKv5wc4SNv1 nSR5zNtigm49UJP919RMJizN9+VkE7SWFH85gnDpn2s1MTfHTvrEuAZASChbKVQjFWTs ZdApcFjxFy1iQ6F6rsbK+0PoEL+G8yu2eVuQZS9Jed1v5yV39HR6UsrHrIadVxKgJBAY 1qDBlsnVG37/0AQ1baicLp8RXzueXVTUvxcLN7ll4wxN2unxLucpqna214tPWiWIqDtZ 7s/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Tp5TAaboXjZh4+uNgqbbixbFBgUJ2dwArVmWl3kTlS0=; b=Tceswp1/LMp8Ha/d3wKUMgRdNgcJYUd0JkaG3dq99846XQuyMDt5/pY92Fky3n/093 IvjxPw2FBbak+KkGDhIO0S7uFdNt+NSpQFojm8Nv6NRvAU4sGURmtby+j0KrdzhXvGed yP0fvk6taO6ZjPga9x129s0QTcDEcFPDCLLvRC+lf4fczk9OTA5rgUUzPw0XXgvhUfM0 zvH6BPiUbOLwMVGWV9jJYYZSzOWLuj+Gh7kZdAYsrS5cWV6XZaVg0DByBFBFXn9wQzB5 Ndc4QSfeBPAdKx/8GfusMiTwGHu3VrNMd2an46vXx4BPcFTY+KijCSATpMDRmoByMbWK 34nQ== X-Gm-Message-State: AOAM533rdOIgwMgXNqL4H6EJPvKx9RQ9dsj42thChpzKWo/BHhiFMxwq nWWNqUfGh48htrgIPIBnChem+pbXxno= X-Google-Smtp-Source: ABdhPJyC6/k5JWEriaPZwMPdL7UvPwxA31ywgl2eedjT0MvK6CpexlZEszp/hM3NMAd9mhygFsrdxg== X-Received: by 2002:aed:358c:: with SMTP id c12mr2941213qte.214.1592569561206; Fri, 19 Jun 2020 05:26:01 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id z77sm6519818qka.59.2020.06.19.05.25.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Jun 2020 05:26:00 -0700 (PDT) From: Pavel Tatashin To: stable@vger.kernel.org, akpm@linux-foundation.org, mhocko@suse.com, dan.j.williams@intel.com, shile.zhang@linux.alibaba.com, daniel.m.jordan@oracle.com, pasha.tatashin@soleen.com, ktkhai@virtuozzo.com, david@redhat.com, jmorris@namei.org, sashal@kernel.org, vbabka@suse.cz, gregkh@linuxfoundation.org, torvalds@linux-foundation.org Subject: [PATCH 3/3] mm: call cond_resched() from deferred_init_memmap() Date: Fri, 19 Jun 2020 08:25:55 -0400 Message-Id: <20200619122555.372957-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200619122555.372957-1-pasha.tatashin@soleen.com> References: <20200619122555.372957-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Pavel Tatashin commit 3d060856adfc59afb9d029c233141334cfaba418 upstream. Now that deferred pages are initialized with interrupts enabled we can replace touch_nmi_watchdog() with cond_resched(), as it was before 3a2d7fa8a3d5. For now, we cannot do the same in deferred_grow_zone() as it is still initializes pages with interrupts disabled. This change fixes RCU problem described in https://lkml.kernel.org/r/20200401104156.11564-2-david@redhat.com [ 60.474005] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: [ 60.475000] rcu: 1-...0: (0 ticks this GP) idle=02a/1/0x4000000000000000 softirq=1/1 fqs=15000 [ 60.475000] rcu: (detected by 0, t=60002 jiffies, g=-1199, q=1) [ 60.475000] Sending NMI from CPU 0 to CPUs 1: [ 1.760091] NMI backtrace for cpu 1 [ 1.760091] CPU: 1 PID: 20 Comm: pgdatinit0 Not tainted 4.18.0-147.9.1.el8_1.x86_64 #1 [ 1.760091] Hardware name: Red Hat KVM, BIOS 1.13.0-1.module+el8.2.0+5520+4e5817f3 04/01/2014 [ 1.760091] RIP: 0010:__init_single_page.isra.65+0x10/0x4f [ 1.760091] Code: 48 83 cf 63 48 89 f8 0f 1f 40 00 48 89 c6 48 89 d7 e8 6b 18 80 ff 66 90 5b c3 31 c0 b9 10 00 00 00 49 89 f8 48 c1 e6 33 f3 ab 07 00 00 00 48 c1 e2 36 41 c7 40 34 01 00 00 00 48 c1 e0 33 41 [ 1.760091] RSP: 0000:ffffba783123be40 EFLAGS: 00000006 [ 1.760091] RAX: 0000000000000000 RBX: fffffad34405e300 RCX: 0000000000000000 [ 1.760091] RDX: 0000000000000000 RSI: 0010000000000000 RDI: fffffad34405e340 [ 1.760091] RBP: 0000000033f3177e R08: fffffad34405e300 R09: 0000000000000002 [ 1.760091] R10: 000000000000002b R11: ffff98afb691a500 R12: 0000000000000002 [ 1.760091] R13: 0000000000000000 R14: 000000003f03ea00 R15: 000000003e10178c [ 1.760091] FS: 0000000000000000(0000) GS:ffff9c9ebeb00000(0000) knlGS:0000000000000000 [ 1.760091] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1.760091] CR2: 00000000ffffffff CR3: 000000a1cf20a001 CR4: 00000000003606e0 [ 1.760091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1.760091] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 1.760091] Call Trace: [ 1.760091] deferred_init_pages+0x8f/0xbf [ 1.760091] deferred_init_memmap+0x184/0x29d [ 1.760091] ? deferred_free_pages.isra.97+0xba/0xba [ 1.760091] kthread+0x112/0x130 [ 1.760091] ? kthread_flush_work_fn+0x10/0x10 [ 1.760091] ret_from_fork+0x35/0x40 [ 89.123011] node 0 initialised, 1055935372 pages in 88650ms Fixes: 3a2d7fa8a3d5 ("mm: disable interrupts while initializing deferred pages") Reported-by: Yiqian Wei Signed-off-by: Pavel Tatashin Signed-off-by: Andrew Morton Tested-by: David Hildenbrand Reviewed-by: Daniel Jordan Reviewed-by: David Hildenbrand Reviewed-by: Pankaj Gupta Acked-by: Michal Hocko Cc: Dan Williams Cc: James Morris Cc: Kirill Tkhai Cc: Sasha Levin Cc: Shile Zhang Cc: Vlastimil Babka Cc: [4.17+] Link: http://lkml.kernel.org/r/20200403140952.17177-4-pasha.tatashin@soleen.com Signed-off-by: Linus Torvalds (cherry picked from commit da97f2d56bbd880b4138916a7ef96f9881a551b2) --- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b03da51dee5d..d0c0d9364aa6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1869,7 +1869,7 @@ static int __init deferred_init_memmap(void *data) */ while (spfn < epfn) { nr_pages += deferred_init_maxorder(&i, zone, &spfn, &epfn); - touch_nmi_watchdog(); + cond_resched(); } zone_empty: /* Sanity check that the next zone really is unpopulated */