From patchwork Mon Jul 29 15:40:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Wagner X-Patchwork-Id: 170018 Delivered-To: patch@linaro.org Received: by 2002:a92:512:0:0:0:0:0 with SMTP id q18csp1427518ile; Mon, 29 Jul 2019 08:40:58 -0700 (PDT) X-Google-Smtp-Source: APXvYqyLjTm2zwL2srDxg+7X+3qO3UWVzQAjDtY+sFYo0UPT5FtQVTKQ2G7semuQCfZ3k9vCer+M X-Received: by 2002:a17:90a:f488:: with SMTP id bx8mr111842985pjb.91.1564414857723; Mon, 29 Jul 2019 08:40:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564414857; cv=none; d=google.com; s=arc-20160816; b=cMcT2I0FHrFsOX/2NfZPTKYkVtdQPu+LSJa2EhmFEDUkFwwdcqKRvp1EZc3eOfCUpZ l5x8oVnrn3FRQjm7Uq9cl72VfKSu2dM3oCYlWtzBpVPS+zYu+mEc2DbMKxlarGdhFkbT xZwMkeP3QGaCxBUmmBaIKxAkC1qTQVA/OYg+h9aiP38g9BBm3W+fil20yHBQj8uqvyCd nYuaKsb8gw0yHIbhzzObwaZBUGISvTiAvnUBz7ccKQZMTtrc6qoenuITysMr9wDAuF+U J4PbwGbaFkYF9ykJdz0RUIXkamWQMBBBycg6Y9OZPsfcecA+Ou2u9eeVR1pTaEK267k1 OjbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=loioVCoTKsMVvmmTkaQp/BDJfjR6FcYDuTi4m90+06Q=; b=y4IcOOhsRyUczdayFy0OqDnu1JYygfyEkxQfEKIp36c+PYXl37+ho9uTk8DTqOwMey BiROkl4iccaTtFkRKN7c2zPu6QuNG3u0i8/fMSucYzkTcCuneW68pRq5TnY2DhO6WTA2 Jd3Q9ErjexanV1WxCXZ2N74Ywq4A4ETCpcvb7nsr5VdtIK5uv1mFBVsyyr3ZYacEL/7y px0q8dznC8CiCMxQzao7cwXkNXnI9ZTxexwQeSFp1ybfzZusNQGssLbmj2PxkBTZ3+sJ 4nIH/GZGrWq3M/lMcEwy1cckAnVCf+KlOmZ3iguECdkSIYWvOMXsP7w7LaAr885uoj3c LxKw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=monom.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n6si33301521pgs.333.2019.07.29.08.40.57; Mon, 29 Jul 2019 08:40:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=monom.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388128AbfG2Pk4 (ORCPT + 29 others); Mon, 29 Jul 2019 11:40:56 -0400 Received: from mail.monom.org ([188.138.9.77]:49936 "EHLO mail.monom.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388096AbfG2Pky (ORCPT ); Mon, 29 Jul 2019 11:40:54 -0400 Received: from mail.monom.org (localhost [127.0.0.1]) by filter.mynetwork.local (Postfix) with ESMTP id 8A657500767; Mon, 29 Jul 2019 17:40:51 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.monom.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (b9168f76.cgn.dg-w.de [185.22.143.118]) by mail.monom.org (Postfix) with ESMTPSA id 56685500959; Mon, 29 Jul 2019 17:40:51 +0200 (CEST) From: Daniel Wagner To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Greg KH , Jon Hunter Subject: [PATCH 4.4 1/2] vmstat: Remove BUG_ON from vmstat_update Date: Mon, 29 Jul 2019 17:40:45 +0200 Message-Id: <20190729154046.8824-2-wagi@monom.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190729154046.8824-1-wagi@monom.org> References: <20190729154046.8824-1-wagi@monom.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Christoph Lameter [ Upstream commit 587198ba5206cdf0d30855f7361af950a4172cd6 ] If we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag is set by any code then the shepherd will start dealing with the situation and reenable the vmstat workers when necessary again. Since commit 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") quiet_vmstat might update cpu_stat_off and mark a particular cpu to be handled by vmstat_shepherd. This might trigger a VM_BUG_ON in vmstat_update because the work item might have been sleeping during the idle period and see the cpu_stat_off updated after the wake up. The VM_BUG_ON is therefore misleading and no more appropriate. Moreover it doesn't really suite any protection from real bugs because vmstat_shepherd will simply reschedule the vmstat_work anytime it sees a particular cpu set or vmstat_update would do the same from the worker context directly. Even when the two would race the result wouldn't be incorrect as the counters update is fully idempotent. Reported-by: Sasha Levin Signed-off-by: Christoph Lameter Acked-by: Michal Hocko Cc: Johannes Weiner Cc: Tetsuo Handa Cc: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Daniel Wagner --- mm/vmstat.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) -- 2.20.1 diff --git a/mm/vmstat.c b/mm/vmstat.c index dd0a13013cb4..233045057a30 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1407,17 +1407,7 @@ static void vmstat_update(struct work_struct *w) * Defer the checking for differentials to the * shepherd thread on a different processor. */ - int r; - /* - * Shepherd work thread does not race since it never - * changes the bit if its zero but the cpu - * online / off line code may race if - * worker threads are still allowed during - * shutdown / startup. - */ - r = cpumask_test_and_set_cpu(smp_processor_id(), - cpu_stat_off); - VM_BUG_ON(r); + cpumask_set_cpu(smp_processor_id(), cpu_stat_off); } } From patchwork Mon Jul 29 15:40:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Wagner X-Patchwork-Id: 170020 Delivered-To: patch@linaro.org Received: by 2002:a92:512:0:0:0:0:0 with SMTP id q18csp1427569ile; Mon, 29 Jul 2019 08:41:02 -0700 (PDT) X-Google-Smtp-Source: APXvYqySWCD0CKKQZjDndAEw9VT9qGHKjTjXfKDO9j+PoU3f1R2Hn5XG1bgJEAH3IU65infFSQEX X-Received: by 2002:a17:902:900a:: with SMTP id a10mr112556014plp.281.1564414862055; Mon, 29 Jul 2019 08:41:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564414862; cv=none; d=google.com; s=arc-20160816; b=Chn6Go+GgYADHe6D0ZwzjJbaommhgKa+Jax1kXRDqzdhbY+LYbyz9RF6wHfv7yw4W2 ufPGYYHfAsNZLRqIm2VSc4o1REzyCA96DygY7PkdbkOd+oxYXJUqIu+vYmNRRSEU6Z6F dtUcPQxuYBakGuvC7nNZpwIHPtcr+6ssymkMeZbgI7SVM1u6sXmZKsh8Y3rs9oVwyz9v C4hMge2EjwVj5oj7dFTfxNg6m9x8Zbwi+igd8FOITl9jIri5pGswAn6HzE8xZi3Xd0SQ qgiL0x3Qw8hWSjt97aAJcMovAkOxVPf5S70xdTMvCd+zu1W56b/BHWiL/V3OWBqntg5r mfAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=nbWG9zVN6c1+bHV7+mv0H52KgOCY8EmRFjSmhIDk04I=; b=p0BMeNq4yM/wBqfCw7oYGiZXx6JOezP/2jJrA8RVaEER3+LQMGdD32rpbRyZ+E+Xq4 aV8rBG96zf/j+fsTIwnClsn8hvNAbx0xKXr9GxFX2M84KCgeBNXBxEiVjjVaT8RqlwE2 4yLT+qxrAHAfCSFnFg62l2gmWtjgQjTwXAKvFn42oV2o0xBON5R9rTJylYhdo8lcKpKH zauTRJxaiuh0+Pw6D7HMSn6oY4Ag4eJiN+LGOaYOiZQddq3NLtUR4uUGAL7xK59F3xnB x6rz4o0vZ3D+MZwuGqUfK7Zgq9tdn2kJUaMu+sdLr1pngqPi/T1ag0pTsJy7j4G7+7W+ ecFg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=monom.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u38si27554533pgn.79.2019.07.29.08.41.01; Mon, 29 Jul 2019 08:41:02 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=monom.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388139AbfG2Pk6 (ORCPT + 29 others); Mon, 29 Jul 2019 11:40:58 -0400 Received: from mail.monom.org ([188.138.9.77]:49942 "EHLO mail.monom.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388111AbfG2Pkz (ORCPT ); Mon, 29 Jul 2019 11:40:55 -0400 Received: from mail.monom.org (localhost [127.0.0.1]) by filter.mynetwork.local (Postfix) with ESMTP id 155B45006D3; Mon, 29 Jul 2019 17:40:52 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.monom.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (b9168f76.cgn.dg-w.de [185.22.143.118]) by mail.monom.org (Postfix) with ESMTPSA id C8B04500984; Mon, 29 Jul 2019 17:40:51 +0200 (CEST) From: Daniel Wagner To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Greg KH , Jon Hunter Subject: [PATCH 4.4 2/2] mm, vmstat: make quiet_vmstat lighter Date: Mon, 29 Jul 2019 17:40:46 +0200 Message-Id: <20190729154046.8824-3-wagi@monom.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190729154046.8824-1-wagi@monom.org> References: <20190729154046.8824-1-wagi@monom.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Michal Hocko [ Upstream commit f01f17d3705bb6081c9e5728078f64067982be36 ] Mike has reported a considerable overhead of refresh_cpu_vm_stats from the idle entry during pipe test: 12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_to This is caused by commit 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") which has placed quiet_vmstat into cpu_idle_loop. The main reason here seems to be that the idle entry has to get over all zones and perform atomic operations for each vmstat entry even though there might be no per cpu diffs. This is a pointless overhead for _each_ idle entry. Make sure that quiet_vmstat is as light as possible. First of all it doesn't make any sense to do any local sync if the current cpu is already set in oncpu_stat_off because vmstat_update puts itself there only if there is nothing to do. Then we can check need_update which should be a cheap way to check for potential per-cpu diffs and only then do refresh_cpu_vm_stats. The original patch also did cancel_delayed_work which we are not doing here. There are two reasons for that. Firstly cancel_delayed_work from idle context will blow up on RT kernels (reported by Mike): CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0-rt3 #7 Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013 Call Trace: dump_stack+0x49/0x67 ___might_sleep+0xf5/0x180 rt_spin_lock+0x20/0x50 try_to_grab_pending+0x69/0x240 cancel_delayed_work+0x26/0xe0 quiet_vmstat+0x75/0xa0 cpu_idle_loop+0x38/0x3e0 cpu_startup_entry+0x13/0x20 start_secondary+0x114/0x140 And secondly, even on !RT kernels it might add some non trivial overhead which is not necessary. Even if the vmstat worker wakes up and preempts idle then it will be most likely a single shot noop because the stats were already synced and so it would end up on the oncpu_stat_off anyway. We just need to teach both vmstat_shepherd and vmstat_update to stop scheduling the worker if there is nothing to do. [mgalbraith@suse.de: cancel pending work of the cpu_stat_off CPU] Signed-off-by: Michal Hocko Reported-by: Mike Galbraith Acked-by: Christoph Lameter Signed-off-by: Mike Galbraith Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Daniel Wagner --- mm/vmstat.c | 68 ++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 46 insertions(+), 22 deletions(-) -- 2.20.1 diff --git a/mm/vmstat.c b/mm/vmstat.c index 233045057a30..59e131e82b81 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1395,10 +1395,15 @@ static void vmstat_update(struct work_struct *w) * Counters were updated so we expect more updates * to occur in the future. Keep on running the * update worker thread. + * If we were marked on cpu_stat_off clear the flag + * so that vmstat_shepherd doesn't schedule us again. */ - queue_delayed_work_on(smp_processor_id(), vmstat_wq, - this_cpu_ptr(&vmstat_work), - round_jiffies_relative(sysctl_stat_interval)); + if (!cpumask_test_and_clear_cpu(smp_processor_id(), + cpu_stat_off)) { + queue_delayed_work_on(smp_processor_id(), vmstat_wq, + this_cpu_ptr(&vmstat_work), + round_jiffies_relative(sysctl_stat_interval)); + } } else { /* * We did not update any counters so the app may be in @@ -1416,18 +1421,6 @@ static void vmstat_update(struct work_struct *w) * until the diffs stay at zero. The function is used by NOHZ and can only be * invoked when tick processing is not active. */ -void quiet_vmstat(void) -{ - if (system_state != SYSTEM_RUNNING) - return; - - do { - if (!cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off)) - cancel_delayed_work(this_cpu_ptr(&vmstat_work)); - - } while (refresh_cpu_vm_stats(false)); -} - /* * Check if the diffs for a certain cpu indicate that * an update is needed. @@ -1451,6 +1444,30 @@ static bool need_update(int cpu) return false; } +void quiet_vmstat(void) +{ + if (system_state != SYSTEM_RUNNING) + return; + + /* + * If we are already in hands of the shepherd then there + * is nothing for us to do here. + */ + if (cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off)) + return; + + if (!need_update(smp_processor_id())) + return; + + /* + * Just refresh counters and do not care about the pending delayed + * vmstat_update. It doesn't fire that often to matter and canceling + * it would be too expensive from this path. + * vmstat_shepherd will take care about that for us. + */ + refresh_cpu_vm_stats(false); +} + /* * Shepherd worker thread that checks the @@ -1468,18 +1485,25 @@ static void vmstat_shepherd(struct work_struct *w) get_online_cpus(); /* Check processors whose vmstat worker threads have been disabled */ - for_each_cpu(cpu, cpu_stat_off) - if (need_update(cpu) && - cpumask_test_and_clear_cpu(cpu, cpu_stat_off)) - - queue_delayed_work_on(cpu, vmstat_wq, - &per_cpu(vmstat_work, cpu), 0); + for_each_cpu(cpu, cpu_stat_off) { + struct delayed_work *dw = &per_cpu(vmstat_work, cpu); + if (need_update(cpu)) { + if (cpumask_test_and_clear_cpu(cpu, cpu_stat_off)) + queue_delayed_work_on(cpu, vmstat_wq, dw, 0); + } else { + /* + * Cancel the work if quiet_vmstat has put this + * cpu on cpu_stat_off because the work item might + * be still scheduled + */ + cancel_delayed_work(dw); + } + } put_online_cpus(); schedule_delayed_work(&shepherd, round_jiffies_relative(sysctl_stat_interval)); - } static void __init start_shepherd_timer(void)