From patchwork Wed Feb 19 13:14:54 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juri Lelli X-Patchwork-Id: 24955 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ig0-f200.google.com (mail-ig0-f200.google.com [209.85.213.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id D51E520143 for ; Wed, 19 Feb 2014 13:14:54 +0000 (UTC) Received: by mail-ig0-f200.google.com with SMTP id hl1sf2197580igb.3 for ; Wed, 19 Feb 2014 05:14:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:sender:precedence :list-id:x-original-sender:x-original-authentication-results :mailing-list:list-post:list-help:list-archive:list-unsubscribe :content-type:content-transfer-encoding; bh=yy8HY0G1Cik7qRXPj/LrFPyHQIhUPYZ8PrgOyhB77Hs=; b=c7lN0lHQHkSQr2dY3X7mT5g9bA1J6HcPVIxrBuAt+w5TJChMlgL21rO4gG+TRI++VP deJUOjXf7N62zqW9wGaskcK8kzzh1hGqGpaPfF8HeCaQtuMIohxCwxl9ECVKhJGhCGKa d5NMPe29cL3q7F5X9UGI/ESIvfY7Vr/143BvuO9RaHuOGcqyu9aJIUQs7MgDnXu1saDZ k6+F7vYPYS4ZJ5Q5sYQCa7aZdbkvicGOOZSDjUwjEKYiPOhTzwfSQ1+fV0Rz6kfDb5Fx kBAkp4ASmgcc/N7We3geEqHxX0E+BoPngJSdZn0PAOi0vM22MBWKJ55e4dz0IhIYpmgp rIww== X-Gm-Message-State: ALoCoQmjM7L65YmL7jIf6lxcdBE6qROqeO3sjC+rFYR3T531GQidU5T5rKaVItWKAfLzH08wWiyF X-Received: by 10.50.78.231 with SMTP id e7mr579567igx.5.1392815694335; Wed, 19 Feb 2014 05:14:54 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.26.40 with SMTP id 37ls80898qgu.46.gmail; Wed, 19 Feb 2014 05:14:54 -0800 (PST) X-Received: by 10.52.183.106 with SMTP id el10mr347000vdc.73.1392815694157; Wed, 19 Feb 2014 05:14:54 -0800 (PST) Received: from mail-ve0-x231.google.com (mail-ve0-x231.google.com [2607:f8b0:400c:c01::231]) by mx.google.com with ESMTPS id os4si51933vcb.97.2014.02.19.05.14.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 19 Feb 2014 05:14:54 -0800 (PST) Received-SPF: neutral (google.com: 2607:f8b0:400c:c01::231 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=2607:f8b0:400c:c01::231; Received: by mail-ve0-f177.google.com with SMTP id jz11so333858veb.8 for ; Wed, 19 Feb 2014 05:14:54 -0800 (PST) X-Received: by 10.220.81.71 with SMTP id w7mr401142vck.71.1392815694046; Wed, 19 Feb 2014 05:14:54 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.174.196 with SMTP id u4csp296306vcz; Wed, 19 Feb 2014 05:14:53 -0800 (PST) X-Received: by 10.68.201.163 with SMTP id kb3mr2119014pbc.168.1392815692853; Wed, 19 Feb 2014 05:14:52 -0800 (PST) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id zk9si160581pac.231.2014.02.19.05.14.51; Wed, 19 Feb 2014 05:14:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753598AbaBSNOp (ORCPT + 27 others); Wed, 19 Feb 2014 08:14:45 -0500 Received: from mail-ee0-f51.google.com ([74.125.83.51]:38235 "EHLO mail-ee0-f51.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753094AbaBSNOn (ORCPT ); Wed, 19 Feb 2014 08:14:43 -0500 Received: by mail-ee0-f51.google.com with SMTP id b57so219901eek.38 for ; Wed, 19 Feb 2014 05:14:42 -0800 (PST) X-Received: by 10.15.76.1 with SMTP id m1mr39938648eey.36.1392815682595; Wed, 19 Feb 2014 05:14:42 -0800 (PST) Received: from [10.30.3.54] (nat-cataldo.sssup.it. [193.205.81.5]) by mx.google.com with ESMTPSA id g1sm640094eet.6.2014.02.19.05.14.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 19 Feb 2014 05:14:41 -0800 (PST) Message-ID: <5304AE4E.6030208@gmail.com> Date: Wed, 19 Feb 2014 14:14:54 +0100 From: Juri Lelli User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Peter Zijlstra , Steven Rostedt CC: LKML , Linus Torvalds , Ingo Molnar , Thomas Gleixner , Andrew Morton Subject: Re: [PATCH v3] sched/deadline: Fix bad accounting of nr_running References: <20140214235946.60a89b65@gandalf.local.home> <53022F2D.8040301@gmail.com> <20140218215012.209059c0@gandalf.local.home> <20140219084618.GF27965@twins.programming.kicks-ass.net> <53048849.3000601@gmail.com> In-Reply-To: <53048849.3000601@gmail.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Original-Sender: juri.lelli@gmail.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 2607:f8b0:400c:c01::231 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=neutral (bad format) header.i=@gmail.com; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , On 02/19/2014 11:32 AM, Juri Lelli wrote: > On 02/19/2014 09:46 AM, Peter Zijlstra wrote: >> On Tue, Feb 18, 2014 at 09:50:12PM -0500, Steven Rostedt wrote: >>> >>>> Rationale for this odd behavior is that, when a task is throttled, it >>>> is removed only from the dl_rq, but we keep it on_rq (as this is not >>>> a "full dequeue", that is the task is not actually sleeping). But, it >>>> is also true that, while throttled a task behaves like it is sleeping >>>> (e.g., its timer will fire on a new CPU if the old one is dead). So, >>>> Steven's fix sounds also semantically correct. >>> >>> Actually, it seems that I was hitting it again, but this time getting a >>> negative number. OK, after looking at the code a bit more, I think we >>> should update the runqueue nr_running only when the task is officially >>> enqueued and dequeued, and all accounting within, will not touch that >>> number. > > This is a different way to get the same result (mildly tested on my box): > > --- > kernel/sched/deadline.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c > index 0dd5e09..675dad3 100644 > --- a/kernel/sched/deadline.c > +++ b/kernel/sched/deadline.c > @@ -837,7 +837,8 @@ static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags) > if (!task_current(rq, p) && p->nr_cpus_allowed > 1) > enqueue_pushable_dl_task(rq, p); > > - inc_nr_running(rq); > + if (!(flags & ENQUEUE_REPLENISH)) > + inc_nr_running(rq); > } > > static void __dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags) > -- > > We touch nr_running only when we don't enqueue back as a consequence > of a replenishment. > >> >> But if the task is throttled it should still very much decrement the >> number. There's places that very much rely on nr_running be exactly the >> number of runnable tasks. >> > > This is a different thing, and V2 seemed to implement this behavior > (that's why I said it looked semantically correct). > So, both my last approach and Steven's V2 were causing nr_running to become negative, as they double decrement it when dequeuing a task that also exceeded its budget. What follows seems to solve the issue, and correcly account for throttled tasks as !nr_running. --- kernel/sched/deadline.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0dd5e09..b819577 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -717,6 +717,7 @@ void inc_dl_tasks(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq) WARN_ON(!dl_prio(prio)); dl_rq->dl_nr_running++; + inc_nr_running(rq_of_dl_rq(dl_rq)); inc_dl_deadline(dl_rq, deadline); inc_dl_migration(dl_se, dl_rq); @@ -730,6 +731,7 @@ void dec_dl_tasks(struct sched_dl_entity *dl_se, struct dl_rq *dl_rq) WARN_ON(!dl_prio(prio)); WARN_ON(!dl_rq->dl_nr_running); dl_rq->dl_nr_running--; + dec_nr_running(rq_of_dl_rq(dl_rq)); dec_dl_deadline(dl_rq, dl_se->deadline); dec_dl_migration(dl_se, dl_rq); @@ -836,8 +838,6 @@ static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags) if (!task_current(rq, p) && p->nr_cpus_allowed > 1) enqueue_pushable_dl_task(rq, p); - - inc_nr_running(rq); } static void __dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags) @@ -850,8 +850,6 @@ static void dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags) { update_curr_dl(rq); __dequeue_task_dl(rq, p, flags); - - dec_nr_running(rq); } /*