From patchwork Mon Apr 20 08:22:48 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xunlei Pang X-Patchwork-Id: 47335 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f70.google.com (mail-la0-f70.google.com [209.85.215.70]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 2C2A820553 for ; Mon, 20 Apr 2015 08:25:54 +0000 (UTC) Received: by layy10 with SMTP id y10sf37082200lay.0 for ; Mon, 20 Apr 2015 01:25:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=E2hi4Dk5wlRXy0MnduWCG2tSuPWE41O/HnzqNH+0GSo=; b=I6X4ZR3YHViU/2LWnwE480AOSo6brHGfrjiuJ6xW095z8U+NShSLFciW8GYr2hRhfQ 2m2km3N/K8wnC7Eq0KYgKFsMBG1FqMsNGjY63qUHTbM6YU2MbmL95NEfiSjQYOSxfl7Y yj8wTFTx9HCi8v+HQzAOoT/0E/09jQthYoA6y+gANmhumeSQwqW86OcpvYxtXlrEjVlH 3NhHv0sTQcYoPYpTzk2cYOl37Zq9KkNYk2yMkYqGuiNkkDqLE302iB5vHuIngLvkZVCW u7qKJSCA0HmC8B970+PJ5cMEtnwGBuGMegMhJPC8JaJDZANt0lUqeRgELbES9P8bEAIo dApQ== X-Gm-Message-State: ALoCoQnxOwMuioJCHE/obXgkorBQlpTtsZZHWuQdMsySw9rrIIx9nK+QXaWrDMFYSE30/aHeTcHF X-Received: by 10.112.142.170 with SMTP id rx10mr6620147lbb.12.1429518352794; Mon, 20 Apr 2015 01:25:52 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.29.66 with SMTP id i2ls797533lah.19.gmail; Mon, 20 Apr 2015 01:25:52 -0700 (PDT) X-Received: by 10.152.121.42 with SMTP id lh10mr15023048lab.0.1429518352621; Mon, 20 Apr 2015 01:25:52 -0700 (PDT) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com. [2a00:1450:4010:c03::22d]) by mx.google.com with ESMTPS id db3si14446109lad.150.2015.04.20.01.25.52 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Apr 2015 01:25:52 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c03::22d as permitted sender) client-ip=2a00:1450:4010:c03::22d; Received: by lagv1 with SMTP id v1so121081531lag.3 for ; Mon, 20 Apr 2015 01:25:52 -0700 (PDT) X-Received: by 10.112.198.74 with SMTP id ja10mr2841392lbc.19.1429518352524; Mon, 20 Apr 2015 01:25:52 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.67.65 with SMTP id l1csp1297282lbt; Mon, 20 Apr 2015 01:25:51 -0700 (PDT) X-Received: by 10.66.164.98 with SMTP id yp2mr25770849pab.114.1429518350673; Mon, 20 Apr 2015 01:25:50 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id ij3si14300664pbb.30.2015.04.20.01.25.49; Mon, 20 Apr 2015 01:25:50 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754743AbbDTIZq (ORCPT + 26 others); Mon, 20 Apr 2015 04:25:46 -0400 Received: from m50-110.126.com ([123.125.50.110]:40646 "EHLO m50-110.126.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754633AbbDTIZJ (ORCPT ); Mon, 20 Apr 2015 04:25:09 -0400 Received: from localhost.localdomain (unknown [210.21.223.3]) by smtp4 (Coremail) with SMTP id jdKowAD3__uLtzRVzV2uAQ--.4394S4; Mon, 20 Apr 2015 16:23:51 +0800 (CST) From: Xunlei Pang To: linux-kernel@vger.kernel.org Cc: Peter Zijlstra , Steven Rostedt , Juri Lelli , Xunlei Pang Subject: [PATCH v6 3/3] sched/rt: Check to push the task when changing its affinity Date: Mon, 20 Apr 2015 16:22:48 +0800 Message-Id: <1429518168-7965-3-git-send-email-xlpang@126.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1429518168-7965-1-git-send-email-xlpang@126.com> References: <1429518168-7965-1-git-send-email-xlpang@126.com> X-CM-TRANSID: jdKowAD3__uLtzRVzV2uAQ--.4394S4 X-Coremail-Antispam: 1Uf129KBjvJXoWxXFy3Jr13XrWrJFyDuFW7Arb_yoWrCF17pa 1vk390gr4DJa1SgF1fZw4DZr45K3sav34rJrnxtw1FkFs8tr4Fv3W5tF1ayF93ur1Y9F4a qr4Dtr42gF1jva7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07bUSoXUUUUU= X-Originating-IP: [210.21.223.3] X-CM-SenderInfo: p0ost0bj6rjloofrz/1tbiJxbhv01sBUvfIAABs+ Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Original-Sender: xlpang@126.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c03::22d as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=neutral (body hash did not verify) header.i=@; dmarc=fail (p=NONE dis=NONE) header.from=126.com Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Xunlei Pang We may suffer from extra rt overload rq due to the affinity, so when the affinity of any runnable rt task is changed, we should check to trigger balancing, otherwise it will cause some unnecessary delayed real-time response. Unfortunately, current RT global scheduler does nothing about this. For example: a 2-cpu system with two runnable FIFO tasks(same rt_priority) bound on CPU0, let's name them rt1(running) and rt2(runnable) respectively; CPU1 has no RTs. Then, someone sets the affinity of rt2 to 0x3(i.e. CPU0 and CPU1), but after this, rt2 still can't be scheduled until rt1 enters schedule(), this definitely causes some/big response latency for rt2. So, when doing set_cpus_allowed_rt(), if detecting such cases, check to trigger a push behaviour. Signed-off-by: Xunlei Pang --- kernel/sched/rt.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 71 insertions(+), 10 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 8679eff..846b59c 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1460,10 +1460,9 @@ static struct sched_rt_entity *pick_next_rt_entity(struct rq *rq, return next; } -static struct task_struct *_pick_next_task_rt(struct rq *rq) +static struct task_struct *peek_next_task_rt(struct rq *rq) { struct sched_rt_entity *rt_se; - struct task_struct *p; struct rt_rq *rt_rq = &rq->rt; do { @@ -1472,7 +1471,14 @@ static struct task_struct *_pick_next_task_rt(struct rq *rq) rt_rq = group_rt_rq(rt_se); } while (rt_rq); - p = rt_task_of(rt_se); + return rt_task_of(rt_se); +} + +static inline struct task_struct *_pick_next_task_rt(struct rq *rq) +{ + struct task_struct *p; + + p = peek_next_task_rt(rq); p->se.exec_start = rq_clock_task(rq); return p; @@ -2096,28 +2102,77 @@ static void set_cpus_allowed_rt(struct task_struct *p, const struct cpumask *new_mask) { struct rq *rq; - int weight; + int old_weight, new_weight; + int preempt_push = 0, direct_push = 0; BUG_ON(!rt_task(p)); if (!task_on_rq_queued(p)) return; - weight = cpumask_weight(new_mask); + old_weight = p->nr_cpus_allowed; + new_weight = cpumask_weight(new_mask); + rq = task_rq(p); + + if (new_weight > 1 && + rt_task(rq->curr) && + rq->rt.rt_nr_total > 1 && + !test_tsk_need_resched(rq->curr)) { + /* + * We own p->pi_lock and rq->lock. rq->lock might + * get released when doing direct pushing, however + * p->pi_lock is always held, so it's safe to assign + * new_mask and new_weight to p below. + */ + if (!task_running(rq, p)) { + cpumask_copy(&p->cpus_allowed, new_mask); + p->nr_cpus_allowed = new_weight; + direct_push = 1; + } else if (cpumask_test_cpu(task_cpu(p), new_mask)) { + struct task_struct *next; + + cpumask_copy(&p->cpus_allowed, new_mask); + p->nr_cpus_allowed = new_weight; + if (!cpupri_find(&rq->rd->cpupri, p, NULL)) + goto update; + + /* + * At this point, current task gets migratable most + * likely due to the change of its affinity, let's + * figure out if we can migrate it. + * + * Can we find any task with the same priority as + * current? To accomplish this, firstly we requeue + * current to the tail and peek next, then restore + * current to the head. + */ + requeue_task_rt(rq, p, 0); + next = peek_next_task_rt(rq); + requeue_task_rt(rq, p, 1); + if (next != p && next->prio == p->prio) { + /* + * Target found, so let's reschedule to try + * and push current away. + */ + requeue_task_rt(rq, next, 1); + preempt_push = 1; + } + } + } + +update: /* * Only update if the process changes its state from whether it * can migrate or not. */ - if ((p->nr_cpus_allowed > 1) == (weight > 1)) - return; - - rq = task_rq(p); + if ((old_weight > 1) == (new_weight > 1)) + goto out; /* * The process used to be able to migrate OR it can now migrate */ - if (weight <= 1) { + if (new_weight <= 1) { if (!task_current(rq, p)) dequeue_pushable_task(rq, p); BUG_ON(!rq->rt.rt_nr_migratory); @@ -2129,6 +2184,12 @@ static void set_cpus_allowed_rt(struct task_struct *p, } update_rt_migration(&rq->rt); + +out: + if (direct_push) + push_rt_tasks(rq); + else if (preempt_push) + resched_curr(rq); } /* Assumes rq->lock is held */