From patchwork Sun Mar 10 18:11:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160026 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11089884jad; Sun, 10 Mar 2019 11:12:14 -0700 (PDT) X-Google-Smtp-Source: APXvYqyz7FGwjJVmUjlePc2ZxE4LCm2hshMA755lh/cyWz7fHjQJUb5heh3GT3Rhb+5CYsq0L8d0 X-Received: by 2002:a17:902:9a98:: with SMTP id w24mr29326624plp.247.1552241534641; Sun, 10 Mar 2019 11:12:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241534; cv=none; d=google.com; s=arc-20160816; b=hkQJyhQhnUY99fjlzGuykKYxODkCbjwftLE8t3GCU1cpVoVKqG4VWhuG5vf50h28UJ xeyz7EI4i2ItgGtNyLcrqjSIcsyWR6SibXIBP+Vi7eF5Ec6z58gH4Fr0aLiVhtKgbkOq WYZJ15pLo1o6gH8vHwZ8v1NAfG/mRhuhBCpdHhdj8+rgnR0KZxPtL7sZ6Flu+JfWm6l/ L2b2ErwN2yDF6rF5L8TpWtcIKZhenagq/HRWu4fwzvClvvo+xEPD1qWxF6GHH1XzQ/DX 8eddxaael75RsFzlXVV866RHyTxvwBZa/CsV2dPQn5lb20C4VUemqU4FPq/Y6r49ZoLn Y2fQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SY00KoFWeTwZXwxCAxRn3goBCIvGzbExPxId3YVmDAI=; b=bkQiMI79P78QOMhuTHl41mUlqc3EEr/hZjSL8JnC6QSFuhscgzM9emCLETZffy3kkw A9pedt3s10qTLlDLp0L7txwgy7MRTMSm9i15lEzPW1ls5XbWqG0tj4rPArjAdpv2drzE 4BC/N8rYnXZzcR+N7+Bl5ea+exnAbY5YWopl4y+p8oj9lz04SLmq9l32pxuzj369lf1e 9e4ZWaFvLsbYQIZ/zTWcfAj8m67rfXJFHA1nEUPsN0qu2E6mzaU2t/uuEkt6MwKhj7tX ztxnt/V4kNiLiZHmIR1Lgus119S6y+DxPGt2q28U8ufveMis7moJn69W2VO5QxI7l01s Hoxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="L/K3M0uY"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z23si3129436pgu.132.2019.03.10.11.12.14; Sun, 10 Mar 2019 11:12:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="L/K3M0uY"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726881AbfCJSMM (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:12 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:53175 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726865AbfCJSMJ (ORCPT ); Sun, 10 Mar 2019 14:12:09 -0400 Received: by mail-wm1-f65.google.com with SMTP id f65so2213238wma.2 for ; Sun, 10 Mar 2019 11:12:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SY00KoFWeTwZXwxCAxRn3goBCIvGzbExPxId3YVmDAI=; b=L/K3M0uYUBbqVjojJG+/UOlGrYywm685K9LWp12a8TuqJIqoSV9LLER2Wga7GgCP7I iLpBKDlCtw3K61Com+z0O7IohUEHMADZL7UXzXZBeB1TeJrdg+Tka1GCJtth0+Jfyrkd Xx2hwFjuBCEcpmNp25XaJYpqayHDsPz0uFYJjnYXz+edXJ6OPiRS4jMUxzjnLyzCJf38 qPygtOV/g1r6s4Zb74dgntggVsXMO56V4XgIM9C9UNW3AtvMiA3Sx+yFh4aZ3mXL84a3 +KKFqe8g/6za9sI5EJdlC9fp5WCjhN+1GM9ZU4lA1dXghb5JW4Prwru/RAswNQZMISDB 7Mpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SY00KoFWeTwZXwxCAxRn3goBCIvGzbExPxId3YVmDAI=; b=YRB1AoYzgySx0y6IHYkMYWQz/EcKoFmrX7lPiARqz0ON9rMLrC/0Zn+YPL0RZwOV5t f0odUvZjlISV+3xyJx9Dza0t4I+JkvWpPcA92PzwKehsWFLY354ovHy7/WvcUt/8awhV mICgeuafXQHm7rAbnLMF2TKDY2p5Xaqds9tVTNjYNmJEo0C5T6qzW3ucriaUBWMspLZO d/cKh52+ZQMZUcxzyFYpICjGjwJm6vtzmZZGZUxq6RUDjd7LE3Mfteb3Hs0lDfl/TsWB 5bz8dACFHRfxdavypPt/cVuqi8mhCVbOLTZKgHpdYW3arR3nyqDTO1FBOV+Jt4R90qOW byWQ== X-Gm-Message-State: APjAAAWkjOoAwhkRodsfaQvD/sF4/aRqZjoLySmJGkPI26sMO6puxp+B 8if5p2Ao/hKJPdLCnnuvT1ek+A== X-Received: by 2002:a7b:c1c1:: with SMTP id a1mr8875305wmj.77.1552241527459; Sun, 10 Mar 2019 11:12:07 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:06 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 1/9] block, bfq: increase idling for weight-raised queues Date: Sun, 10 Mar 2019 19:11:29 +0100 Message-Id: <20190310181137.2604-2-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If a sync bfq_queue has a higher weight than some other queue, and remains temporarily empty while in service, then, to preserve the bandwidth share of the queue, it is necessary to plug I/O dispatching until a new request arrives for the queue. In addition, a timeout needs to be set, to avoid waiting for ever if the process associated with the queue has actually finished its I/O. Even with the above timeout, the device is however not fed with new I/O for a while, if the process has finished its I/O. If this happens often, then throughput drops and latencies grow. For this reason, the timeout is kept rather low: 8 ms is the current default. Unfortunately, such a low value may cause, on the opposite end, a violation of bandwidth guarantees for a process that happens to issue new I/O too late. The higher the system load, the higher the probability that this happens to some process. This is a problem in scenarios where service guarantees matter more than throughput. One important case are weight-raised queues, which need to be granted a very high fraction of the bandwidth. To address this issue, this commit lower-bounds the plugging timeout for weight-raised queues to 20 ms. This simple change provides relevant benefits. For example, on a PLEXTOR PX-256M5S, with which gnome-terminal starts in 0.6 seconds if there is no other I/O in progress, the same applications starts in - 0.8 seconds, instead of 1.2 seconds, if ten files are being read sequentially in parallel - 1 second, instead of 2 seconds, if, in parallel, five files are being read sequentially, and five more files are being written sequentially Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 2 ++ 1 file changed, 2 insertions(+) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 4c592496a16a..eb658de3cc40 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2545,6 +2545,8 @@ static void bfq_arm_slice_timer(struct bfq_data *bfqd) if (BFQQ_SEEKY(bfqq) && bfqq->wr_coeff == 1 && bfq_symmetric_scenario(bfqd)) sl = min_t(u64, sl, BFQ_MIN_TT); + else if (bfqq->wr_coeff > 1) + sl = max_t(u32, sl, 20ULL * NSEC_PER_MSEC); bfqd->last_idling_start = ktime_get(); hrtimer_start(&bfqd->idle_slice_timer, ns_to_ktime(sl), From patchwork Sun Mar 10 18:11:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160027 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11089932jad; Sun, 10 Mar 2019 11:12:18 -0700 (PDT) X-Google-Smtp-Source: APXvYqx0Lsu2JiR0t2dvLqSwUIv/rXoGpwgd2oDI2sz6faa5IS0l20esoUUbcVjoRF8b9RKbsfrW X-Received: by 2002:a62:2b88:: with SMTP id r130mr29090561pfr.93.1552241537912; Sun, 10 Mar 2019 11:12:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241537; cv=none; d=google.com; s=arc-20160816; b=cCOv/xPN+y70lsI0AeoC+D1hepeKMuwVfv8lwMKTgVT3mdHj0SI8Mwx/9/eMgSxC0B cIH+0NOFqZ5KTJccCjCkDskqLjCOiUy8V2GnwVHVHGw85NlFos/8/K/FsRhVXhb0NtWN o4EnsDhEsTj/xHNE1r6XyrXXc/nMEAAcYIgTH4rcKFsBCedgY9vFG+oMpHj/ijt3ykxR +AhhcwIY91GMp2PQVZDGSlBPKisTYsZe2JN8lAFkOJBbCW4KvYeZrvB5EDIqcBGUFh4d pR0vgxDyeSup6SPBsVqsvZByw4BgJuXbwkd9KXYdd4s6a12EiEgM5bO/JN9RxkZEBpoG mVsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0NAvrfo3Asn/Q7s9KDBK/T3qV+c+kR0mqhtwlAgJI8Y=; b=A+PBzO7zwvmpWwnJ95FoKSKuW18Jn7txSB0fAkVUaULLj+GEsGzVPZFPsQHtCYVNjC AQyhItakhj1LjWBXziwNHxtcgWPWF6mPS9ryXB3tOca4a8WbCLdZgp5XICkwswDLhfMC X2HtDNRVSZWpOJ9BM8GbYCjArM/g64k5BOR/aDRmA4NEBegJDDvpu5Rv4nJWuhiouLVz 4QAt94AGal7LYnzPlEkKVEohervsvE52KCuWwXDnShpMzBpSXvqfU545Y3AuIAjtvpLt +9Nu65CGL9JCQHYqNDKsGm+iTU0NF6a5qGrNHQRat1FhRcpn3ly9P4tqv1CJGeX5FJGG b4bA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=c4hx76em; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z23si3129436pgu.132.2019.03.10.11.12.17; Sun, 10 Mar 2019 11:12:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=c4hx76em; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726892AbfCJSMQ (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:16 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:34857 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726872AbfCJSMN (ORCPT ); Sun, 10 Mar 2019 14:12:13 -0400 Received: by mail-wm1-f68.google.com with SMTP id y15so2187421wma.0 for ; Sun, 10 Mar 2019 11:12:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0NAvrfo3Asn/Q7s9KDBK/T3qV+c+kR0mqhtwlAgJI8Y=; b=c4hx76em+IHitbZH37Q1sbPLDVKnk172i/6gWuTFcdq9WW7nC/m+XYZ3Fi+37+LgJ5 Ke1Balzq9cTOFXFoanFlpAsDnycjzqLnHsCDOqh/MamYWp+Zr/EgfzH2emp79mgoLk/C JFRqbik6lK6b3KLIQ3HJtozi3nRaI9D8MBm+y9Oh0zwPGR+xjr8yZLDtNCdMIPOVFL3H F9nIraTLHOiKQrydYT9+ZLXr1hdtpMbNxVwNWgeBWAzybhkOG6JaxTW7GjZFjGCCHheF nKzmDgyE/2ikOunY/N17VpuWaKgaHmDH4AaTQ6hN3NmKzukI+/R9bu1Nv+Jk52cxrKJM AQew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0NAvrfo3Asn/Q7s9KDBK/T3qV+c+kR0mqhtwlAgJI8Y=; b=VygYScfKrCxcPxZrpn/OhL3xWO3MWmh2HlSmqY/ETl+ZYjYMvgkJM+fdkYk+A0nd2+ 1ReWA/y05psHvq/rrDIMWGYw1AGlrxVIVfXVrvLh4+sf4vbPW3xUlbXeuTkTJodMrLgT 9UVGACx5vu+J6Hj5nKq0IvaWEkfoRzG9OvGPpSYffzOY/6WUXm40OYDNGtQ7dz5L9nFA bMcsb68u+w34fjsxemR3/KJ2RCxuxVnb9u2SSb0y/faYFkGA2Qfl1Hbs+ZIBWVy8cryf lWUcoJhi10vKK5MWV+TdTwms+zdVmaNIB9pFE/T0N2MSIh2ZYpZrBed4gLEOnWpPmHyG mUCA== X-Gm-Message-State: APjAAAVwJeW0Bi8GlcyrCqMplbue9DUTbaNeCHskpQUqMidiJWbD7ymh kJL9FVAoevd5FD4Ey8BCulGsJA== X-Received: by 2002:a1c:4c08:: with SMTP id z8mr13615369wmf.99.1552241529534; Sun, 10 Mar 2019 11:12:09 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:08 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 2/9] block, bfq: do not idle for lowest-weight queues Date: Sun, 10 Mar 2019 19:11:30 +0100 Message-Id: <20190310181137.2604-3-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In most cases, it is detrimental for throughput to plug I/O dispatch when the in-service bfq_queue becomes temporarily empty (plugging is performed to wait for the possible arrival, soon, of new I/O from the in-service queue). There is however a case where plugging is needed for service guarantees. If a bfq_queue, say Q, has a higher weight than some other active bfq_queue, and is sync, i.e., contains sync I/O, then, to guarantee that Q does receive a higher share of the throughput than other lower-weight queues, it is necessary to plug I/O dispatch when Q remains temporarily empty while being served. For this reason, BFQ performs I/O plugging when some active bfq_queue has a higher weight than some other active bfq_queue. But this is unnecessarily overkill. In fact, if the in-service bfq_queue actually has a weight lower than or equal to the other queues, then the queue *must not* be guaranteed a higher share of the throughput than the other queues. So, not plugging I/O cannot cause any harm to the queue. And can boost throughput. Taking advantage of this fact, this commit does not plug I/O for sync bfq_queues with a weight lower than or equal to the weights of the other queues. Here is an example of the resulting throughput boost with the dbench workload, which is particularly nasty for BFQ. With the dbench test in the Phoronix suite, BFQ reaches its lowest total throughput with 6 clients on a filesystem with journaling, in case the journaling daemon has a higher weight than normal processes. Before this commit, the total throughput was ~80 MB/sec on a PLEXTOR PX-256M5, after this commit it is ~100 MB/sec. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 204 +++++++++++++++++++++++++------------------- block/bfq-iosched.h | 6 +- block/bfq-wf2q.c | 2 +- 3 files changed, 118 insertions(+), 94 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index eb658de3cc40..2be504f25b09 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -629,12 +629,19 @@ void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) } /* - * The following function returns true if every queue must receive the - * same share of the throughput (this condition is used when deciding - * whether idling may be disabled, see the comments in the function - * bfq_better_to_idle()). + * The following function returns false either if every active queue + * must receive the same share of the throughput (symmetric scenario), + * or, as a special case, if bfqq must receive a share of the + * throughput lower than or equal to the share that every other active + * queue must receive. If bfqq does sync I/O, then these are the only + * two cases where bfqq happens to be guaranteed its share of the + * throughput even if I/O dispatching is not plugged when bfqq remains + * temporarily empty (for more details, see the comments in the + * function bfq_better_to_idle()). For this reason, the return value + * of this function is used to check whether I/O-dispatch plugging can + * be avoided. * - * Such a scenario occurs when: + * The above first case (symmetric scenario) occurs when: * 1) all active queues have the same weight, * 2) all active queues belong to the same I/O-priority class, * 3) all active groups at the same level in the groups tree have the same @@ -654,30 +661,36 @@ void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) * support or the cgroups interface are not enabled, thus no state * needs to be maintained in this case. */ -static bool bfq_symmetric_scenario(struct bfq_data *bfqd) +static bool bfq_asymmetric_scenario(struct bfq_data *bfqd, + struct bfq_queue *bfqq) { + bool smallest_weight = bfqq && + bfqq->weight_counter && + bfqq->weight_counter == + container_of( + rb_first_cached(&bfqd->queue_weights_tree), + struct bfq_weight_counter, + weights_node); + /* * For queue weights to differ, queue_weights_tree must contain * at least two nodes. */ - bool varied_queue_weights = !RB_EMPTY_ROOT(&bfqd->queue_weights_tree) && - (bfqd->queue_weights_tree.rb_node->rb_left || - bfqd->queue_weights_tree.rb_node->rb_right); + bool varied_queue_weights = !smallest_weight && + !RB_EMPTY_ROOT(&bfqd->queue_weights_tree.rb_root) && + (bfqd->queue_weights_tree.rb_root.rb_node->rb_left || + bfqd->queue_weights_tree.rb_root.rb_node->rb_right); bool multiple_classes_busy = (bfqd->busy_queues[0] && bfqd->busy_queues[1]) || (bfqd->busy_queues[0] && bfqd->busy_queues[2]) || (bfqd->busy_queues[1] && bfqd->busy_queues[2]); - /* - * For queue weights to differ, queue_weights_tree must contain - * at least two nodes. - */ - return !(varied_queue_weights || multiple_classes_busy + return varied_queue_weights || multiple_classes_busy #ifdef BFQ_GROUP_IOSCHED_ENABLED || bfqd->num_groups_with_pending_reqs > 0 #endif - ); + ; } /* @@ -694,10 +707,11 @@ static bool bfq_symmetric_scenario(struct bfq_data *bfqd) * should be low too. */ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, - struct rb_root *root) + struct rb_root_cached *root) { struct bfq_entity *entity = &bfqq->entity; - struct rb_node **new = &(root->rb_node), *parent = NULL; + struct rb_node **new = &(root->rb_root.rb_node), *parent = NULL; + bool leftmost = true; /* * Do not insert if the queue is already associated with a @@ -726,8 +740,10 @@ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, } if (entity->weight < __counter->weight) new = &((*new)->rb_left); - else + else { new = &((*new)->rb_right); + leftmost = false; + } } bfqq->weight_counter = kzalloc(sizeof(struct bfq_weight_counter), @@ -736,7 +752,7 @@ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, /* * In the unlucky event of an allocation failure, we just * exit. This will cause the weight of queue to not be - * considered in bfq_symmetric_scenario, which, in its turn, + * considered in bfq_asymmetric_scenario, which, in its turn, * causes the scenario to be deemed wrongly symmetric in case * bfqq's weight would have been the only weight making the * scenario asymmetric. On the bright side, no unbalance will @@ -750,7 +766,8 @@ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, bfqq->weight_counter->weight = entity->weight; rb_link_node(&bfqq->weight_counter->weights_node, parent, new); - rb_insert_color(&bfqq->weight_counter->weights_node, root); + rb_insert_color_cached(&bfqq->weight_counter->weights_node, root, + leftmost); inc_counter: bfqq->weight_counter->num_active++; @@ -765,7 +782,7 @@ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, */ void __bfq_weights_tree_remove(struct bfq_data *bfqd, struct bfq_queue *bfqq, - struct rb_root *root) + struct rb_root_cached *root) { if (!bfqq->weight_counter) return; @@ -774,7 +791,7 @@ void __bfq_weights_tree_remove(struct bfq_data *bfqd, if (bfqq->weight_counter->num_active > 0) goto reset_entity_pointer; - rb_erase(&bfqq->weight_counter->weights_node, root); + rb_erase_cached(&bfqq->weight_counter->weights_node, root); kfree(bfqq->weight_counter); reset_entity_pointer: @@ -889,7 +906,7 @@ static unsigned long bfq_serv_to_charge(struct request *rq, struct bfq_queue *bfqq) { if (bfq_bfqq_sync(bfqq) || bfqq->wr_coeff > 1 || - !bfq_symmetric_scenario(bfqq->bfqd)) + bfq_asymmetric_scenario(bfqq->bfqd, bfqq)) return blk_rq_sectors(rq); return blk_rq_sectors(rq) * bfq_async_charge_factor; @@ -2543,7 +2560,7 @@ static void bfq_arm_slice_timer(struct bfq_data *bfqd) * queue). */ if (BFQQ_SEEKY(bfqq) && bfqq->wr_coeff == 1 && - bfq_symmetric_scenario(bfqd)) + !bfq_asymmetric_scenario(bfqd, bfqq)) sl = min_t(u64, sl, BFQ_MIN_TT); else if (bfqq->wr_coeff > 1) sl = max_t(u32, sl, 20ULL * NSEC_PER_MSEC); @@ -3500,8 +3517,9 @@ static bool idling_boosts_thr_without_issues(struct bfq_data *bfqd, } /* - * There is a case where idling must be performed not for - * throughput concerns, but to preserve service guarantees. + * There is a case where idling does not have to be performed for + * throughput concerns, but to preserve the throughput share of + * the process associated with bfqq. * * To introduce this case, we can note that allowing the drive * to enqueue more than one request at a time, and hence @@ -3517,77 +3535,83 @@ static bool idling_boosts_thr_without_issues(struct bfq_data *bfqd, * concern about per-process throughput distribution, and * makes its decisions only on a per-request basis. Therefore, * the service distribution enforced by the drive's internal - * scheduler is likely to coincide with the desired - * device-throughput distribution only in a completely - * symmetric scenario where: - * (i) each of these processes must get the same throughput as - * the others; - * (ii) the I/O of each process has the same properties, in - * terms of locality (sequential or random), direction - * (reads or writes), request sizes, greediness - * (from I/O-bound to sporadic), and so on. - * In fact, in such a scenario, the drive tends to treat - * the requests of each of these processes in about the same - * way as the requests of the others, and thus to provide - * each of these processes with about the same throughput - * (which is exactly the desired throughput distribution). In - * contrast, in any asymmetric scenario, device idling is - * certainly needed to guarantee that bfqq receives its - * assigned fraction of the device throughput (see [1] for - * details). - * The problem is that idling may significantly reduce - * throughput with certain combinations of types of I/O and - * devices. An important example is sync random I/O, on flash - * storage with command queueing. So, unless bfqq falls in the - * above cases where idling also boosts throughput, it would - * be important to check conditions (i) and (ii) accurately, - * so as to avoid idling when not strictly needed for service - * guarantees. + * scheduler is likely to coincide with the desired throughput + * distribution only in a completely symmetric, or favorably + * skewed scenario where: + * (i-a) each of these processes must get the same throughput as + * the others, + * (i-b) in case (i-a) does not hold, it holds that the process + * associated with bfqq must receive a lower or equal + * throughput than any of the other processes; + * (ii) the I/O of each process has the same properties, in + * terms of locality (sequential or random), direction + * (reads or writes), request sizes, greediness + * (from I/O-bound to sporadic), and so on; + + * In fact, in such a scenario, the drive tends to treat the requests + * of each process in about the same way as the requests of the + * others, and thus to provide each of these processes with about the + * same throughput. This is exactly the desired throughput + * distribution if (i-a) holds, or, if (i-b) holds instead, this is an + * even more convenient distribution for (the process associated with) + * bfqq. + * + * In contrast, in any asymmetric or unfavorable scenario, device + * idling (I/O-dispatch plugging) is certainly needed to guarantee + * that bfqq receives its assigned fraction of the device throughput + * (see [1] for details). + * + * The problem is that idling may significantly reduce throughput with + * certain combinations of types of I/O and devices. An important + * example is sync random I/O on flash storage with command + * queueing. So, unless bfqq falls in cases where idling also boosts + * throughput, it is important to check conditions (i-a), i(-b) and + * (ii) accurately, so as to avoid idling when not strictly needed for + * service guarantees. * - * Unfortunately, it is extremely difficult to thoroughly - * check condition (ii). And, in case there are active groups, - * it becomes very difficult to check condition (i) too. In - * fact, if there are active groups, then, for condition (i) - * to become false, it is enough that an active group contains - * more active processes or sub-groups than some other active - * group. More precisely, for condition (i) to hold because of - * such a group, it is not even necessary that the group is - * (still) active: it is sufficient that, even if the group - * has become inactive, some of its descendant processes still - * have some request already dispatched but still waiting for - * completion. In fact, requests have still to be guaranteed - * their share of the throughput even after being - * dispatched. In this respect, it is easy to show that, if a - * group frequently becomes inactive while still having - * in-flight requests, and if, when this happens, the group is - * not considered in the calculation of whether the scenario - * is asymmetric, then the group may fail to be guaranteed its - * fair share of the throughput (basically because idling may - * not be performed for the descendant processes of the group, - * but it had to be). We address this issue with the - * following bi-modal behavior, implemented in the function - * bfq_symmetric_scenario(). + * Unfortunately, it is extremely difficult to thoroughly check + * condition (ii). And, in case there are active groups, it becomes + * very difficult to check conditions (i-a) and (i-b) too. In fact, + * if there are active groups, then, for conditions (i-a) or (i-b) to + * become false 'indirectly', it is enough that an active group + * contains more active processes or sub-groups than some other active + * group. More precisely, for conditions (i-a) or (i-b) to become + * false because of such a group, it is not even necessary that the + * group is (still) active: it is sufficient that, even if the group + * has become inactive, some of its descendant processes still have + * some request already dispatched but still waiting for + * completion. In fact, requests have still to be guaranteed their + * share of the throughput even after being dispatched. In this + * respect, it is easy to show that, if a group frequently becomes + * inactive while still having in-flight requests, and if, when this + * happens, the group is not considered in the calculation of whether + * the scenario is asymmetric, then the group may fail to be + * guaranteed its fair share of the throughput (basically because + * idling may not be performed for the descendant processes of the + * group, but it had to be). We address this issue with the following + * bi-modal behavior, implemented in the function + * bfq_asymmetric_scenario(). * * If there are groups with requests waiting for completion * (as commented above, some of these groups may even be * already inactive), then the scenario is tagged as * asymmetric, conservatively, without checking any of the - * conditions (i) and (ii). So the device is idled for bfqq. + * conditions (i-a), (i-b) or (ii). So the device is idled for bfqq. * This behavior matches also the fact that groups are created * exactly if controlling I/O is a primary concern (to * preserve bandwidth and latency guarantees). * - * On the opposite end, if there are no groups with requests - * waiting for completion, then only condition (i) is actually - * controlled, i.e., provided that condition (i) holds, idling - * is not performed, regardless of whether condition (ii) - * holds. In other words, only if condition (i) does not hold, - * then idling is allowed, and the device tends to be - * prevented from queueing many requests, possibly of several - * processes. Since there are no groups with requests waiting - * for completion, then, to control condition (i) it is enough - * to check just whether all the queues with requests waiting - * for completion also have the same weight. + * On the opposite end, if there are no groups with requests waiting + * for completion, then only conditions (i-a) and (i-b) are actually + * controlled, i.e., provided that conditions (i-a) or (i-b) holds, + * idling is not performed, regardless of whether condition (ii) + * holds. In other words, only if conditions (i-a) and (i-b) do not + * hold, then idling is allowed, and the device tends to be prevented + * from queueing many requests, possibly of several processes. Since + * there are no groups with requests waiting for completion, then, to + * control conditions (i-a) and (i-b) it is enough to check just + * whether all the queues with requests waiting for completion also + * have the same weight. * * Not checking condition (ii) evidently exposes bfqq to the * risk of getting less throughput than its fair share. @@ -3639,7 +3663,7 @@ static bool idling_boosts_thr_without_issues(struct bfq_data *bfqd, * compound condition that is checked below for deciding * whether the scenario is asymmetric. To explain this * compound condition, we need to add that the function - * bfq_symmetric_scenario checks the weights of only + * bfq_asymmetric_scenario checks the weights of only * non-weight-raised queues, for efficiency reasons (see * comments on bfq_weights_tree_add()). Then the fact that * bfqq is weight-raised is checked explicitly here. More @@ -3667,7 +3691,7 @@ static bool idling_needed_for_service_guarantees(struct bfq_data *bfqd, return (bfqq->wr_coeff > 1 && bfqd->wr_busy_queues < bfq_tot_busy_queues(bfqd)) || - !bfq_symmetric_scenario(bfqd); + bfq_asymmetric_scenario(bfqd, bfqq); } /* @@ -5505,7 +5529,7 @@ static int bfq_init_queue(struct request_queue *q, struct elevator_type *e) HRTIMER_MODE_REL); bfqd->idle_slice_timer.function = bfq_idle_slice_timer; - bfqd->queue_weights_tree = RB_ROOT; + bfqd->queue_weights_tree = RB_ROOT_CACHED; bfqd->num_groups_with_pending_reqs = 0; INIT_LIST_HEAD(&bfqd->active_list); diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 062e1c4787f4..81cabf51a87e 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -450,7 +450,7 @@ struct bfq_data { * weight-raised @bfq_queue (see the comments to the functions * bfq_weights_tree_[add|remove] for further details). */ - struct rb_root queue_weights_tree; + struct rb_root_cached queue_weights_tree; /* * Number of groups with at least one descendant process that @@ -898,10 +898,10 @@ void bic_set_bfqq(struct bfq_io_cq *bic, struct bfq_queue *bfqq, bool is_sync); struct bfq_data *bic_to_bfqd(struct bfq_io_cq *bic); void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq); void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, - struct rb_root *root); + struct rb_root_cached *root); void __bfq_weights_tree_remove(struct bfq_data *bfqd, struct bfq_queue *bfqq, - struct rb_root *root); + struct rb_root_cached *root); void bfq_weights_tree_remove(struct bfq_data *bfqd, struct bfq_queue *bfqq); void bfq_bfqq_expire(struct bfq_data *bfqd, struct bfq_queue *bfqq, diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index 63311d1ff1ed..0e3f344cc4d3 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -737,7 +737,7 @@ __bfq_entity_update_weight_prio(struct bfq_service_tree *old_st, struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); unsigned int prev_weight, new_weight; struct bfq_data *bfqd = NULL; - struct rb_root *root; + struct rb_root_cached *root; #ifdef CONFIG_BFQ_GROUP_IOSCHED struct bfq_sched_data *sd; struct bfq_group *bfqg; From patchwork Sun Mar 10 18:11:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160028 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11090012jad; Sun, 10 Mar 2019 11:12:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqzO+o+nuH+prphX+Cz8iB7NGBet34KV7MAoZ9Ba299NeNtNU8+1QkU/fo3scdecGuJOdopb X-Received: by 2002:a62:4117:: with SMTP id o23mr29226622pfa.248.1552241543001; Sun, 10 Mar 2019 11:12:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241542; cv=none; d=google.com; s=arc-20160816; b=bYR9qtKZsaGiYkOvK4L54WVhNKoxUeNeap41t9YnJgowHZUZFJhrcDslOXKDGkQN6L QlBqLLcwc6hkTDMDfU2gH7r4DWXolX7KOgnwFaxWfWdrZhxxu1kv6P8/jIGEFvhvTpwD ToUYdfIxWZIiPm6vfmkr2Gnj9uHS7Lv2t5KPTmfvoldXC2jFfJSCNsmnMmnx0tSaIywc efW5DkO4Ml1ajh8ARkHKSoArDR/vs37Ad6dMhahaKStksfXtSOO/rOSynjuriHm+plZZ +ZWTf2M/T0wTtTFdp6yN37Iq1GINJypsmtrKwGaKfmkoZnq+dSWGkXlzAZV3HRLV1xrX acUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=eowyCyg6fE0mBjV4jlMIOF/lyn5g/MwdTODgsX3vJKg=; b=SNp/KKXUsCwFVzjNocbPtzFQsw6izLOo0OWmMMawqkCCapDgTmR1qGzYPnW0+NmEQW +6y4OFEpz38p54/ZFaeoaJDez8lll2ytqIzm7mEq3yxdtf7/gTC36xn2gNtryaGX4XFq 0ZJhagnV7ek7E2uaXrgzhwMHk5p8/hAI3+YAA1G9GTx238DdrZDtaGb9NLWg593SxGkM luknnt+b593ASpv8ECWb4tCVOBxtBBHkZj7nUyTnJq7riLO2rVT5gZF21SrbL3YUewx5 LEiQtwM5hMNZNy03yH7DOw7bYPCC0WF7a0vj/o3tI6fynty/Y/Q9MtUtJKpbqPUWHmsf j8XQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AvqR+OO0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id gn18si3329194plb.185.2019.03.10.11.12.22; Sun, 10 Mar 2019 11:12:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AvqR+OO0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726911AbfCJSMV (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:21 -0400 Received: from mail-wm1-f49.google.com ([209.85.128.49]:51921 "EHLO mail-wm1-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726872AbfCJSMU (ORCPT ); Sun, 10 Mar 2019 14:12:20 -0400 Received: by mail-wm1-f49.google.com with SMTP id n19so2217438wmi.1 for ; Sun, 10 Mar 2019 11:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eowyCyg6fE0mBjV4jlMIOF/lyn5g/MwdTODgsX3vJKg=; b=AvqR+OO0Zd+mpSQzlxKiEhTNVZ9rLfGWcWPIjjRfmTehE+JlUpbsbmiPzxikbCnPwi C1NTx4kejQKyVsE6u7TYWDdsrbLcCxHQH66d2MgAeX9Q7jmbn4FfB0PCCYG8OSbRIdMe wEKCwLZAt3AwhNHLqqLTLUhoCJkrdgVN6VJsQT5CJcM+lqYL0wYO/Ox6EmewK6YxQshH 8apB7jsTAA+FBGXw+35OQtXhQKlcLwZ0LDx5/ag/Ad6iYFBr/INANU+djH/BLbddWXP3 jDiIXR4hfmRLMD6ihlz+qEQmRiTlrbvtMNT+kf7ZdskPCQvN8P7UC3Pj9myUhdU5mUH+ 3twQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eowyCyg6fE0mBjV4jlMIOF/lyn5g/MwdTODgsX3vJKg=; b=mGPW+Fht5q5Onoi0Dze9aDZTXjnyksTsTJ2Eox+6XbQ5TpbpBa7bGG9JX6KD25cZor ic8AG+nE+x+6KH+lfw42FOxmA+a1R3XWl3bMpceWk8oguAQc5anxQH00IzIBp7nr/9gw YyJ2zEuZXd3Pd1cO/t0oOH4u98U2P7HOk7o8abYYu8/CXRNOvGwXspqVdkRGKo3o5sVu 1AgUzeFUlnrHtMnZWtHembFPjHw4gz7oi9MZ+XDMxLGiwClDFda5jrzsCT25Qwmj95HB KR9csPZWTAPH0aAlQyyDZTi2Aof0CwMM8xTy9vc/gR/hT8YH2N4PzqMtbAru6Xpt1p7Y y7Hg== X-Gm-Message-State: APjAAAXGr4tpOWj0imte/VBcxgV++BHEgO+/5I91Q/9m9bp1Z2t/BKdy 7kwaK2g/4/U2708+mncHhdlrjg== X-Received: by 2002:a1c:e2c2:: with SMTP id z185mr14435309wmg.84.1552241536235; Sun, 10 Mar 2019 11:12:16 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:15 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 3/9] block, bfq: tune service injection basing on request service times Date: Sun, 10 Mar 2019 19:11:31 +0100 Message-Id: <20190310181137.2604-4-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The processes associated with a bfq_queue, say Q, may happen to generate their cumulative I/O at a lower rate than the rate at which the device could serve the same I/O. This is rather probable, e.g., if only one process is associated with Q and the device is an SSD. It results in Q becoming often empty while in service. If BFQ is not allowed to switch to another queue when Q becomes empty, then, during the service of Q, there will be frequent "service holes", i.e., time intervals during which Q gets empty and the device can only consume the I/O already queued in its hardware queues. This easily causes considerable losses of throughput. To counter this problem, BFQ implements a request injection mechanism, which tries to fill the above service holes with I/O requests taken from other bfq_queues. The hard part in this mechanism is finding the right amount of I/O to inject, so as to both boost throughput and not break Q's bandwidth and latency guarantees. To this goal, the current version of this mechanism measures the bandwidth enjoyed by Q while it is being served, and tries to inject the maximum possible amount of extra service that does not cause Q's bandwidth to decrease too much. This solution has an important shortcoming. For bandwidth measurements to be stable and reliable, Q must remain in service for a much longer time than that needed to serve a single I/O request. Unfortunately, this does not hold with many workloads. This commit addresses this issue by changing the way the amount of injection allowed is dynamically computed. It tunes injection as a function of the service times of single I/O requests of Q, instead of Q's bandwidth. Single-request service times are evidently meaningful even if Q gets very few I/O requests completed while it is in service. As a testbed for this new solution, we measured the throughput reached by BFQ for one of the nastiest workloads and configurations for this scheduler: the workload generated by the dbench test (in the Phoronix suite), with 6 clients, on a filesystem with journaling, and with the journaling daemon enjoying a higher weight than normal processes. With this commit, the throughput grows from ~100 MB/s to ~150 MB/s on a PLEXTOR PX-256M5. Tested-by: Francesco Pollicino Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 417 ++++++++++++++++++++++++++++++++++++++++---- block/bfq-iosched.h | 51 +++--- 2 files changed, 409 insertions(+), 59 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 2be504f25b09..41364c0cca8c 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -1721,6 +1721,123 @@ static void bfq_add_request(struct request *rq) bfqq->queued[rq_is_sync(rq)]++; bfqd->queued++; + if (RB_EMPTY_ROOT(&bfqq->sort_list) && bfq_bfqq_sync(bfqq)) { + /* + * Periodically reset inject limit, to make sure that + * the latter eventually drops in case workload + * changes, see step (3) in the comments on + * bfq_update_inject_limit(). + */ + if (time_is_before_eq_jiffies(bfqq->decrease_time_jif + + msecs_to_jiffies(1000))) { + /* invalidate baseline total service time */ + bfqq->last_serv_time_ns = 0; + + /* + * Reset pointer in case we are waiting for + * some request completion. + */ + bfqd->waited_rq = NULL; + + /* + * If bfqq has a short think time, then start + * by setting the inject limit to 0 + * prudentially, because the service time of + * an injected I/O request may be higher than + * the think time of bfqq, and therefore, if + * one request was injected when bfqq remains + * empty, this injected request might delay + * the service of the next I/O request for + * bfqq significantly. In case bfqq can + * actually tolerate some injection, then the + * adaptive update will however raise the + * limit soon. This lucky circumstance holds + * exactly because bfqq has a short think + * time, and thus, after remaining empty, is + * likely to get new I/O enqueued---and then + * completed---before being expired. This is + * the very pattern that gives the + * limit-update algorithm the chance to + * measure the effect of injection on request + * service times, and then to update the limit + * accordingly. + * + * On the opposite end, if bfqq has a long + * think time, then start directly by 1, + * because: + * a) on the bright side, keeping at most one + * request in service in the drive is unlikely + * to cause any harm to the latency of bfqq's + * requests, as the service time of a single + * request is likely to be lower than the + * think time of bfqq; + * b) on the downside, after becoming empty, + * bfqq is likely to expire before getting its + * next request. With this request arrival + * pattern, it is very hard to sample total + * service times and update the inject limit + * accordingly (see comments on + * bfq_update_inject_limit()). So the limit is + * likely to be never, or at least seldom, + * updated. As a consequence, by setting the + * limit to 1, we avoid that no injection ever + * occurs with bfqq. On the downside, this + * proactive step further reduces chances to + * actually compute the baseline total service + * time. Thus it reduces chances to execute the + * limit-update algorithm and possibly raise the + * limit to more than 1. + */ + if (bfq_bfqq_has_short_ttime(bfqq)) + bfqq->inject_limit = 0; + else + bfqq->inject_limit = 1; + bfqq->decrease_time_jif = jiffies; + } + + /* + * The following conditions must hold to setup a new + * sampling of total service time, and then a new + * update of the inject limit: + * - bfqq is in service, because the total service + * time is evaluated only for the I/O requests of + * the queues in service; + * - this is the right occasion to compute or to + * lower the baseline total service time, because + * there are actually no requests in the drive, + * or + * the baseline total service time is available, and + * this is the right occasion to compute the other + * quantity needed to update the inject limit, i.e., + * the total service time caused by the amount of + * injection allowed by the current value of the + * limit. It is the right occasion because injection + * has actually been performed during the service + * hole, and there are still in-flight requests, + * which are very likely to be exactly the injected + * requests, or part of them; + * - the minimum interval for sampling the total + * service time and updating the inject limit has + * elapsed. + */ + if (bfqq == bfqd->in_service_queue && + (bfqd->rq_in_driver == 0 || + (bfqq->last_serv_time_ns > 0 && + bfqd->rqs_injected && bfqd->rq_in_driver > 0)) && + time_is_before_eq_jiffies(bfqq->decrease_time_jif + + msecs_to_jiffies(100))) { + bfqd->last_empty_occupied_ns = ktime_get_ns(); + /* + * Start the state machine for measuring the + * total service time of rq: setting + * wait_dispatch will cause bfqd->waited_rq to + * be set when rq will be dispatched. + */ + bfqd->wait_dispatch = true; + bfqd->rqs_injected = false; + } + } + elv_rb_add(&bfqq->sort_list, rq); /* @@ -2566,6 +2683,8 @@ static void bfq_arm_slice_timer(struct bfq_data *bfqd) sl = max_t(u32, sl, 20ULL * NSEC_PER_MSEC); bfqd->last_idling_start = ktime_get(); + bfqd->last_idling_start_jiffies = jiffies; + hrtimer_start(&bfqd->idle_slice_timer, ns_to_ktime(sl), HRTIMER_MODE_REL); bfqg_stats_set_start_idle_time(bfqq_group(bfqq)); @@ -3240,13 +3359,6 @@ static unsigned long bfq_bfqq_softrt_next_start(struct bfq_data *bfqd, jiffies + nsecs_to_jiffies(bfqq->bfqd->bfq_slice_idle) + 4); } -static bool bfq_bfqq_injectable(struct bfq_queue *bfqq) -{ - return BFQQ_SEEKY(bfqq) && bfqq->wr_coeff == 1 && - blk_queue_nonrot(bfqq->bfqd->queue) && - bfqq->bfqd->hw_tag; -} - /** * bfq_bfqq_expire - expire a queue. * @bfqd: device owning the queue. @@ -3361,6 +3473,14 @@ void bfq_bfqq_expire(struct bfq_data *bfqd, "expire (%d, slow %d, num_disp %d, short_ttime %d)", reason, slow, bfqq->dispatched, bfq_bfqq_has_short_ttime(bfqq)); + /* + * bfqq expired, so no total service time needs to be computed + * any longer: reset state machine for measuring total service + * times. + */ + bfqd->rqs_injected = bfqd->wait_dispatch = false; + bfqd->waited_rq = NULL; + /* * Increase, decrease or leave budget unchanged according to * reason. @@ -3372,8 +3492,6 @@ void bfq_bfqq_expire(struct bfq_data *bfqd, if (ref == 1) /* bfqq is gone, no more actions on it */ return; - bfqq->injected_service = 0; - /* mark bfqq as waiting a request only if a bic still points to it */ if (!bfq_bfqq_busy(bfqq) && reason != BFQQE_BUDGET_TIMEOUT && @@ -3767,26 +3885,98 @@ static bool bfq_bfqq_must_idle(struct bfq_queue *bfqq) return RB_EMPTY_ROOT(&bfqq->sort_list) && bfq_better_to_idle(bfqq); } -static struct bfq_queue *bfq_choose_bfqq_for_injection(struct bfq_data *bfqd) +/* + * This function chooses the queue from which to pick the next extra + * I/O request to inject, if it finds a compatible queue. See the + * comments on bfq_update_inject_limit() for details on the injection + * mechanism, and for the definitions of the quantities mentioned + * below. + */ +static struct bfq_queue * +bfq_choose_bfqq_for_injection(struct bfq_data *bfqd) { - struct bfq_queue *bfqq; + struct bfq_queue *bfqq, *in_serv_bfqq = bfqd->in_service_queue; + unsigned int limit = in_serv_bfqq->inject_limit; + /* + * If + * - bfqq is not weight-raised and therefore does not carry + * time-critical I/O, + * or + * - regardless of whether bfqq is weight-raised, bfqq has + * however a long think time, during which it can absorb the + * effect of an appropriate number of extra I/O requests + * from other queues (see bfq_update_inject_limit for + * details on the computation of this number); + * then injection can be performed without restrictions. + */ + bool in_serv_always_inject = in_serv_bfqq->wr_coeff == 1 || + !bfq_bfqq_has_short_ttime(in_serv_bfqq); /* - * A linear search; but, with a high probability, very few - * steps are needed to find a candidate queue, i.e., a queue - * with enough budget left for its next request. In fact: + * If + * - the baseline total service time could not be sampled yet, + * so the inject limit happens to be still 0, and + * - a lot of time has elapsed since the plugging of I/O + * dispatching started, so drive speed is being wasted + * significantly; + * then temporarily raise inject limit to one request. + */ + if (limit == 0 && in_serv_bfqq->last_serv_time_ns == 0 && + bfq_bfqq_wait_request(in_serv_bfqq) && + time_is_before_eq_jiffies(bfqd->last_idling_start_jiffies + + bfqd->bfq_slice_idle) + ) + limit = 1; + + if (bfqd->rq_in_driver >= limit) + return NULL; + + /* + * Linear search of the source queue for injection; but, with + * a high probability, very few steps are needed to find a + * candidate queue, i.e., a queue with enough budget left for + * its next request. In fact: * - BFQ dynamically updates the budget of every queue so as * to accommodate the expected backlog of the queue; * - if a queue gets all its requests dispatched as injected * service, then the queue is removed from the active list - * (and re-added only if it gets new requests, but with - * enough budget for its new backlog). + * (and re-added only if it gets new requests, but then it + * is assigned again enough budget for its new backlog). */ list_for_each_entry(bfqq, &bfqd->active_list, bfqq_list) if (!RB_EMPTY_ROOT(&bfqq->sort_list) && + (in_serv_always_inject || bfqq->wr_coeff > 1) && bfq_serv_to_charge(bfqq->next_rq, bfqq) <= - bfq_bfqq_budget_left(bfqq)) - return bfqq; + bfq_bfqq_budget_left(bfqq)) { + /* + * Allow for only one large in-flight request + * on non-rotational devices, for the + * following reason. On non-rotationl drives, + * large requests take much longer than + * smaller requests to be served. In addition, + * the drive prefers to serve large requests + * w.r.t. to small ones, if it can choose. So, + * having more than one large requests queued + * in the drive may easily make the next first + * request of the in-service queue wait for so + * long to break bfqq's service guarantees. On + * the bright side, large requests let the + * drive reach a very high throughput, even if + * there is only one in-flight large request + * at a time. + */ + if (blk_queue_nonrot(bfqd->queue) && + blk_rq_sectors(bfqq->next_rq) >= + BFQQ_SECT_THR_NONROT) + limit = min_t(unsigned int, 1, limit); + else + limit = in_serv_bfqq->inject_limit; + + if (bfqd->rq_in_driver < limit) { + bfqd->rqs_injected = true; + return bfqq; + } + } return NULL; } @@ -3873,14 +4063,32 @@ static struct bfq_queue *bfq_select_queue(struct bfq_data *bfqd) * for a new request, or has requests waiting for a completion and * may idle after their completion, then keep it anyway. * - * Yet, to boost throughput, inject service from other queues if - * possible. + * Yet, inject service from other queues if it boosts + * throughput and is possible. */ if (bfq_bfqq_wait_request(bfqq) || (bfqq->dispatched != 0 && bfq_better_to_idle(bfqq))) { - if (bfq_bfqq_injectable(bfqq) && - bfqq->injected_service * bfqq->inject_coeff < - bfqq->entity.service * 10) + struct bfq_queue *async_bfqq = + bfqq->bic && bfqq->bic->bfqq[0] && + bfq_bfqq_busy(bfqq->bic->bfqq[0]) ? + bfqq->bic->bfqq[0] : NULL; + + /* + * If the process associated with bfqq has also async + * I/O pending, then inject it + * unconditionally. Injecting I/O from the same + * process can cause no harm to the process. On the + * contrary, it can only increase bandwidth and reduce + * latency for the process. + */ + if (async_bfqq && + icq_to_bic(async_bfqq->next_rq->elv.icq) == bfqq->bic && + bfq_serv_to_charge(async_bfqq->next_rq, async_bfqq) <= + bfq_bfqq_budget_left(async_bfqq)) + bfqq = bfqq->bic->bfqq[0]; + else if (!idling_boosts_thr_without_issues(bfqd, bfqq) && + (bfqq->wr_coeff == 1 || bfqd->wr_busy_queues > 1 || + !bfq_bfqq_has_short_ttime(bfqq))) bfqq = bfq_choose_bfqq_for_injection(bfqd); else bfqq = NULL; @@ -3972,15 +4180,15 @@ static struct request *bfq_dispatch_rq_from_bfqq(struct bfq_data *bfqd, bfq_bfqq_served(bfqq, service_to_charge); - bfq_dispatch_remove(bfqd->queue, rq); + if (bfqq == bfqd->in_service_queue && bfqd->wait_dispatch) { + bfqd->wait_dispatch = false; + bfqd->waited_rq = rq; + } - if (bfqq != bfqd->in_service_queue) { - if (likely(bfqd->in_service_queue)) - bfqd->in_service_queue->injected_service += - bfq_serv_to_charge(rq, bfqq); + bfq_dispatch_remove(bfqd->queue, rq); + if (bfqq != bfqd->in_service_queue) goto return_rq; - } /* * If weight raising has to terminate for bfqq, then next @@ -4411,13 +4619,6 @@ static void bfq_init_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq, bfq_mark_bfqq_has_short_ttime(bfqq); bfq_mark_bfqq_sync(bfqq); bfq_mark_bfqq_just_created(bfqq); - /* - * Aggressively inject a lot of service: up to 90%. - * This coefficient remains constant during bfqq life, - * but this behavior might be changed, after enough - * testing and tuning. - */ - bfqq->inject_coeff = 1; } else bfq_clear_bfqq_sync(bfqq); @@ -4976,6 +5177,147 @@ static void bfq_finish_requeue_request_body(struct bfq_queue *bfqq) bfq_put_queue(bfqq); } +/* + * The processes associated with bfqq may happen to generate their + * cumulative I/O at a lower rate than the rate at which the device + * could serve the same I/O. This is rather probable, e.g., if only + * one process is associated with bfqq and the device is an SSD. It + * results in bfqq becoming often empty while in service. In this + * respect, if BFQ is allowed to switch to another queue when bfqq + * remains empty, then the device goes on being fed with I/O requests, + * and the throughput is not affected. In contrast, if BFQ is not + * allowed to switch to another queue---because bfqq is sync and + * I/O-dispatch needs to be plugged while bfqq is temporarily + * empty---then, during the service of bfqq, there will be frequent + * "service holes", i.e., time intervals during which bfqq gets empty + * and the device can only consume the I/O already queued in its + * hardware queues. During service holes, the device may even get to + * remaining idle. In the end, during the service of bfqq, the device + * is driven at a lower speed than the one it can reach with the kind + * of I/O flowing through bfqq. + * + * To counter this loss of throughput, BFQ implements a "request + * injection mechanism", which tries to fill the above service holes + * with I/O requests taken from other queues. The hard part in this + * mechanism is finding the right amount of I/O to inject, so as to + * both boost throughput and not break bfqq's bandwidth and latency + * guarantees. In this respect, the mechanism maintains a per-queue + * inject limit, computed as below. While bfqq is empty, the injection + * mechanism dispatches extra I/O requests only until the total number + * of I/O requests in flight---i.e., already dispatched but not yet + * completed---remains lower than this limit. + * + * A first definition comes in handy to introduce the algorithm by + * which the inject limit is computed. We define as first request for + * bfqq, an I/O request for bfqq that arrives while bfqq is in + * service, and causes bfqq to switch from empty to non-empty. The + * algorithm updates the limit as a function of the effect of + * injection on the service times of only the first requests of + * bfqq. The reason for this restriction is that these are the + * requests whose service time is affected most, because they are the + * first to arrive after injection possibly occurred. + * + * To evaluate the effect of injection, the algorithm measures the + * "total service time" of first requests. We define as total service + * time of an I/O request, the time that elapses since when the + * request is enqueued into bfqq, to when it is completed. This + * quantity allows the whole effect of injection to be measured. It is + * easy to see why. Suppose that some requests of other queues are + * actually injected while bfqq is empty, and that a new request R + * then arrives for bfqq. If the device does start to serve all or + * part of the injected requests during the service hole, then, + * because of this extra service, it may delay the next invocation of + * the dispatch hook of BFQ. Then, even after R gets eventually + * dispatched, the device may delay the actual service of R if it is + * still busy serving the extra requests, or if it decides to serve, + * before R, some extra request still present in its queues. As a + * conclusion, the cumulative extra delay caused by injection can be + * easily evaluated by just comparing the total service time of first + * requests with and without injection. + * + * The limit-update algorithm works as follows. On the arrival of a + * first request of bfqq, the algorithm measures the total time of the + * request only if one of the three cases below holds, and, for each + * case, it updates the limit as described below: + * + * (1) If there is no in-flight request. This gives a baseline for the + * total service time of the requests of bfqq. If the baseline has + * not been computed yet, then, after computing it, the limit is + * set to 1, to start boosting throughput, and to prepare the + * ground for the next case. If the baseline has already been + * computed, then it is updated, in case it results to be lower + * than the previous value. + * + * (2) If the limit is higher than 0 and there are in-flight + * requests. By comparing the total service time in this case with + * the above baseline, it is possible to know at which extent the + * current value of the limit is inflating the total service + * time. If the inflation is below a certain threshold, then bfqq + * is assumed to be suffering from no perceivable loss of its + * service guarantees, and the limit is even tentatively + * increased. If the inflation is above the threshold, then the + * limit is decreased. Due to the lack of any hysteresis, this + * logic makes the limit oscillate even in steady workload + * conditions. Yet we opted for it, because it is fast in reaching + * the best value for the limit, as a function of the current I/O + * workload. To reduce oscillations, this step is disabled for a + * short time interval after the limit happens to be decreased. + * + * (3) Periodically, after resetting the limit, to make sure that the + * limit eventually drops in case the workload changes. This is + * needed because, after the limit has gone safely up for a + * certain workload, it is impossible to guess whether the + * baseline total service time may have changed, without measuring + * it again without injection. A more effective version of this + * step might be to just sample the baseline, by interrupting + * injection only once, and then to reset/lower the limit only if + * the total service time with the current limit does happen to be + * too large. + * + * More details on each step are provided in the comments on the + * pieces of code that implement these steps: the branch handling the + * transition from empty to non empty in bfq_add_request(), the branch + * handling injection in bfq_select_queue(), and the function + * bfq_choose_bfqq_for_injection(). These comments also explain some + * exceptions, made by the injection mechanism in some special cases. + */ +static void bfq_update_inject_limit(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + u64 tot_time_ns = ktime_get_ns() - bfqd->last_empty_occupied_ns; + unsigned int old_limit = bfqq->inject_limit; + + if (bfqq->last_serv_time_ns > 0) { + u64 threshold = (bfqq->last_serv_time_ns * 3)>>1; + + if (tot_time_ns >= threshold && old_limit > 0) { + bfqq->inject_limit--; + bfqq->decrease_time_jif = jiffies; + } else if (tot_time_ns < threshold && + old_limit < bfqd->max_rq_in_driver<<1) + bfqq->inject_limit++; + } + + /* + * Either we still have to compute the base value for the + * total service time, and there seem to be the right + * conditions to do it, or we can lower the last base value + * computed. + */ + if ((bfqq->last_serv_time_ns == 0 && bfqd->rq_in_driver == 0) || + tot_time_ns < bfqq->last_serv_time_ns) { + bfqq->last_serv_time_ns = tot_time_ns; + /* + * Now we certainly have a base value: make sure we + * start trying injection. + */ + bfqq->inject_limit = max_t(unsigned int, 1, old_limit); + } + + /* update complete, not waiting for any request completion any longer */ + bfqd->waited_rq = NULL; +} + /* * Handle either a requeue or a finish for rq. The things to do are * the same in both cases: all references to rq are to be dropped. In @@ -5020,6 +5362,9 @@ static void bfq_finish_requeue_request(struct request *rq) spin_lock_irqsave(&bfqd->lock, flags); + if (rq == bfqd->waited_rq) + bfq_update_inject_limit(bfqd, bfqq); + bfq_completed_request(bfqq, bfqd); bfq_finish_requeue_request_body(bfqq); diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 81cabf51a87e..26869cfbbfa9 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -240,6 +240,13 @@ struct bfq_queue { /* next ioprio and ioprio class if a change is in progress */ unsigned short new_ioprio, new_ioprio_class; + /* last total-service-time sample, see bfq_update_inject_limit() */ + u64 last_serv_time_ns; + /* limit for request injection */ + unsigned int inject_limit; + /* last time the inject limit has been decreased, in jiffies */ + unsigned long decrease_time_jif; + /* * Shared bfq_queue if queue is cooperating with one or more * other queues. @@ -357,29 +364,6 @@ struct bfq_queue { /* max service rate measured so far */ u32 max_service_rate; - /* - * Ratio between the service received by bfqq while it is in - * service, and the cumulative service (of requests of other - * queues) that may be injected while bfqq is empty but still - * in service. To increase precision, the coefficient is - * measured in tenths of unit. Here are some example of (1) - * ratios, (2) resulting percentages of service injected - * w.r.t. to the total service dispatched while bfqq is in - * service, and (3) corresponding values of the coefficient: - * 1 (50%) -> 10 - * 2 (33%) -> 20 - * 10 (9%) -> 100 - * 9.9 (9%) -> 99 - * 1.5 (40%) -> 15 - * 0.5 (66%) -> 5 - * 0.1 (90%) -> 1 - * - * So, if the coefficient is lower than 10, then - * injected service is more than bfqq service. - */ - unsigned int inject_coeff; - /* amount of service injected in current service slot */ - unsigned int injected_service; }; /** @@ -544,6 +528,26 @@ struct bfq_data { /* time of last request completion (ns) */ u64 last_completion; + /* time of last transition from empty to non-empty (ns) */ + u64 last_empty_occupied_ns; + + /* + * Flag set to activate the sampling of the total service time + * of a just-arrived first I/O request (see + * bfq_update_inject_limit()). This will cause the setting of + * waited_rq when the request is finally dispatched. + */ + bool wait_dispatch; + /* + * If set, then bfq_update_inject_limit() is invoked when + * waited_rq is eventually completed. + */ + struct request *waited_rq; + /* + * True if some request has been injected during the last service hole. + */ + bool rqs_injected; + /* time of first rq dispatch in current observation interval (ns) */ u64 first_dispatch; /* time of last rq dispatch in current observation interval (ns) */ @@ -553,6 +557,7 @@ struct bfq_data { ktime_t last_budget_start; /* beginning of the last idle slice */ ktime_t last_idling_start; + unsigned long last_idling_start_jiffies; /* number of samples in current observation interval */ int peak_rate_samples; From patchwork Sun Mar 10 18:11:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160034 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11090478jad; Sun, 10 Mar 2019 11:12:54 -0700 (PDT) X-Google-Smtp-Source: APXvYqwKya9agp8q6A0aTjY+d3OcSCjLej9XHyJHUeP13GcefipuOrNK9NZYbjz/9ueiuJJGiKgv X-Received: by 2002:a62:e216:: with SMTP id a22mr28992352pfi.20.1552241574917; Sun, 10 Mar 2019 11:12:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241574; cv=none; d=google.com; s=arc-20160816; b=tmZpJGE4jkSiBve77RJgwRRRU3rKl6Jq7UMM9G1aMqaALZGmBBGAeMPM3/ptcHHqWC TJddYiPcBUE50OP4JQitQUErDrMu3QMKXaL+fxip+zMUDDmefe2HejJGMLBzGxqrgSFR N79m01+RZiFNzK550c+Og9TYv7QJyCWFzf3d0VUPa1OgBjJ4gaRs0hOiXTzFpW11yijZ UWMemKxGV0cMWw6sksmS4LGk3Z5Wh898P7k3KXRS0uEWl6oW6WHcIjEmHMkPwC6k1+Ke TCdMJzswOfB/X3J0R/VJz1+TAxDrhprEuhdDIo1KI/+n9HHY6fw5X5nj1EJvbIs8pRGX B8iQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9QZLgskSCrdBD9un8CU62IVDYuz5MmLQQB3oIqrmxZE=; b=g4K0luiPFOIzcQFq0Tb+5sXEPO/0E9vjH6yO1NGHHxvpDQpXcWXGUXzZtVaA2bXKpa jva7e3H7reDnNqC1pG0+yOBUN9YUkot2wftv0iJgSA8UbRfaxrFvp4sW9pjtUbh3LrjZ psEQ9lV4+nJungdgpVn3kYUaS35c4QMGBTBcKdAoHoPLXGopuKiVNEHFBiuGZo/vFPmc a0PfVUWSjVmsHfsMAnhx5fpmPmoiPMHeRSAUfjkClt4skso2JTv52v0QE0PaKmP53g4N FcS+EEEvXORfYr18K7zb8o/lcZ0eR4UEE4ePxV0uhbhIUsZHAakNz7bsoRq/Pv31cJ/R LTMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nO3pUx4O; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b2si3348094pls.31.2019.03.10.11.12.54; Sun, 10 Mar 2019 11:12:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nO3pUx4O; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727019AbfCJSMx (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:53 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:54506 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726897AbfCJSMU (ORCPT ); Sun, 10 Mar 2019 14:12:20 -0400 Received: by mail-wm1-f66.google.com with SMTP id f3so2221634wmj.4 for ; Sun, 10 Mar 2019 11:12:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9QZLgskSCrdBD9un8CU62IVDYuz5MmLQQB3oIqrmxZE=; b=nO3pUx4OMmDgoQO5L2YrY+nrS1Hq8rf5ZJP8jui67Shle2skP7E8ufW5Ep0Dm654C2 i/kkCc2op5YOwwSamzjNbgbCeVkC9dHweYik0iOrudeQXruBK9F6/P0goq27qVnlpbiE NfxDNVVcNCpM5XAqBjV93Bt3ji4ua2B+vYf3KxZkohaPe9SlMLsk28f60ok7OYQNmzmz jg52gentUnxoRJBCy0oHphDHHSYOW6i5SxKgCDNCAgGSMMidgQgZ+jEPTRF3g5U4Os6s jxrzI7OcoS8tru1H2l8w62YgSCnLJbqyuPD8MoPFPPvTXzzdSxAh3EbAsmC4ozXy8ByF lHXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9QZLgskSCrdBD9un8CU62IVDYuz5MmLQQB3oIqrmxZE=; b=n8aCL1ePmCbo7HPi3CF+zkOKWEa0Ri5CAXROncvpgAPojNDgBIGH9VnkyYXlohWWD+ dyrfLuZvF26cWKNCAo4I1iFATjxaxJMFg4G1a22GSrQL5LEWKkHzY6gYEpIX2gD77++L imYHrKNm2SJxQTib+hZ/m7Hk13F1W5nY4C2HEm7abTz00UvzNJnL5E5N/NSQ56VZsJ9c VtDSU8ODAODTjggxPo5xoejbk3cSM0bYh1PHGVX6Cr4oKVxAlufJoE8nW0jfWpp2nYtv GDvFY9XR5R3MHgtG/Xjds3ZIGHT5C58zQByrcOm1UyLJYjMIgdUgjv0m93HyuoIxIrvm Frug== X-Gm-Message-State: APjAAAWe06iLLExwgcIgZ9uufBQVJQXpAZ+43VBStvFB/Sd6tGQhwm0J BaSjVHQpu4j8C7OboChEdLYFBA== X-Received: by 2002:a1c:b603:: with SMTP id g3mr14919968wmf.65.1552241538035; Sun, 10 Mar 2019 11:12:18 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:17 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 4/9] block, bfq: do not merge queues on flash storage with queueing Date: Sun, 10 Mar 2019 19:11:32 +0100 Message-Id: <20190310181137.2604-5-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To boost throughput with a set of processes doing interleaved I/O (i.e., a set of processes whose individual I/O is random, but whose merged cumulative I/O is sequential), BFQ merges the queues associated with these processes, i.e., redirects the I/O of these processes into a common, shared queue. In the shared queue, I/O requests are ordered by their position on the medium, thus sequential I/O gets dispatched to the device when the shared queue is served. Queue merging costs execution time, because, to detect which queues to merge, BFQ must maintain a list of the head I/O requests of active queues, ordered by request positions. Measurements showed that this costs about 10% of BFQ's total per-request processing time. Request processing time becomes more and more critical as the speed of the underlying storage device grows. Yet, fortunately, queue merging is basically useless on the very devices that are so fast to make request processing time critical. To reach a high throughput, these devices must have many requests queued at the same time. But, in this configuration, the internal scheduling algorithms of these devices do also the job of queue merging: they reorder requests so as to obtain as much as possible a sequential I/O pattern. As a consequence, with processes doing interleaved I/O, the throughput reached by one such device is likely to be the same, with and without queue merging. In view of this fact, this commit disables queue merging, and all related housekeeping, for non-rotational devices with internal queueing. The total, single-lock-protected, per-request processing time of BFQ drops to, e.g., 1.9 us on an Intel Core i7-2760QM@2.40GHz (time measured with simple code instrumentation, and using the throughput-sync.sh script of the S suite [1], in performance-profiling mode). To put this result into context, the total, single-lock-protected, per-request execution time of the lightest I/O scheduler available in blk-mq, mq-deadline, is 0.7 us (mq-deadline is ~800 LOC, against ~10500 LOC for BFQ). Disabling merging provides a further, remarkable benefit in terms of throughput. Merging tends to make many workloads artificially more uneven, mainly because of shared queues remaining non empty for incomparably more time than normal queues. So, if, e.g., one of the queues in a set of merged queues has a higher weight than a normal queue, then the shared queue may inherit such a high weight and, by staying almost always active, may force BFQ to perform I/O plugging most of the time. This evidently makes it harder for BFQ to let the device reach a high throughput. As a practical example of this problem, and of the benefits of this commit, we measured again the throughput in the nasty scenario considered in previous commit messages: dbench test (in the Phoronix suite), with 6 clients, on a filesystem with journaling, and with the journaling daemon enjoying a higher weight than normal processes. With this commit, the throughput grows from ~150 MB/s to ~200 MB/s on a PLEXTOR PX-256M5 SSD. This is the same peak throughput reached by any of the other I/O schedulers. As such, this is also likely to be the maximum possible throughput reachable with this workload on this device, because I/O is mostly random, and the other schedulers basically just pass I/O requests to the drive as fast as possible. [1] https://github.com/Algodev-github/S Tested-by: Francesco Pollicino Signed-off-by: Alessio Masola Signed-off-by: Paolo Valente --- block/bfq-cgroup.c | 3 +- block/bfq-iosched.c | 73 +++++++++++++++++++++++++++++++++++++++++---- block/bfq-iosched.h | 3 ++ 3 files changed, 73 insertions(+), 6 deletions(-) -- 2.20.1 diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index c6113af31960..2a74a3f2a8f7 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -578,7 +578,8 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, bfqg_and_blkg_get(bfqg); if (bfq_bfqq_busy(bfqq)) { - bfq_pos_tree_add_move(bfqd, bfqq); + if (unlikely(!bfqd->nonrot_with_queueing)) + bfq_pos_tree_add_move(bfqd, bfqq); bfq_activate_bfqq(bfqd, bfqq); } diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 41364c0cca8c..b96be3764b8a 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -595,7 +595,16 @@ static bool bfq_too_late_for_merging(struct bfq_queue *bfqq) bfq_merge_time_limit); } -void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) +/* + * The following function is not marked as __cold because it is + * actually cold, but for the same performance goal described in the + * comments on the likely() at the beginning of + * bfq_setup_cooperator(). Unexpectedly, to reach an even lower + * execution time for the case where this function is not invoked, we + * had to add an unlikely() in each involved if(). + */ +void __cold +bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) { struct rb_node **p, *parent; struct bfq_queue *__bfqq; @@ -1849,8 +1858,9 @@ static void bfq_add_request(struct request *rq) /* * Adjust priority tree position, if next_rq changes. + * See comments on bfq_pos_tree_add_move() for the unlikely(). */ - if (prev != bfqq->next_rq) + if (unlikely(!bfqd->nonrot_with_queueing && prev != bfqq->next_rq)) bfq_pos_tree_add_move(bfqd, bfqq); if (!bfq_bfqq_busy(bfqq)) /* switching to busy ... */ @@ -1990,7 +2000,9 @@ static void bfq_remove_request(struct request_queue *q, bfqq->pos_root = NULL; } } else { - bfq_pos_tree_add_move(bfqd, bfqq); + /* see comments on bfq_pos_tree_add_move() for the unlikely() */ + if (unlikely(!bfqd->nonrot_with_queueing)) + bfq_pos_tree_add_move(bfqd, bfqq); } if (rq->cmd_flags & REQ_META) @@ -2075,7 +2087,12 @@ static void bfq_request_merged(struct request_queue *q, struct request *req, */ if (prev != bfqq->next_rq) { bfq_updated_next_req(bfqd, bfqq); - bfq_pos_tree_add_move(bfqd, bfqq); + /* + * See comments on bfq_pos_tree_add_move() for + * the unlikely(). + */ + if (unlikely(!bfqd->nonrot_with_queueing)) + bfq_pos_tree_add_move(bfqd, bfqq); } } } @@ -2357,6 +2374,46 @@ bfq_setup_cooperator(struct bfq_data *bfqd, struct bfq_queue *bfqq, { struct bfq_queue *in_service_bfqq, *new_bfqq; + /* + * Do not perform queue merging if the device is non + * rotational and performs internal queueing. In fact, such a + * device reaches a high speed through internal parallelism + * and pipelining. This means that, to reach a high + * throughput, it must have many requests enqueued at the same + * time. But, in this configuration, the internal scheduling + * algorithm of the device does exactly the job of queue + * merging: it reorders requests so as to obtain as much as + * possible a sequential I/O pattern. As a consequence, with + * the workload generated by processes doing interleaved I/O, + * the throughput reached by the device is likely to be the + * same, with and without queue merging. + * + * Disabling merging also provides a remarkable benefit in + * terms of throughput. Merging tends to make many workloads + * artificially more uneven, because of shared queues + * remaining non empty for incomparably more time than + * non-merged queues. This may accentuate workload + * asymmetries. For example, if one of the queues in a set of + * merged queues has a higher weight than a normal queue, then + * the shared queue may inherit such a high weight and, by + * staying almost always active, may force BFQ to perform I/O + * plugging most of the time. This evidently makes it harder + * for BFQ to let the device reach a high throughput. + * + * Finally, the likely() macro below is not used because one + * of the two branches is more likely than the other, but to + * have the code path after the following if() executed as + * fast as possible for the case of a non rotational device + * with queueing. We want it because this is the fastest kind + * of device. On the opposite end, the likely() may lengthen + * the execution time of BFQ for the case of slower devices + * (rotational or at least without queueing). But in this case + * the execution time of BFQ matters very little, if not at + * all. + */ + if (likely(bfqd->nonrot_with_queueing)) + return NULL; + /* * Prevent bfqq from being merged if it has been created too * long ago. The idea is that true cooperating processes, and @@ -2986,8 +3043,10 @@ static void __bfq_bfqq_expire(struct bfq_data *bfqd, struct bfq_queue *bfqq) bfq_requeue_bfqq(bfqd, bfqq, true); /* * Resort priority tree of potential close cooperators. + * See comments on bfq_pos_tree_add_move() for the unlikely(). */ - bfq_pos_tree_add_move(bfqd, bfqq); + if (unlikely(!bfqd->nonrot_with_queueing)) + bfq_pos_tree_add_move(bfqd, bfqq); } /* @@ -5051,6 +5110,9 @@ static void bfq_update_hw_tag(struct bfq_data *bfqd) bfqd->hw_tag = bfqd->max_rq_in_driver > BFQ_HW_QUEUE_THRESHOLD; bfqd->max_rq_in_driver = 0; bfqd->hw_tag_samples = 0; + + bfqd->nonrot_with_queueing = + blk_queue_nonrot(bfqd->queue) && bfqd->hw_tag; } static void bfq_completed_request(struct bfq_queue *bfqq, struct bfq_data *bfqd) @@ -5882,6 +5944,7 @@ static int bfq_init_queue(struct request_queue *q, struct elevator_type *e) INIT_HLIST_HEAD(&bfqd->burst_list); bfqd->hw_tag = -1; + bfqd->nonrot_with_queueing = blk_queue_nonrot(bfqd->queue); bfqd->bfq_max_budget = bfq_default_max_budget; diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 26869cfbbfa9..829730b96fb2 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -497,6 +497,9 @@ struct bfq_data { /* number of requests dispatched and waiting for completion */ int rq_in_driver; + /* true if the device is non rotational and performs queueing */ + bool nonrot_with_queueing; + /* * Maximum number of requests in driver in the last * @hw_tag_samples completed requests. From patchwork Sun Mar 10 18:11:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160029 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11090070jad; Sun, 10 Mar 2019 11:12:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqwK0prTAD0/t+bL822lhHFkPdX9Aejs3yOZl2Zs4J8txqVzmBz+BnpRX4OWhjq4aepUh3Pv X-Received: by 2002:a65:4342:: with SMTP id k2mr11123816pgq.445.1552241547157; Sun, 10 Mar 2019 11:12:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241547; cv=none; d=google.com; s=arc-20160816; b=iJSTQkbeT9hTH3Axw4I3YyvZedX3l85rbf/E9TOp/QUUtm7YwKftYjblQx5Xf0Mh4W i7qWjtfr4ONZB+qvQG9QsJn1M2DU2h6uWO9S1zDz/TVi7ty0dGQp9MnoSFQIvhQEPgPo gtEMQVjoA97AWXstg440jvefL5xcYsJJfjGpY/T6qJvy5W4RnKFs7AFCyjS8c6cIfOjL /UmxIkjeC2wvsfyAm/B8jRw3z+E5Qy/ju+PQEg+iM/k73sJd/iK3X+HaXFDbzxBGgR13 c/MyU3cW+5kLa9ZxefKUTeG6dtuIQ/wG7LWp3sJ63GzYohAd6DpcYFBKkh82WkQ3RE9N 6C3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QUHw4uD/6PdjAyafvFS/vFaSGIdhMTKmdDdLyjrgLw8=; b=q76570r4ElwED0u4RsEfRPrgSva/PF61yP37qyfh8MFWhIAQoIYM8cnC6BLhZruQgP 07vIszdXRDJufi1q3oRPQ3xJZD338uZzJSOOgM3fIc28bWs3A9U0BeOk4107n4Zd/G5X 7TMuoazJbvzAuspg86iJfZ1O2rM6E0BPswLG4g3VE5Uu6rJeF2WEEYix6mmS4sw0DpIV +gfiOdYhHnOKLPI6L0DEvT8wIqua1vuXiY023HBaTJhusc40ProTOPk+zXCySNJq+x3h w0VlmlN4amODfXZoFNytMDTeNDpYy/sodl+kTH7TOr8NpcmHNc37AJB8h84p9lmkC5WE sI0w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gtdmQx2Z; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g19si3134476pgk.300.2019.03.10.11.12.26; Sun, 10 Mar 2019 11:12:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gtdmQx2Z; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726955AbfCJSMZ (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:25 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:37623 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726905AbfCJSMW (ORCPT ); Sun, 10 Mar 2019 14:12:22 -0400 Received: by mail-wr1-f68.google.com with SMTP id y15so833968wro.4 for ; Sun, 10 Mar 2019 11:12:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QUHw4uD/6PdjAyafvFS/vFaSGIdhMTKmdDdLyjrgLw8=; b=gtdmQx2Zk6ww/kOS2ogjkJUpLQtwK3crXM1AJgeofygnkUT9nw5qv9MWszMcLDTTSP StOKhxeSNxBCEBcshrFDwNaO6mALCJqcW761baZGdJKTvJWE7vU4n1QYsPY0+MwPDgUu 9JnnWzfKHLu4V0NpzpirdIO4vYfwx4dObLII9rGtWnJpw6cHBgdgDxx4pf363mXXQizX lxGG5cvUbIyDnayRqgBml8xHhJAU62O7YsDv2IoqeZcn/zDOzfsfX8PMXm524exgiwvN jZg+tcLcHZd7W+Mpxf+z8LoeZ4H1z9WHmNaJfY5ON16KIo6CoQP0fZ4KMafXsyCU5EVD VF/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QUHw4uD/6PdjAyafvFS/vFaSGIdhMTKmdDdLyjrgLw8=; b=gg4ChADhH7tTf1QiaqTCvnE2GtJTZ5zIx7pfnUyt2RmobMeBjGz6jMABeRt6t8YJro 8dYr+9POmD/34KK0q90xMH4/nQMSf3OLpfQQR6oLTeJZuduJ1/2D6iQVFsf1QRR0/zgE 3UgaMRdr5KLrhnaJq5I0hrE+8q427842ZwV8OIvnmfjEXqcuu0uguL/1GpvIQY8fzixR TIjajT32bMNuQxtJSRhwMWjx6TMBv/Oaxf2O+E78mTr7O9oQ6d4sQAoqhILwJm+sObW/ TqmVlcSjJhlIZS/U6fwa+nWiPeb6mxkkUhMNv0S1wo/cHM2DzipGn2iwAv8Bbn8K5XSl 3QIA== X-Gm-Message-State: APjAAAXmMhy2sqewPUvWtDrNGH6MzfkmDuIq7ywiQGBGqEtXwR84nSAU KOys1WymFOb/g+2H7XGA5y6C3w== X-Received: by 2002:adf:dd12:: with SMTP id a18mr16495341wrm.4.1552241540362; Sun, 10 Mar 2019 11:12:20 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:19 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 5/9] block, bfq: do not tag totally seeky queues as soft rt Date: Sun, 10 Mar 2019 19:11:33 +0100 Message-Id: <20190310181137.2604-6-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Sync random I/O is likely to be confused with soft real-time I/O, because it is characterized by limited throughput and apparently isochronous arrival pattern. To avoid false positives, this commits prevents bfq_queues containing only random (seeky) I/O from being tagged as soft real-time. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index b96be3764b8a..d34b80e5c47d 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -242,6 +242,14 @@ static struct kmem_cache *bfq_pool; blk_rq_sectors(rq) < BFQQ_SECT_THR_NONROT)) #define BFQQ_CLOSE_THR (sector_t)(8 * 1024) #define BFQQ_SEEKY(bfqq) (hweight32(bfqq->seek_history) > 19) +/* + * Sync random I/O is likely to be confused with soft real-time I/O, + * because it is characterized by limited throughput and apparently + * isochronous arrival pattern. To avoid false positives, queues + * containing only random (seeky) I/O are prevented from being tagged + * as soft real-time. + */ +#define BFQQ_TOTALLY_SEEKY(bfqq) (bfqq->seek_history & -1) /* Min number of samples required to perform peak-rate update */ #define BFQ_RATE_MIN_SAMPLES 32 @@ -1622,6 +1630,7 @@ static void bfq_bfqq_handle_idle_busy_switch(struct bfq_data *bfqd, */ in_burst = bfq_bfqq_in_large_burst(bfqq); soft_rt = bfqd->bfq_wr_max_softrt_rate > 0 && + !BFQQ_TOTALLY_SEEKY(bfqq) && !in_burst && time_is_before_jiffies(bfqq->soft_rt_next_start) && bfqq->dispatched == 0; @@ -4816,6 +4825,11 @@ bfq_update_io_seektime(struct bfq_data *bfqd, struct bfq_queue *bfqq, { bfqq->seek_history <<= 1; bfqq->seek_history |= BFQ_RQ_SEEKY(bfqd, bfqq->last_request_pos, rq); + + if (bfqq->wr_coeff > 1 && + bfqq->wr_cur_max_time == bfqd->bfq_wr_rt_max_time && + BFQQ_TOTALLY_SEEKY(bfqq)) + bfq_bfqq_end_wr(bfqq); } static void bfq_update_has_short_ttime(struct bfq_data *bfqd, From patchwork Sun Mar 10 18:11:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160031 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11090194jad; Sun, 10 Mar 2019 11:12:37 -0700 (PDT) X-Google-Smtp-Source: APXvYqyBMaioCvFttoI2rLe8ZqhHz1qXD66TNqX52s9BwUwfQ8uFHyvS3aMZDAoG7IBtIKuUC9O+ X-Received: by 2002:a17:902:168:: with SMTP id 95mr7483684plb.212.1552241557119; Sun, 10 Mar 2019 11:12:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241557; cv=none; d=google.com; s=arc-20160816; b=e4EvIqdaL/Xse3GdzGroPFsqbMXkc441Bam7GTRNvsXG8YAlbgcqg5oTtlHnzhyPHv 5OIF3XiYMUQyBnY2Di0K4qcjwidSWfpXLV9h/TyS5WqKpiFS2pcK2FlLcXFjlI3KXF/9 FnbnGTQj+ALE6QcBrtUsHRwNm5caVBrsOyRMQOidt1nACLeLB/A1Q3sA4SfuO9qh5nvd 5jsDg+F3eOedWIhZFwGUdRYsL7VEI9dq9rGl8mY8M8V1Ym6GkbAwHL6qbhG+DOE5LN9t sNXmQNtrRHVcHxacwKEW4kaV7Owb/qFyjdcnrPMABvpDvMBLJmyfZezUyWJqgVOAeiBF bkqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=7PDjwfDL/90N589WNTZ9R1L1ulLSIazVR8UDi7lsijE=; b=o5sVgL0edUlmCP52o1zXo9uxlSZcnA5Kl2xo2K2303zayCCM8QGK/UkFkL9fBtfHFf H8GeGWp+pyCzjCdjODA+5Zl6okTtCIjlbKKmFvboUMsuf9kE8sGiLDKXJZ5KrJKRbeKG eFmslQmOavdGaKv49j2O3chl01U+6MuuLRwjggImsh07MRMrhYZsdo0mBTEXC/jspPvk Ksem7jVgMNY7CrIKzMCe/YPAgZ9hJPuBiv1EpDFpACrHFKo5kGc2iTlxr2aiFSh4DoeU HluR8RK+2xOkCTJcSyQt7LEwlYcx1Yx+m69DoX5pRpOD+KZ9nJldvl8P/0izT+LN+URF 0vnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Lr4ertf0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x1si3521239plb.245.2019.03.10.11.12.36; Sun, 10 Mar 2019 11:12:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Lr4ertf0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726982AbfCJSMa (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:30 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:41288 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726915AbfCJSMY (ORCPT ); Sun, 10 Mar 2019 14:12:24 -0400 Received: by mail-wr1-f68.google.com with SMTP id n2so2639890wrw.8 for ; Sun, 10 Mar 2019 11:12:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7PDjwfDL/90N589WNTZ9R1L1ulLSIazVR8UDi7lsijE=; b=Lr4ertf0tzp5hH0U25sR+JkJzcx66Ugrnyw3/M1VV8GpvKJ+srbt6fPHavCY+BIWXY zSDDAHt8Dbmdch71AW3EN9EesYKa4GWuIna+Z8H4tBUaAPvo/hTcIy1gwIsifbV7zcyT VMSuiE1aXZkhP43vwfIWg/DfMK8ND7/njZSkh4rnokOcxi6TmiY0JkR812RZjk23T5F2 t87hNhejsy4WZo76g5XJX78Am5+W1eaJFEhgAv8XGjkriqcUtKBPXDRu3JJ8RJsPkzYc V9ymFUgmxysjmid36aZEOfqIp/e/1MNweHbxJHgRo3CZ9cX1fIm2fbMLEc7LC8geg9b8 kiKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7PDjwfDL/90N589WNTZ9R1L1ulLSIazVR8UDi7lsijE=; b=lRGjEyiMbcQ/BlhCaG36JnQoK1o6yrcitTlSfs48yrM4ysy2zSNOAFgM34edfrMTwg NUz1I/vQOtGwJ7MaYC0OWN2mA40yOoLqaqanjfKUh8ETSuM0PxPHAK4YdlvQsxQGatsw Jtj5/lvBz5GreWl5UL5FaNGcRgdm0EzubEg/O0F7n7Q9mHKNqulMvPpPYj6TddUcZXMJ IU5QeYyZc4lKK0oM3efe9iygJ/Kai9eKNPGq9BZztLlHh4t7XLxhMa59ifRUyj+S9sAM 0Vkqz9UwVb0cBXZtGkDgTUAFmilCDMfmN2MDHgrlQf7a0y50+hZgl1DPecLM59E50fg5 BSxQ== X-Gm-Message-State: APjAAAV7o0ymJ33lbAErWUehdm30bjIuzebSzMtjSp5KwvRlvfBD6vfX 5bu/s6H2+RPnQs5sTiddOa64D6CTiWc= X-Received: by 2002:adf:e64c:: with SMTP id b12mr8935701wrn.112.1552241542338; Sun, 10 Mar 2019 11:12:22 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:21 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 6/9] block, bfq: always protect newly-created queues from existing active queues Date: Sun, 10 Mar 2019 19:11:34 +0100 Message-Id: <20190310181137.2604-7-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If many bfq_queues belonging to the same group happen to be created shortly after each other, then the processes associated with these queues have typically a common goal. In particular, bursts of queue creations are usually caused by services or applications that spawn many parallel threads/processes. Examples are systemd during boot, or git grep. If there are no other active queues, then, to help these processes get their job done as soon as possible, the best thing to do is to reach a high throughput. To this goal, it is usually better to not grant either weight-raising or device idling to the queues associated with these processes. And this is exactly what BFQ currently does. There is however a drawback: if, in contrast, some other queues are already active, then the newly created queues must be protected from the I/O flowing through the already existing queues. In this case, the best thing to do is the opposite as in the other case: it is much better to grant weight-raising and device idling to the newly-created queues, if they deserve it. This commit addresses this issue by doing so if there are already other active queues. This change also helps eliminating false positives, which occur when the newly-created queues do not belong to an actual large burst of creations, but some background task (e.g., a service) happens to trigger the creation of new queues in the middle, i.e., very close to when the victim queues are created. These false positive may cause total loss of control on process latencies. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 64 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 51 insertions(+), 13 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index d34b80e5c47d..500b04df9efa 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -1075,8 +1075,18 @@ static void bfq_reset_burst_list(struct bfq_data *bfqd, struct bfq_queue *bfqq) hlist_for_each_entry_safe(item, n, &bfqd->burst_list, burst_list_node) hlist_del_init(&item->burst_list_node); - hlist_add_head(&bfqq->burst_list_node, &bfqd->burst_list); - bfqd->burst_size = 1; + + /* + * Start the creation of a new burst list only if there is no + * active queue. See comments on the conditional invocation of + * bfq_handle_burst(). + */ + if (bfq_tot_busy_queues(bfqd) == 0) { + hlist_add_head(&bfqq->burst_list_node, &bfqd->burst_list); + bfqd->burst_size = 1; + } else + bfqd->burst_size = 0; + bfqd->burst_parent_entity = bfqq->entity.parent; } @@ -1132,7 +1142,8 @@ static void bfq_add_to_burst(struct bfq_data *bfqd, struct bfq_queue *bfqq) * many parallel threads/processes. Examples are systemd during boot, * or git grep. To help these processes get their job done as soon as * possible, it is usually better to not grant either weight-raising - * or device idling to their queues. + * or device idling to their queues, unless these queues must be + * protected from the I/O flowing through other active queues. * * In this comment we describe, firstly, the reasons why this fact * holds, and, secondly, the next function, which implements the main @@ -1144,7 +1155,10 @@ static void bfq_add_to_burst(struct bfq_data *bfqd, struct bfq_queue *bfqq) * cumulatively served, the sooner the target job of these queues gets * completed. As a consequence, weight-raising any of these queues, * which also implies idling the device for it, is almost always - * counterproductive. In most cases it just lowers throughput. + * counterproductive, unless there are other active queues to isolate + * these new queues from. If there no other active queues, then + * weight-raising these new queues just lowers throughput in most + * cases. * * On the other hand, a burst of queue creations may be caused also by * the start of an application that does not consist of a lot of @@ -1178,14 +1192,16 @@ static void bfq_add_to_burst(struct bfq_data *bfqd, struct bfq_queue *bfqq) * are very rare. They typically occur if some service happens to * start doing I/O exactly when the interactive task starts. * - * Turning back to the next function, it implements all the steps - * needed to detect the occurrence of a large burst and to properly - * mark all the queues belonging to it (so that they can then be - * treated in a different way). This goal is achieved by maintaining a - * "burst list" that holds, temporarily, the queues that belong to the - * burst in progress. The list is then used to mark these queues as - * belonging to a large burst if the burst does become large. The main - * steps are the following. + * Turning back to the next function, it is invoked only if there are + * no active queues (apart from active queues that would belong to the + * same, possible burst bfqq would belong to), and it implements all + * the steps needed to detect the occurrence of a large burst and to + * properly mark all the queues belonging to it (so that they can then + * be treated in a different way). This goal is achieved by + * maintaining a "burst list" that holds, temporarily, the queues that + * belong to the burst in progress. The list is then used to mark + * these queues as belonging to a large burst if the burst does become + * large. The main steps are the following. * * . when the very first queue is created, the queue is inserted into the * list (as it could be the first queue in a possible burst) @@ -5695,7 +5711,29 @@ static struct bfq_queue *bfq_init_rq(struct request *rq) } } - if (unlikely(bfq_bfqq_just_created(bfqq))) + /* + * Consider bfqq as possibly belonging to a burst of newly + * created queues only if: + * 1) A burst is actually happening (bfqd->burst_size > 0) + * or + * 2) There is no other active queue. In fact, if, in + * contrast, there are active queues not belonging to the + * possible burst bfqq may belong to, then there is no gain + * in considering bfqq as belonging to a burst, and + * therefore in not weight-raising bfqq. See comments on + * bfq_handle_burst(). + * + * This filtering also helps eliminating false positives, + * occurring when bfqq does not belong to an actual large + * burst, but some background task (e.g., a service) happens + * to trigger the creation of new queues very close to when + * bfqq and its possible companion queues are created. See + * comments on bfq_handle_burst() for further details also on + * this issue. + */ + if (unlikely(bfq_bfqq_just_created(bfqq) && + (bfqd->burst_size > 0 || + bfq_tot_busy_queues(bfqd) == 0))) bfq_handle_burst(bfqd, bfqq); return bfqq; From patchwork Sun Mar 10 18:11:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160033 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11090325jad; Sun, 10 Mar 2019 11:12:46 -0700 (PDT) X-Google-Smtp-Source: APXvYqwS4ZVluPCtSdB+cJCoBVI9J9C7q4tMfaZPI2vwFmG+vPmuSUNV9YaPVWsKdcmCELqnAz03 X-Received: by 2002:a63:83:: with SMTP id 125mr26725746pga.403.1552241566662; Sun, 10 Mar 2019 11:12:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241566; cv=none; d=google.com; s=arc-20160816; b=k5yAOnlS7uvNKrHfqNvmcKlmdufBoGQS69+FrPSs31pmNCgCkh8B3DUzMOjnFXn6Zp iFwRwO7PtoOG+i/Y9p8Sw67f3QqGT6r+r2FNVvDBgXDyy5qdodq2fuAJRGxYA1ZAZ0oa KOszhJgaqSbKs9kv9oSnZHxSBBAd028SPem1KZ+SYFTr2WKw953VIX1420bn/nyElATl HrfvcTBxXwm1Vf2qdN3UckTl9VkmQmpPNmZkHv+Ji7HtI2AKPBg6w8HjBwOkCz3M3rZz crxSVDFKLstuqP8qgeROAAXV2YJzwAdptFmjG/W31MfMFTJtMV2Zc4Xje+ZcdGLTHWB/ 7VPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Rzi6we5rwF3Y64l19PsLoFd2B5RuM9WGk0GjrjaWejk=; b=z3W2fWPhZ3wUOLUsscGu2m5Q1nPAxtJ9eVcm99u0fT3KgAxvUCtxq0NgAQ0HUinCGt YwM+TxOclWNccAFPEFCEIjcJXpXtehKN4BIcbx1fct3BcZaX1YDBBU+jEalTQg99NZon hdFAJJRZ/NgbI7SAGtWTNgimiE/YFNNIipEsib3i1bp9NgNudgRmJFZYB0lzQ1F/Tum+ +bHA+YTlL/tt5/fuyOJoQMwaTHvmguSy6UdRKkLIrRZ7ywyAUxMAOjrZCY6WTNmKA+EF GI2/CUIfaT4E5HqrBEhlNgCAtA2HjwSgYBnB2YaC8x7VQ/0ccQgNxLoRG1C/fbZuxgJA 0v6w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QippPLGl; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j69si3032797pgd.31.2019.03.10.11.12.46; Sun, 10 Mar 2019 11:12:46 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QippPLGl; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727011AbfCJSMp (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:45 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:39524 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726945AbfCJSM0 (ORCPT ); Sun, 10 Mar 2019 14:12:26 -0400 Received: by mail-wr1-f68.google.com with SMTP id p8so1036910wrq.6 for ; Sun, 10 Mar 2019 11:12:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rzi6we5rwF3Y64l19PsLoFd2B5RuM9WGk0GjrjaWejk=; b=QippPLGlc/JE4mt1hPQnApMT6VQ2+P2ihx/NcxK9/f1FpFPKP6s5TLVUVYacaKqkkC 6ebcAHsbiy82QhweTPwXTB/zkESgxYcXWN553XYhhc6OOAQzfid12lBW6S3VvuwB7C0W ZFRop+pnOxeU3G3u7OIoMC4COenmsodLBR0srU32+t6oZ+GBBVIVoFE1phTqrZhzjKPR qVkIo2nkL6qfuXTYFUxwUZX85iDscB7A6ivF062zY3zDhCjRxJOs/W6teMBfpT/msHii cSnGfOX4CNw45oQ5vfZd+0sD2U5CP8dRhrTgLGrw+dqn1pe8U/Jor5QQeGdGHKH8s4L+ 0g/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Rzi6we5rwF3Y64l19PsLoFd2B5RuM9WGk0GjrjaWejk=; b=SvZ7p5tt9vATBzMOLcjpB/wgCn+wKcLERtvNTy6MMsadNZfVns/iBvgouGllgLNqoN DVqQKnfq1QoIZyq+W3GK+W0boUOetxbiAiCgeH2KhkxIT4Ct5Xbd5CqmerUlfmd+STVB y8MsKfc9YW8lyVUWW4VA8JLZLJSJG1xXI15Gxvll44y/ZLkEiL6BDVrLqviHJMZw7IpZ wjxyGGwp0RsUpaFt2KqcFr3S26O4e0Dtik1OqCnzYh16SA8Z77eQCnLAw9uKXOZ6s3Cs NOhIyRseRoFzTHrjJH7JsrHdLcaA8v/yd9GKF2DWnt/EBQFjaYoJKdF5VF8Ss9+c1K/W 2xAw== X-Gm-Message-State: APjAAAX45eWY8YoQ2Uusu+q30rUFOyKqKsgVkch9oubvKqetg/BaQ4El ixeoA9NpLpNcDNnXm6X33pJnUQ== X-Received: by 2002:adf:ec41:: with SMTP id w1mr18411906wrn.184.1552241544158; Sun, 10 Mar 2019 11:12:24 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:23 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 7/9] block, bfq: print SHARED instead of pid for shared queues in logs Date: Sun, 10 Mar 2019 19:11:35 +0100 Message-Id: <20190310181137.2604-8-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Francesco Pollicino The function "bfq_log_bfqq" prints the pid of the process associated with the queue passed as input. Unfortunately, if the queue is shared, then more than one process is associated with the queue. The pid that gets printed in this case is the pid of one of the associated processes. Which process gets printed depends on the exact sequence of merge events the queue underwent. So printing such a pid is rather useless and above all is often rather confusing because it reports a random pid between those of the associated processes. This commit addresses this issue by printing SHARED instead of a pid if the queue is shared. Signed-off-by: Francesco Pollicino Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 10 ++++++++++ block/bfq-iosched.h | 23 +++++++++++++++++++---- 2 files changed, 29 insertions(+), 4 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 500b04df9efa..7d95d9c01036 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2590,6 +2590,16 @@ bfq_merge_bfqqs(struct bfq_data *bfqd, struct bfq_io_cq *bic, * assignment causes no harm). */ new_bfqq->bic = NULL; + /* + * If the queue is shared, the pid is the pid of one of the associated + * processes. Which pid depends on the exact sequence of merge events + * the queue underwent. So printing such a pid is useless and confusing + * because it reports a random pid between those of the associated + * processes. + * We mark such a queue with a pid -1, and then print SHARED instead of + * a pid in logging messages. + */ + new_bfqq->pid = -1; bfqq->bic = NULL; /* release process reference to bfqq */ bfq_put_queue(bfqq); diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 829730b96fb2..6410cc9a064d 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -32,6 +32,8 @@ #define BFQ_DEFAULT_GRP_IOPRIO 0 #define BFQ_DEFAULT_GRP_CLASS IOPRIO_CLASS_BE +#define MAX_PID_STR_LENGTH 12 + /* * Soft real-time applications are extremely more latency sensitive * than interactive ones. Over-raise the weight of the former to @@ -1016,13 +1018,23 @@ void bfq_add_bfqq_busy(struct bfq_data *bfqd, struct bfq_queue *bfqq); /* --------------- end of interface of B-WF2Q+ ---------------- */ /* Logging facilities. */ +static void bfq_pid_to_str(int pid, char *str, int len) +{ + if (pid != -1) + snprintf(str, len, "%d", pid); + else + snprintf(str, len, "SHARED-"); +} + #ifdef CONFIG_BFQ_GROUP_IOSCHED struct bfq_group *bfqq_group(struct bfq_queue *bfqq); #define bfq_log_bfqq(bfqd, bfqq, fmt, args...) do { \ + char pid_str[MAX_PID_STR_LENGTH]; \ + bfq_pid_to_str((bfqq)->pid, pid_str, MAX_PID_STR_LENGTH); \ blk_add_cgroup_trace_msg((bfqd)->queue, \ bfqg_to_blkg(bfqq_group(bfqq))->blkcg, \ - "bfq%d%c " fmt, (bfqq)->pid, \ + "bfq%s%c " fmt, pid_str, \ bfq_bfqq_sync((bfqq)) ? 'S' : 'A', ##args); \ } while (0) @@ -1033,10 +1045,13 @@ struct bfq_group *bfqq_group(struct bfq_queue *bfqq); #else /* CONFIG_BFQ_GROUP_IOSCHED */ -#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) \ - blk_add_trace_msg((bfqd)->queue, "bfq%d%c " fmt, (bfqq)->pid, \ +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) do { \ + char pid_str[MAX_PID_STR_LENGTH]; \ + bfq_pid_to_str((bfqq)->pid, pid_str, MAX_PID_STR_LENGTH); \ + blk_add_trace_msg((bfqd)->queue, "bfq%s%c " fmt, pid_str, \ bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ - ##args) + ##args) \ +} while (0) #define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do {} while (0) #endif /* CONFIG_BFQ_GROUP_IOSCHED */ From patchwork Sun Mar 10 18:11:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160030 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11090135jad; Sun, 10 Mar 2019 11:12:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqweKfNALvIlYNkzciiDbbi5GGv8ns71aRT3aq37UiE/7I3MYbS/TBL/AyjJB12AEf+hgmw4 X-Received: by 2002:a65:64d5:: with SMTP id t21mr26063346pgv.266.1552241552556; Sun, 10 Mar 2019 11:12:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241552; cv=none; d=google.com; s=arc-20160816; b=iY/7rd/Ft+JTOpGu9HeQKZup2H4Dbq3cSsMCqeGYeq8y9ZdiRUttHEAZQ9zOOTjeow kdYNVEB3vwfVnGE0ptBZIa47k42/NvTaAOhzhvJM//mQfVVtOSJbMDGu6fZoLMpxIqlr 8V0IOF78o4VMQLwMCXhSxwHAWjkmyTqk+BNsv6J6480tSlwgb6VeS4dp0q2MxN4X+M0U 7n7T5NGtlcVpEolFC3ULKWihV/h2EVar12DKb0qFfKSn/SuDX2+TwAzYHPhkK7QD2W0N g8N6HMrb+9lEc2d8+OLVqHsnAGcJhz/CVP4m8zbKGe6Frumg0q5CV29fFgXPiRiygiy6 gI3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=l1TfFyjmju0trauNzj8mLdRPLJSW2/vzxX4W6lPtYQU=; b=xV3KyKpMzSYKMiKbRGC5SdYTzxgq12THruQHI94E9XWO/18QvgJMuZEH2Z/0cnp1/9 UsMBVUPt7TjjY9uEzDQTV4RweMn+0pa+EnvyRB1k4gdrKcHlewxPy8vwZ34oJmnrJozB BO/skHAkKQpXr2O3dohGGw4CbRO8FOW7R5Z3I7bQ8r7yjT2kY5uHj8iXJ8VBu3umi4G9 /zVRgDtw7BjCyyXZTehAcPsVBEyZ2YtVMdl8bSmHNsWJdXF22NIst+LTl5IFPECVyICO XtBAca+Sd+ZVo4VuZJLxLXA8TNtxvLJkLTV0LUWfZbjFrizHD8NWFhWDEyJuIW+82Hcg 4ntw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K1MI6x7r; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g19si3134476pgk.300.2019.03.10.11.12.32; Sun, 10 Mar 2019 11:12:32 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K1MI6x7r; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726993AbfCJSMb (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:31 -0400 Received: from mail-wr1-f66.google.com ([209.85.221.66]:45159 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726963AbfCJSM2 (ORCPT ); Sun, 10 Mar 2019 14:12:28 -0400 Received: by mail-wr1-f66.google.com with SMTP id o7so2622599wrp.12 for ; Sun, 10 Mar 2019 11:12:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l1TfFyjmju0trauNzj8mLdRPLJSW2/vzxX4W6lPtYQU=; b=K1MI6x7r3euOnbBHikS92atb94FGWXGGScoBwetmCeXSMUrKI1PH3sC+IShkd/WY3f 2SqVuUnAZ1m5d0KfSuxwS/SB+pldGFkmA/2rlvw9FhRRwOj7Maq9RFsSN0hOnoT41MV+ 21ffC/1GjDBd5l8UR6JxhLVxTSgLBokO9FwrPnZ7ymK+4geUdmpoaaVekP7qtfGgRVvf 9Xhq6J8Tw5bWYagtmkcpF6TJLbXIAPxdsMpyJW8Ok7YeU2nskO2Wx/t0hisq2f4HypEm Tl320rPWZWNj/YkId116uWGHmeBh+RlPgQLpVCzQL4ds6GXqG8odFFi8E5bOPAagrE2B hBKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l1TfFyjmju0trauNzj8mLdRPLJSW2/vzxX4W6lPtYQU=; b=pHqqFYiXbk78/LjHGKoL+HEh9iJP+QNu8m03qc0ol1cqzPx5ATuMfAubsqg66h/k+U Y4ZUA54VAWwKAHVw0WWGBZX7aQ7F02AdCe2NLeb5DvsZifefNTSsbi7M8HJvj63dt/hC eEOKhbXtFti3E+m65XdCm0c43zsELqUGRk+eXcNcCeV+1Emcm8NgSBYSMz9QmDryOQZ9 Gv131uFfgjphI9zmY+GIElhjzmAPu1cruakwsxTx5KVNZV9/w6vm5v5fisqTizZvv8ST /FbkLxImD8zw/2sB3GQlMCBq4eWkOxCL0l5bWSfeD3BnfDzQHXODOZKSjku7hpEBgec5 hNMw== X-Gm-Message-State: APjAAAW7Q/Yu1cM8Mlev7z6TLnsj0lF6etwoGfgByRChg75OGhSM7bg0 ysQ3cP3jNkPJfYdd5be7EpcYOA== X-Received: by 2002:adf:e442:: with SMTP id t2mr4773085wrm.256.1552241546350; Sun, 10 Mar 2019 11:12:26 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:25 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 8/9] block, bfq: save & resume weight on a queue merge/split Date: Sun, 10 Mar 2019 19:11:36 +0100 Message-Id: <20190310181137.2604-9-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Francesco Pollicino bfq saves the state of a queue each time a merge occurs, to be able to resume such a state when the queue is associated again with its original process, on a split. Unfortunately bfq does not save & restore also the weight of the queue. If the weight is not correctly resumed when the queue is recycled, then the weight of the recycled queue could differ from the weight of the original queue. This commit adds the missing save & resume of the weight. Signed-off-by: Francesco Pollicino Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 2 ++ block/bfq-iosched.h | 9 +++++++++ 2 files changed, 11 insertions(+) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 7d95d9c01036..1712d12340c0 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -1028,6 +1028,7 @@ bfq_bfqq_resume_state(struct bfq_queue *bfqq, struct bfq_data *bfqd, else bfq_clear_bfqq_IO_bound(bfqq); + bfqq->entity.new_weight = bic->saved_weight; bfqq->ttime = bic->saved_ttime; bfqq->wr_coeff = bic->saved_wr_coeff; bfqq->wr_start_at_switch_to_srt = bic->saved_wr_start_at_switch_to_srt; @@ -2502,6 +2503,7 @@ static void bfq_bfqq_save_state(struct bfq_queue *bfqq) if (!bic) return; + bic->saved_weight = bfqq->entity.orig_weight; bic->saved_ttime = bfqq->ttime; bic->saved_has_short_ttime = bfq_bfqq_has_short_ttime(bfqq); bic->saved_IO_bound = bfq_bfqq_IO_bound(bfqq); diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 6410cc9a064d..1e34cce59ba7 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -404,6 +404,15 @@ struct bfq_io_cq { */ bool was_in_burst_list; + /* + * Save the weight when a merge occurs, to be able + * to restore it in case of split. If the weight is not + * correctly resumed when the queue is recycled, + * then the weight of the recycled queue could differ + * from the weight of the original queue. + */ + unsigned int saved_weight; + /* * Similar to previous fields: save wr information. */ From patchwork Sun Mar 10 18:11:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 160032 Delivered-To: patch@linaro.org Received: by 2002:a02:5cc1:0:0:0:0:0 with SMTP id w62csp11090234jad; Sun, 10 Mar 2019 11:12:40 -0700 (PDT) X-Google-Smtp-Source: APXvYqzgmVS6NCsETr23jV0EkSK3oFFKiAeDNyddYFIXoBvYy39yVKP5h9o23WEKBQkJJXjgJCs5 X-Received: by 2002:a17:902:76c7:: with SMTP id j7mr29249402plt.121.1552241560646; Sun, 10 Mar 2019 11:12:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552241560; cv=none; d=google.com; s=arc-20160816; b=o+L5W7Xw5jCKxGZAfUf0C/rKDuEZJVdKriKCso0709fruG24jyQc97J4IEMiGVAHvA WVEOg6H5MpHahBgKO8fhXxUFdDWDbxsEDtMj5T+oV51u4eheqgybOLx0oFcFvrL92bsM OY4VkIYk5ugsXPU9Pr2n7Vp2JtKfr7mnm1bIjDxEa5egirdDimwhYpi6c9vdP2O2VLOy 2uySWs9vTZpaBqorX8i9kOLzQBrwNpUChVE11agL+gGqz6cMQDqLEWsjtctvGPqbxFon h7dpBmk6SxFf8kaZAVcJgq+TVkgpTdsSBOjxhQKygYHlig8LB/UfpWUZCG9CY+HwowjM qWwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aRWEDuQLWzp3URwu3pEoQcxfbAgAvx//BghTXCmLhvY=; b=CuJIvasZFRncOspchor4LjG+yf/+kRGRP8CED7j+Nd5PCZUH5xDILlfr/tcV19haTp ZkdOcyk3LvIodtLO5gQ+aA0/6013T4BdNHbN1/syB9gOOrX1+O6rVQgqPkhN/5XeKkQl S6HTj/2t7vi2XBD0p0DI9PObzgaGLP3IcIg+xdPTt3CQcZkvjaeVPIskMf+0eY1hYk5y V2Z6vf5GFAMOf6fd0z0Dsv75lk7WjIYJq82O6cbFOaRbxvixXp2aNMPnybx0x6sm20Ng Mn5rhrI9UZIM3j0Lo6TVVSwes6y475ilX5wHbhWowtHDqBnGE/m+6DO9dsiNPQ3tGibv /zCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DA9QbhuH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x1si3521239plb.245.2019.03.10.11.12.40; Sun, 10 Mar 2019 11:12:40 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DA9QbhuH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727002AbfCJSMi (ORCPT + 31 others); Sun, 10 Mar 2019 14:12:38 -0400 Received: from mail-wr1-f66.google.com ([209.85.221.66]:39524 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726977AbfCJSMa (ORCPT ); Sun, 10 Mar 2019 14:12:30 -0400 Received: by mail-wr1-f66.google.com with SMTP id p8so1037006wrq.6 for ; Sun, 10 Mar 2019 11:12:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aRWEDuQLWzp3URwu3pEoQcxfbAgAvx//BghTXCmLhvY=; b=DA9QbhuHoXNY65XygCjZRskU4pHXsZRtYwhMZ/ppKrLRY/cZUw5C4SP4NKHgtrnia7 L43N9yaqpHuWrBzOBfWEMHKaUwscw6PAOBAYRF4798nsZUPm8hoPSOZOES6zrjRFsWEZ GoM2YLMmYNNXIfu5pDkAR1BuFNWOpqwyL7E+AcVpqEmFCOcBNgfi2cbmn1eKhQM0HcT4 E1NSIMlbtPfDZrYq5fmyB6e7IvD+LnKiLIhX0PE3l6lZcAD5Nqnfe/KC0ysd22m0LNGG rSvxvu+j84rLlnC5+sRmeq/gu5qweWtxUiqQ71Q67WsH3+qPUEV/jysG1IoYUoXS4got VOYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aRWEDuQLWzp3URwu3pEoQcxfbAgAvx//BghTXCmLhvY=; b=C+PriE/6kfpXzp/D/nUhFehiVJcWFrBxxqvpY9NlWlnRRvmxKXBFDcCPGeku86UMAC US90l1TcYQ4pNLd0IgmI3+PdzdsvWSu1A14S5WCn18+cVZubGNAJkW/rOCZNwh7CzanZ 3MPHup/xSEIplgo4m9YeLV2IzG4P/kFVHGKfVKNsFsAKyqamgp7Rw8binrCyFTZNg0g2 ykG0NWpx2vdNFYk95e9uClNT7MWQy4ei6WlSlkqr7SgJOeUICdxAXY9+wICfmLWFysuF 88cBoZIiDtXdvzx3aGt3OzIDdUtaJWFVge7XETt8PS6LT8zvQZPPOsSDQjPQ0fwnj2Bw 8QvA== X-Gm-Message-State: APjAAAXGlfeWV7LmjgWo41SVbr9Jhaa1c3NDOgsL85Sh5sylcRF5gAnZ Mg5aT3pN5e5RUWbphlQS/1bHXw== X-Received: by 2002:adf:d84c:: with SMTP id k12mr8880621wrl.58.1552241548497; Sun, 10 Mar 2019 11:12:28 -0700 (PDT) Received: from localhost.localdomain (146-241-67-113.dyn.eolo.it. [146.241.67.113]) by smtp.gmail.com with ESMTPSA id d206sm24906368wmc.11.2019.03.10.11.12.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Mar 2019 11:12:28 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, fra.fra.800@gmail.com, alessio.masola@gmail.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT V2 9/9] doc, block, bfq: add information on bfq execution time Date: Sun, 10 Mar 2019 19:11:37 +0100 Message-Id: <20190310181137.2604-10-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190310181137.2604-1-paolo.valente@linaro.org> References: <20190310181137.2604-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The execution time of BFQ has been slightly lowered. Report the new execution time in BFQ documentation. Signed-off-by: Paolo Valente --- Documentation/block/bfq-iosched.txt | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) -- 2.20.1 diff --git a/Documentation/block/bfq-iosched.txt b/Documentation/block/bfq-iosched.txt index 98a8dd5ee385..1a0f2ac02eb6 100644 --- a/Documentation/block/bfq-iosched.txt +++ b/Documentation/block/bfq-iosched.txt @@ -20,13 +20,26 @@ for that device, by setting low_latency to 0. See Section 3 for details on how to configure BFQ for the desired tradeoff between latency and throughput, or on how to maximize throughput. -BFQ has a non-null overhead, which limits the maximum IOPS that a CPU -can process for a device scheduled with BFQ. To give an idea of the -limits on slow or average CPUs, here are, first, the limits of BFQ for -three different CPUs, on, respectively, an average laptop, an old -desktop, and a cheap embedded system, in case full hierarchical -support is enabled (i.e., CONFIG_BFQ_GROUP_IOSCHED is set), but -CONFIG_DEBUG_BLK_CGROUP is not set (Section 4-2): +As every I/O scheduler, BFQ adds some overhead to per-I/O-request +processing. To give an idea of this overhead, the total, +single-lock-protected, per-request processing time of BFQ---i.e., the +sum of the execution times of the request insertion, dispatch and +completion hooks---is, e.g., 1.9 us on an Intel Core i7-2760QM@2.40GHz +(dated CPU for notebooks; time measured with simple code +instrumentation, and using the throughput-sync.sh script of the S +suite [1], in performance-profiling mode). To put this result into +context, the total, single-lock-protected, per-request execution time +of the lightest I/O scheduler available in blk-mq, mq-deadline, is 0.7 +us (mq-deadline is ~800 LOC, against ~10500 LOC for BFQ). + +Scheduling overhead further limits the maximum IOPS that a CPU can +process (already limited by the execution of the rest of the I/O +stack). To give an idea of the limits with BFQ, on slow or average +CPUs, here are, first, the limits of BFQ for three different CPUs, on, +respectively, an average laptop, an old desktop, and a cheap embedded +system, in case full hierarchical support is enabled (i.e., +CONFIG_BFQ_GROUP_IOSCHED is set), but CONFIG_DEBUG_BLK_CGROUP is not +set (Section 4-2): - Intel i7-4850HQ: 400 KIOPS - AMD A8-3850: 250 KIOPS - ARM CortexTM-A53 Octa-core: 80 KIOPS @@ -566,3 +579,5 @@ applications. Unset this tunable if you need/want to control weights. Slightly extended version: http://algogroup.unimore.it/people/paolo/disk_sched/bfq-v1-suite- results.pdf + +[3] https://github.com/Algodev-github/S