| Message ID | 20170411134315.44135-1-paolo.valente@linaro.org |
|---|---|
| Headers | show |
| Series | Introduce the BFQ I/O scheduler | expand |
On Tue, 2017-04-11 at 15:42 +0200, Paolo Valente wrote:
> new patch series, addressing (both) issues raised by Bart [1].
Hello Paolo,
Is there a git tree available somewhere with these patches and without
the single queue BFQ scheduler?
Thanks,
Bart.
> Il giorno 11 apr 2017, alle ore 16:37, Bart Van Assche <bart.vanassche@sandisk.com> ha scritto: > > On Tue, 2017-04-11 at 15:42 +0200, Paolo Valente wrote: >> new patch series, addressing (both) issues raised by Bart [1]. > > Hello Paolo, > > Is there a git tree available somewhere with these patches and without > the single queue BFQ scheduler? > Just pushed: https://github.com/Algodev-github/bfq-mq/tree/add-bfq-mq-logical Thanks, Paolo > Thanks, > > Bart.
On Tue, 2017-04-11 at 19:37 +0200, Paolo Valente wrote: > Just pushed: > https://github.com/Algodev-github/bfq-mq/tree/add-bfq-mq-logical Thanks! But are you aware that the code on that branch doesn't build? $ make all [ ... ] ERROR: "bfq_mark_bfqq_busy" [block/bfq-wf2q.ko] undefined! ERROR: "bfqg_stats_update_dequeue" [block/bfq-wf2q.ko] undefined! [ ... ] $ PAGER= git grep bfq_mark_bfqq_busy block/bfq-wf2q.c: bfq_mark_bfqq_busy(bfqq); Bart.
> Il giorno 11 apr 2017, alle ore 20:31, Bart Van Assche <bart.vanassche@sandisk.com> ha scritto: > > On Tue, 2017-04-11 at 19:37 +0200, Paolo Valente wrote: >> Just pushed: >> https://github.com/Algodev-github/bfq-mq/tree/add-bfq-mq-logical > > Thanks! > > But are you aware that the code on that branch doesn't build? > > $ make all > [ ... ] > ERROR: "bfq_mark_bfqq_busy" [block/bfq-wf2q.ko] undefined! > ERROR: "bfqg_stats_update_dequeue" [block/bfq-wf2q.ko] undefined! > [ ... ] > > $ PAGER= git grep bfq_mark_bfqq_busy > block/bfq-wf2q.c: bfq_mark_bfqq_busy(bfqq); > That's exactly the complain of the kbuild test robot. As I wrote, build completes with no problem in my test system (Ubuntu 16.04, gcc 5.4.0), even with the exact offending tree and .config that the robot reports. I didn't understand what is going on. In your case, as well as for the test robot, the compilation of the file block/bfq-wf2q.c as a module component fails, because that file does not contain the definition of the reported functions. But that definition is (uniquely) in the file block/bfq-iosched.c, which is to be compiled with the former file, according to the following rule in block/Makefile: obj-$(CONFIG_IOSCHED_BFQ) += bfq-iosched.o bfq-wf2q.o bfq-cgroup.o I have tried all combinations of configurations for bfq (builti-in or module, with or without cgrousp support), always successfully. If it makes any sense to share this information, these are the exact commands I used to test al combinations (in addition to make full builds in some cases, and try make all as in your case): make O=builddir M=block and make O=builddir M=block modules Where is my mistake? Thanks, Paolo > Bart.
On Wed, 2017-04-12 at 08:01 +0200, Paolo Valente wrote:
> Where is my mistake?
I think in the Makefile. How about the patch below? Please note that I'm no
Kbuild expert.diff --git a/block/Makefile b/block/Makefile
index 546066ee7fa6..b3711af6b637 100644
--- a/block/Makefile
+++ b/block/Makefile
@@ -20,7 +20,8 @@ obj-$(CONFIG_IOSCHED_NOOP) += noop-iosched.o
obj-$(CONFIG_IOSCHED_DEADLINE) += deadline-iosched.o
obj-$(CONFIG_IOSCHED_CFQ) += cfq-iosched.o
obj-$(CONFIG_MQ_IOSCHED_DEADLINE) += mq-deadline.o
-obj-$(CONFIG_IOSCHED_BFQ) += bfq-iosched.o bfq-wf2q.o bfq-cgroup.o
+bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o
+obj-$(CONFIG_IOSCHED_BFQ) += bfq.o
obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o
obj-$(CONFIG_BLK_CMDLINE_PARSER) += cmdline-parser.o
> Il giorno 12 apr 2017, alle ore 17:30, Bart Van Assche <Bart.VanAssche@sandisk.com> ha scritto: > > On Wed, 2017-04-12 at 08:01 +0200, Paolo Valente wrote: >> Where is my mistake? > > I think in the Makefile. How about the patch below? Please note that I'm no > Kbuild expert. > Thank you very much for finding and fixing the bug. I was working exactly on that, and had got to the same solution (which I guess is the only correct one). I'll apply these changes and resubmit. Thanks, Paolo > diff --git a/block/Makefile b/block/Makefile > index 546066ee7fa6..b3711af6b637 100644 > --- a/block/Makefile > +++ b/block/Makefile > @@ -20,7 +20,8 @@ obj-$(CONFIG_IOSCHED_NOOP) += noop-iosched.o > obj-$(CONFIG_IOSCHED_DEADLINE) += deadline-iosched.o > obj-$(CONFIG_IOSCHED_CFQ) += cfq-iosched.o > obj-$(CONFIG_MQ_IOSCHED_DEADLINE) += mq-deadline.o > -obj-$(CONFIG_IOSCHED_BFQ) += bfq-iosched.o bfq-wf2q.o bfq-cgroup.o > +bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o > +obj-$(CONFIG_IOSCHED_BFQ) += bfq.o > obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o > obj-$(CONFIG_BLK_CMDLINE_PARSER) += cmdline-parser.o >
On 11.04.2017, Paolo Valente wrote:
> new patch series, addressing (both) issues raised by Bart [1].
I'm doing a lot of automatic video transcoding in order to get my
collection of homemade videos down to an acceptable size (mainly
landscapes and boats all over the Norwegian west coast, taken with an old
cam that only produces uncompressed files). This process
involves heavy permanent writing to disk, often over a period of 10
min and more. When this happens, the whole system is kind of
unresponsive. I'm running Fedora 25, but with a self-customised kernel
that is fully low-latency, and the machine is a quadcore Intel Xeon
which should have enough power (Intel(R) Xeon(R) CPU E3-1241 v3 @
3.50GHz).
Using plain blk-mq, the system is very sluggish when there is heavy
disk writing, and it can take up to several minutes (up to the point
where the disk writing actually finishes) to start programs like gimp
or Libreoffice. In fact, when I click on the "applications" button
within XFCE, it can take a long time before the window even opens.
I played with deadline-mq too, and the situation remains the same
unless I do some heavy tuning like this:
echo "mq-deadline" > /sys/block/nvme0n1/queue/scheduler
echo "1" > /sys/block/nvme0n1/queue/iosched/fifo_batch
echo "4" > /sys/block/nvme0n1/queue/iosched/writes_starved
echo "100" > /sys/block/nvme0n1/queue/iosched/read_expire
echo "2000" > /sys/block/nvme0n1/queue/iosched/write_expire
With deadline-mq tuned like this, overall responsiveness is a little bit
better, but not nearly as good as when using bfq. With plain bfq, no
tuning is needed. The system is no longer sluggish. Any program starts
within seconds, and all is very much responsive. Max throughput isn't
important to me, the nvme "harddisk" is fast enough that some MB/s
more or less do not really matter.
[root@chiara ~]# lspci -v | grep -i nvme
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe
SSD Controller SM951/PM951 (rev 01) (prog-if 02 [NVM Express])
Kernel driver in use: nvme
Kernel modules: nvme
As an end-user with no relevant programming skills to be able to
contribute, I would wish that developers would combine their forces and
help Paolo to get bfq into the kernel and to make bfq even better.
Thanks,
Heinz