| Message ID | 20241217-isolcpus-io-queues-v4-9-5d355fbb1e14@kernel.org |
|---|---|
| State | New |
| Headers | show |
| Series | blk: honor isolcpus configuration | expand |
On Tue, Dec 17, 2024 at 07:29:43PM +0100, Daniel Wagner wrote: > When we offlining a hardware context which also serves isolcpus mapped > to it, any IO issued by the isolcpus will stall as there is nothing > which handles the interrupts etc. > > This configuration/setup is not supported at this point thus just issue > a warning. > > Signed-off-by: Daniel Wagner <wagi@kernel.org> > --- > block/blk-mq.c | 43 ++++++++++++++++++++++++++++++++++++++++++- > 1 file changed, 42 insertions(+), 1 deletion(-) > > diff --git a/block/blk-mq.c b/block/blk-mq.c > index de15c0c76f874a2a863b05a23e0f3dba20cb6488..f9af0f5dd6aac8da855777acf2ffc61128f15a74 100644 > --- a/block/blk-mq.c > +++ b/block/blk-mq.c > @@ -3619,6 +3619,45 @@ static bool blk_mq_hctx_has_requests(struct blk_mq_hw_ctx *hctx) > return data.has_rq; > } > > +static void blk_mq_hctx_check_isolcpus_online(struct blk_mq_hw_ctx *hctx, unsigned int cpu) Please avoid the overly long line here. > +{ > + const struct cpumask *hk_mask; > + int i; > + > + if (!housekeeping_enabled(HK_TYPE_MANAGED_IRQ)) > + return; > + > + hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ); > + > + for (i = 0; i < hctx->nr_ctx; i++) { > + struct blk_mq_ctx *ctx = hctx->ctxs[i]; > + > + if (ctx->cpu == cpu) > + continue; > + > + /* > + * Check if this context has at least one online > + * housekeeping CPU in this case the hardware context is > + * usable. But here you;re not even using up all 80 characters for the comment.
On Tue, Dec 17, 2024 at 07:29:43PM +0100, Daniel Wagner wrote: > When we offlining a hardware context which also serves isolcpus mapped > to it, any IO issued by the isolcpus will stall as there is nothing > which handles the interrupts etc. > > This configuration/setup is not supported at this point thus just issue > a warning. As I mentioned on patch 8, this io hang is regression on existed applications which can work just fine with 'isolcpus=managed_irq'. Do you think the added warning will prevent people from complaining the regression? :-) Thanks, Ming
diff --git a/block/blk-mq.c b/block/blk-mq.c index de15c0c76f874a2a863b05a23e0f3dba20cb6488..f9af0f5dd6aac8da855777acf2ffc61128f15a74 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3619,6 +3619,45 @@ static bool blk_mq_hctx_has_requests(struct blk_mq_hw_ctx *hctx) return data.has_rq; } +static void blk_mq_hctx_check_isolcpus_online(struct blk_mq_hw_ctx *hctx, unsigned int cpu) +{ + const struct cpumask *hk_mask; + int i; + + if (!housekeeping_enabled(HK_TYPE_MANAGED_IRQ)) + return; + + hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ); + + for (i = 0; i < hctx->nr_ctx; i++) { + struct blk_mq_ctx *ctx = hctx->ctxs[i]; + + if (ctx->cpu == cpu) + continue; + + /* + * Check if this context has at least one online + * housekeeping CPU in this case the hardware context is + * usable. + */ + if (cpumask_test_cpu(ctx->cpu, hk_mask) && + cpu_online(ctx->cpu)) + break; + + /* + * The context doesn't have any online housekeeping CPUs + * but there might be an online isolated CPU mapped to + * it. + */ + if (cpu_is_offline(ctx->cpu)) + continue; + + pr_warn("%s: offlining hctx%d but there is still an online isolcpu CPU %d mapped to it, IO stalls expected\n", + hctx->queue->disk->disk_name, + hctx->queue_num, ctx->cpu); + } +} + static bool blk_mq_hctx_has_online_cpu(struct blk_mq_hw_ctx *hctx, unsigned int this_cpu) { @@ -3638,8 +3677,10 @@ static bool blk_mq_hctx_has_online_cpu(struct blk_mq_hw_ctx *hctx, continue; /* this hctx has at least one online CPU */ - if (this_cpu != cpu) + if (this_cpu != cpu) { + blk_mq_hctx_check_isolcpus_online(hctx, this_cpu); return true; + } } return false;
When we offlining a hardware context which also serves isolcpus mapped to it, any IO issued by the isolcpus will stall as there is nothing which handles the interrupts etc. This configuration/setup is not supported at this point thus just issue a warning. Signed-off-by: Daniel Wagner <wagi@kernel.org> --- block/blk-mq.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-)