| Message ID | 20181211103616.22137-1-anders.roxell@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | dma-debug: fix soft lockup when a lot of debug are enabled | expand |
Hi Anders, On 11/12/2018 10:36, Anders Roxell wrote: > When running a kernel in qemu with enough debugging options (slub-debug, > ftrace, kcov, kasan, ubsan, ...) enabled together, that results in a > slow initcall. So a 'watchdog: BUG: soft lockup' happens: > > [ 44.105619] Call trace: > [ 44.106709] __slab_alloc+0x70/0x88 > [ 44.107757] kmem_cache_alloc_trace+0x13c/0x590 > [ 44.108764] prealloc_memory+0xb8/0x240 > [ 44.109768] dma_debug_init+0x174/0x1f0 > [ 44.110725] do_one_initcall+0x430/0x8b0 > [ 44.111719] do_initcall_level+0x548/0x5b4 > [ 44.112672] do_initcalls+0x28/0x4c > [ 44.113628] do_basic_setup+0x34/0x3c > [ 44.114559] kernel_init_freeable+0x194/0x24c > [ 44.115570] kernel_init+0x24/0x18c > [ 44.116498] ret_from_fork+0x10/0x18 > > Rework to call cond_resched(), in the function prealloc_memory()'s > for-loop. Funnily enough I've just done a bit of work in this area which should make allocation a lot more efficient - would you mind giving this series a spin on the same setup to see how much it helps the performance? https://lore.kernel.org/lkml/cover.1544447601.git.robin.murphy@arm.com/ Thanks, Robin. > > Signed-off-by: Anders Roxell <anders.roxell@linaro.org> > --- > kernel/dma/debug.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c > index 231ca4628062..2abdc265aec4 100644 > --- a/kernel/dma/debug.c > +++ b/kernel/dma/debug.c > @@ -767,6 +767,7 @@ static int prealloc_memory(u32 num_entries) > int i; > > for (i = 0; i < num_entries; ++i) { > + cond_resched(); > entry = kzalloc(sizeof(*entry), GFP_KERNEL); > if (!entry) > goto out_err; >
On Tue, 11 Dec 2018 at 12:51, Robin Murphy <robin.murphy@arm.com> wrote: > > Hi Anders, Hi Robin, > > On 11/12/2018 10:36, Anders Roxell wrote: > > When running a kernel in qemu with enough debugging options (slub-debug, > > ftrace, kcov, kasan, ubsan, ...) enabled together, that results in a > > slow initcall. So a 'watchdog: BUG: soft lockup' happens: > > > > [ 44.105619] Call trace: > > [ 44.106709] __slab_alloc+0x70/0x88 > > [ 44.107757] kmem_cache_alloc_trace+0x13c/0x590 > > [ 44.108764] prealloc_memory+0xb8/0x240 > > [ 44.109768] dma_debug_init+0x174/0x1f0 > > [ 44.110725] do_one_initcall+0x430/0x8b0 > > [ 44.111719] do_initcall_level+0x548/0x5b4 > > [ 44.112672] do_initcalls+0x28/0x4c > > [ 44.113628] do_basic_setup+0x34/0x3c > > [ 44.114559] kernel_init_freeable+0x194/0x24c > > [ 44.115570] kernel_init+0x24/0x18c > > [ 44.116498] ret_from_fork+0x10/0x18 > > > > Rework to call cond_resched(), in the function prealloc_memory()'s > > for-loop. > > Funnily enough I've just done a bit of work in this area which should > make allocation a lot more efficient - would you mind giving this series > a spin on the same setup to see how much it helps the performance? > > https://lore.kernel.org/lkml/cover.1544447601.git.robin.murphy@arm.com/ I tried it on todays next tag and couldn't reproduce the soft lockup. Cheers, Anders > > Thanks, > Robin. > > > > > Signed-off-by: Anders Roxell <anders.roxell@linaro.org> > > --- > > kernel/dma/debug.c | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c > > index 231ca4628062..2abdc265aec4 100644 > > --- a/kernel/dma/debug.c > > +++ b/kernel/dma/debug.c > > @@ -767,6 +767,7 @@ static int prealloc_memory(u32 num_entries) > > int i; > > > > for (i = 0; i < num_entries; ++i) { > > + cond_resched(); > > entry = kzalloc(sizeof(*entry), GFP_KERNEL); > > if (!entry) > > goto out_err; > >
diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 231ca4628062..2abdc265aec4 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -767,6 +767,7 @@ static int prealloc_memory(u32 num_entries) int i; for (i = 0; i < num_entries; ++i) { + cond_resched(); entry = kzalloc(sizeof(*entry), GFP_KERNEL); if (!entry) goto out_err;
When running a kernel in qemu with enough debugging options (slub-debug, ftrace, kcov, kasan, ubsan, ...) enabled together, that results in a slow initcall. So a 'watchdog: BUG: soft lockup' happens: [ 44.105619] Call trace: [ 44.106709] __slab_alloc+0x70/0x88 [ 44.107757] kmem_cache_alloc_trace+0x13c/0x590 [ 44.108764] prealloc_memory+0xb8/0x240 [ 44.109768] dma_debug_init+0x174/0x1f0 [ 44.110725] do_one_initcall+0x430/0x8b0 [ 44.111719] do_initcall_level+0x548/0x5b4 [ 44.112672] do_initcalls+0x28/0x4c [ 44.113628] do_basic_setup+0x34/0x3c [ 44.114559] kernel_init_freeable+0x194/0x24c [ 44.115570] kernel_init+0x24/0x18c [ 44.116498] ret_from_fork+0x10/0x18 Rework to call cond_resched(), in the function prealloc_memory()'s for-loop. Signed-off-by: Anders Roxell <anders.roxell@linaro.org> --- kernel/dma/debug.c | 1 + 1 file changed, 1 insertion(+) -- 2.19.2