| Message ID | 1587566123-9935-5-git-send-email-sumit.garg@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | arm64: Introduce new IPI as IPI_CALL_NMI_FUNC | expand |
On Wed, Apr 22, 2020 at 08:05:23PM +0530, Sumit Garg wrote: > arm64 platforms with GICv3 or later supports pseudo NMIs which can be > leveraged to round up CPUs which are stuck in hard lockup state with > interrupts disabled that wouldn't be possible with a normal IPI. > > So instead switch to round up CPUs using IPI_CALL_NMI_FUNC. And in > case a particular arm64 platform doesn't supports pseudo NMIs, > IPI_CALL_NMI_FUNC will act as a normal IPI which maintains existing > kgdb functionality. > > Also, one thing to note here is that with CPUs running in NMI context, > kernel has special handling for printk() which involves CPU specific > buffers and defering printk() until exit from NMI context. But with kgdb > we don't want to defer printk() especially backtrace on corresponding > CPUs. So switch to normal printk() context instead prior to entering > kgdb context. > > Signed-off-by: Sumit Garg <sumit.garg@linaro.org> > --- > arch/arm64/kernel/kgdb.c | 12 ++++++++++++ > arch/arm64/kernel/smp.c | 17 ++++++++++++++--- > 2 files changed, 26 insertions(+), 3 deletions(-) > > diff --git a/arch/arm64/kernel/kgdb.c b/arch/arm64/kernel/kgdb.c > index 4311992..d6be647 100644 > --- a/arch/arm64/kernel/kgdb.c > +++ b/arch/arm64/kernel/kgdb.c > @@ -14,6 +14,7 @@ > #include <linux/kgdb.h> > #include <linux/kprobes.h> > #include <linux/sched/task_stack.h> > +#include <linux/smp.h> > > #include <asm/debug-monitors.h> > #include <asm/insn.h> > @@ -353,3 +354,14 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) > return aarch64_insn_write((void *)bpt->bpt_addr, > *(u32 *)bpt->saved_instr); > } > + > +#ifdef CONFIG_SMP > +void kgdb_roundup_cpus(void) > +{ > + struct cpumask mask; > + > + cpumask_copy(&mask, cpu_online_mask); > + cpumask_clear_cpu(raw_smp_processor_id(), &mask); > + arch_send_call_nmi_func_ipi_mask(&mask); > +} > +#endif > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c > index 27c8ee1..c7158f6e8 100644 > --- a/arch/arm64/kernel/smp.c > +++ b/arch/arm64/kernel/smp.c > @@ -31,6 +31,7 @@ > #include <linux/of.h> > #include <linux/irq_work.h> > #include <linux/kexec.h> > +#include <linux/kgdb.h> > #include <linux/kvm_host.h> > > #include <asm/alternative.h> > @@ -976,9 +977,19 @@ void handle_IPI(int ipinr, struct pt_regs *regs) > /* Handle it as a normal interrupt if not in NMI context */ > if (!in_nmi()) > irq_enter(); > - > - /* nop, IPI handlers for special features can be added here. */ > - > +#ifdef CONFIG_KGDB > + if (atomic_read(&kgdb_active) != -1) { > + /* > + * For kgdb to work properly, we need printk to operate > + * in normal context. > + */ > + if (in_nmi()) > + printk_nmi_exit(); Is this how things work on x86 or is printk() interception broken on x86? Daniel. > + kgdb_nmicallback(raw_smp_processor_id(), regs); > + if (in_nmi()) > + printk_nmi_enter(); > + } > +#endif > if (!in_nmi()) > irq_exit(); > break; > -- > 2.7.4 >
On Thu, 23 Apr 2020 at 15:02, Daniel Thompson <daniel.thompson@linaro.org> wrote: > > On Wed, Apr 22, 2020 at 08:05:23PM +0530, Sumit Garg wrote: > > arm64 platforms with GICv3 or later supports pseudo NMIs which can be > > leveraged to round up CPUs which are stuck in hard lockup state with > > interrupts disabled that wouldn't be possible with a normal IPI. > > > > So instead switch to round up CPUs using IPI_CALL_NMI_FUNC. And in > > case a particular arm64 platform doesn't supports pseudo NMIs, > > IPI_CALL_NMI_FUNC will act as a normal IPI which maintains existing > > kgdb functionality. > > > > Also, one thing to note here is that with CPUs running in NMI context, > > kernel has special handling for printk() which involves CPU specific > > buffers and defering printk() until exit from NMI context. But with kgdb > > we don't want to defer printk() especially backtrace on corresponding > > CPUs. So switch to normal printk() context instead prior to entering > > kgdb context. > > > > Signed-off-by: Sumit Garg <sumit.garg@linaro.org> > > --- > > arch/arm64/kernel/kgdb.c | 12 ++++++++++++ > > arch/arm64/kernel/smp.c | 17 ++++++++++++++--- > > 2 files changed, 26 insertions(+), 3 deletions(-) > > > > diff --git a/arch/arm64/kernel/kgdb.c b/arch/arm64/kernel/kgdb.c > > index 4311992..d6be647 100644 > > --- a/arch/arm64/kernel/kgdb.c > > +++ b/arch/arm64/kernel/kgdb.c > > @@ -14,6 +14,7 @@ > > #include <linux/kgdb.h> > > #include <linux/kprobes.h> > > #include <linux/sched/task_stack.h> > > +#include <linux/smp.h> > > > > #include <asm/debug-monitors.h> > > #include <asm/insn.h> > > @@ -353,3 +354,14 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) > > return aarch64_insn_write((void *)bpt->bpt_addr, > > *(u32 *)bpt->saved_instr); > > } > > + > > +#ifdef CONFIG_SMP > > +void kgdb_roundup_cpus(void) > > +{ > > + struct cpumask mask; > > + > > + cpumask_copy(&mask, cpu_online_mask); > > + cpumask_clear_cpu(raw_smp_processor_id(), &mask); > > + arch_send_call_nmi_func_ipi_mask(&mask); > > +} > > +#endif > > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c > > index 27c8ee1..c7158f6e8 100644 > > --- a/arch/arm64/kernel/smp.c > > +++ b/arch/arm64/kernel/smp.c > > @@ -31,6 +31,7 @@ > > #include <linux/of.h> > > #include <linux/irq_work.h> > > #include <linux/kexec.h> > > +#include <linux/kgdb.h> > > #include <linux/kvm_host.h> > > > > #include <asm/alternative.h> > > @@ -976,9 +977,19 @@ void handle_IPI(int ipinr, struct pt_regs *regs) > > /* Handle it as a normal interrupt if not in NMI context */ > > if (!in_nmi()) > > irq_enter(); > > - > > - /* nop, IPI handlers for special features can be added here. */ > > - > > +#ifdef CONFIG_KGDB > > + if (atomic_read(&kgdb_active) != -1) { > > + /* > > + * For kgdb to work properly, we need printk to operate > > + * in normal context. > > + */ > > + if (in_nmi()) > > + printk_nmi_exit(); > > Is this how things work on x86 or is printk() interception broken on > x86? After looking at x86 code, it seems like printk() interception in kgdb should be broken [1] after following commit as it follows the common sequence for nmi_(enter/exit) here [2]: commit 42a0bb3f71383b457a7db362f1c69e7afb96732b Author: Petr Mladek <pmladek@suse.com> Date: Fri May 20 17:00:33 2016 -0700 printk/nmi: generic solution for safe printk in NMI Although it may be that I have overlooked some specific x86 bits. [1] Here by broken I mean if I issue the "btc" command on the kdb console, I will get to see backtrace on CPUs rounded up via NMI when they exit kgdb (from NMI context) using deferred printk. [2] https://github.com/torvalds/linux/blob/master/arch/x86/kernel/nmi.c#L537 -Sumit > > > Daniel. > > > + kgdb_nmicallback(raw_smp_processor_id(), regs); > > + if (in_nmi()) > > + printk_nmi_enter(); > > + } > > +#endif > > if (!in_nmi()) > > irq_exit(); > > break; > > -- > > 2.7.4 > >
diff --git a/arch/arm64/kernel/kgdb.c b/arch/arm64/kernel/kgdb.c index 4311992..d6be647 100644 --- a/arch/arm64/kernel/kgdb.c +++ b/arch/arm64/kernel/kgdb.c @@ -14,6 +14,7 @@ #include <linux/kgdb.h> #include <linux/kprobes.h> #include <linux/sched/task_stack.h> +#include <linux/smp.h> #include <asm/debug-monitors.h> #include <asm/insn.h> @@ -353,3 +354,14 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) return aarch64_insn_write((void *)bpt->bpt_addr, *(u32 *)bpt->saved_instr); } + +#ifdef CONFIG_SMP +void kgdb_roundup_cpus(void) +{ + struct cpumask mask; + + cpumask_copy(&mask, cpu_online_mask); + cpumask_clear_cpu(raw_smp_processor_id(), &mask); + arch_send_call_nmi_func_ipi_mask(&mask); +} +#endif diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 27c8ee1..c7158f6e8 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -31,6 +31,7 @@ #include <linux/of.h> #include <linux/irq_work.h> #include <linux/kexec.h> +#include <linux/kgdb.h> #include <linux/kvm_host.h> #include <asm/alternative.h> @@ -976,9 +977,19 @@ void handle_IPI(int ipinr, struct pt_regs *regs) /* Handle it as a normal interrupt if not in NMI context */ if (!in_nmi()) irq_enter(); - - /* nop, IPI handlers for special features can be added here. */ - +#ifdef CONFIG_KGDB + if (atomic_read(&kgdb_active) != -1) { + /* + * For kgdb to work properly, we need printk to operate + * in normal context. + */ + if (in_nmi()) + printk_nmi_exit(); + kgdb_nmicallback(raw_smp_processor_id(), regs); + if (in_nmi()) + printk_nmi_enter(); + } +#endif if (!in_nmi()) irq_exit(); break;
arm64 platforms with GICv3 or later supports pseudo NMIs which can be leveraged to round up CPUs which are stuck in hard lockup state with interrupts disabled that wouldn't be possible with a normal IPI. So instead switch to round up CPUs using IPI_CALL_NMI_FUNC. And in case a particular arm64 platform doesn't supports pseudo NMIs, IPI_CALL_NMI_FUNC will act as a normal IPI which maintains existing kgdb functionality. Also, one thing to note here is that with CPUs running in NMI context, kernel has special handling for printk() which involves CPU specific buffers and defering printk() until exit from NMI context. But with kgdb we don't want to defer printk() especially backtrace on corresponding CPUs. So switch to normal printk() context instead prior to entering kgdb context. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> --- arch/arm64/kernel/kgdb.c | 12 ++++++++++++ arch/arm64/kernel/smp.c | 17 ++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) -- 2.7.4