| Message ID | 1548738199-9403-1-git-send-email-omair.javaid@linaro.org |
|---|---|
| Headers | show |
| Series | Support for Linux kernel thread aware debug | expand |
On 1/28/19 9:03 PM, Omair Javaid wrote: > This patch series implements linux kernel target which allows linux kernel > tasks to be viewed as GDB threads. We have had multiple set of patches > submitted over last few years aiming to insert add-ons into GDB for debugging > Linux kernel. This patch series builds on top of Linux kernel infrastructure > submitted by Philipp Rudo in his various sets of patches aiming to debug > Linux kernel dumps. I just have one minor suggestion / comment about file names. I maintain FreeBSD kernel patches for gdb out-of-tree (for various reasons), and those patches use some similar things (e.g. a different OSABI). My comment has to do with the filenames. Other osabi-specific files tend to use more verbose names such as 'linux-arm-nat.c'. I wonder if it makes sense to spell out linux here as well. I have been using 'arm-fbsd-kern.c' as a complement to 'arm-fbsd-tdep.c' for the kernel gdbarch. The architecture independent files follow the patter 'fbsd-k*.c' (e.g. fbsd-kld.c for modules and fbsd-kthr.c for thread enumeration), but I would be happy to move those to something like 'fbsd-kern-ld.c' and 'fbsd-kern-thr.c'. For your current patchset that might mean something like 'linux-kern-tdep.c' instead of 'lk-tdep.c'. I would also be fine with 'arm-linux-kern-tdep.c' instead of 'arm-linux-kern.c' perhaps if other folks feel like that is more consistent. -- John Baldwin
On Tue, Jan 29 2019, John Baldwin wrote: > I just have one minor suggestion / comment about file names. I maintain > FreeBSD kernel patches for gdb out-of-tree (for various reasons), and those > patches use some similar things (e.g. a different OSABI). My comment has > to do with the filenames. Other osabi-specific files tend to use more > verbose names such as 'linux-arm-nat.c'. I wonder if it makes sense to > spell out linux here as well. I have been using 'arm-fbsd-kern.c' as a > complement to 'arm-fbsd-tdep.c' for the kernel gdbarch. The architecture > independent files follow the patter 'fbsd-k*.c' (e.g. fbsd-kld.c for modules > and fbsd-kthr.c for thread enumeration), but I would be happy to move those > to something like 'fbsd-kern-ld.c' and 'fbsd-kern-thr.c'. For your current > patchset that might mean something like 'linux-kern-tdep.c' instead of > 'lk-tdep.c'. I would also be fine with 'arm-linux-kern-tdep.c' instead of > 'arm-linux-kern.c' perhaps if other folks feel like that is more consistent. I believe it was me who originated the "lk" notation, so maybe I should provide my few cents on its rationale here: 1. The string "linux" in GDB source file names specifically means "GNU/Linux user-space runtime". For instance, "aarch64-linux-tdep.c" implements target-dependent code for GNU/Linux user-space programs on AArch64 hardware. Thus "linux" is reserved for this particular purpose, and collisions should better be avoided. 2. In principle, the Linux kernel runtime is just another runtime environment, like the GNU/Linux user-space runtime or the QNX Neutrino runtime (say). And since all runtimes are named by a single short word so far ("linux", "nbsd", "nto", etc.), it seems consistent to use the same scheme for the Linux kernel runtime as well. In particular this means not to use any dashes or underscores. 3. The runtime name becomes part of many identifiers. Any additional characters make these identifiers more bulky. Thus, in my opinion, the name should be as short as possible. If "lk" is not considered clear enough, alternatives could be something like "lxk", "lkr" (for "Linux kernel runtime"), "linuxk", or "linuxkr". For the FreeBSD kernel I suggest "fbsdk" or "fbsdkr". -- Andreas
On Tue, 29 Jan 2019 at 22:30, John Baldwin <jhb@freebsd.org> wrote: > > On 1/28/19 9:03 PM, Omair Javaid wrote: > > This patch series implements linux kernel target which allows linux kernel > > tasks to be viewed as GDB threads. We have had multiple set of patches > > submitted over last few years aiming to insert add-ons into GDB for debugging > > Linux kernel. This patch series builds on top of Linux kernel infrastructure > > submitted by Philipp Rudo in his various sets of patches aiming to debug > > Linux kernel dumps. > > I just have one minor suggestion / comment about file names. I maintain > FreeBSD kernel patches for gdb out-of-tree (for various reasons), and those > patches use some similar things (e.g. a different OSABI). My comment has > to do with the filenames. Other osabi-specific files tend to use more > verbose names such as 'linux-arm-nat.c'. I wonder if it makes sense to > spell out linux here as well. I have been using 'arm-fbsd-kern.c' as a > complement to 'arm-fbsd-tdep.c' for the kernel gdbarch. The architecture > independent files follow the patter 'fbsd-k*.c' (e.g. fbsd-kld.c for modules > and fbsd-kthr.c for thread enumeration), but I would be happy to move those > to something like 'fbsd-kern-ld.c' and 'fbsd-kern-thr.c'. For your current > patchset that might mean something like 'linux-kern-tdep.c' instead of > 'lk-tdep.c'. I would also be fine with 'arm-linux-kern-tdep.c' instead of > 'arm-linux-kern.c' perhaps if other folks feel like that is more consistent. Hi John, Andreas has aptly described the reason behind choosing the nomenclature of new linux kernel specific files as it is right now and i am open to any suggestion that my come up during reviews. Also I was wondering if you can share details of kernel debugging features which has implemented in your out of tree FreeBSD patches for GDB. Also share some insight on ways to test this patchset. Thanks! > > -- > John Baldwin > >
On 2/4/19 1:12 AM, Omair Javaid wrote: > On Tue, 29 Jan 2019 at 22:30, John Baldwin <jhb@freebsd.org> wrote: >> >> On 1/28/19 9:03 PM, Omair Javaid wrote: >>> This patch series implements linux kernel target which allows linux kernel >>> tasks to be viewed as GDB threads. We have had multiple set of patches >>> submitted over last few years aiming to insert add-ons into GDB for debugging >>> Linux kernel. This patch series builds on top of Linux kernel infrastructure >>> submitted by Philipp Rudo in his various sets of patches aiming to debug >>> Linux kernel dumps. >> >> I just have one minor suggestion / comment about file names. I maintain >> FreeBSD kernel patches for gdb out-of-tree (for various reasons), and those >> patches use some similar things (e.g. a different OSABI). My comment has >> to do with the filenames. Other osabi-specific files tend to use more >> verbose names such as 'linux-arm-nat.c'. I wonder if it makes sense to >> spell out linux here as well. I have been using 'arm-fbsd-kern.c' as a >> complement to 'arm-fbsd-tdep.c' for the kernel gdbarch. The architecture >> independent files follow the patter 'fbsd-k*.c' (e.g. fbsd-kld.c for modules >> and fbsd-kthr.c for thread enumeration), but I would be happy to move those >> to something like 'fbsd-kern-ld.c' and 'fbsd-kern-thr.c'. For your current >> patchset that might mean something like 'linux-kern-tdep.c' instead of >> 'lk-tdep.c'. I would also be fine with 'arm-linux-kern-tdep.c' instead of >> 'arm-linux-kern.c' perhaps if other folks feel like that is more consistent. > > Hi John, > > Andreas has aptly described the reason behind choosing the > nomenclature of new linux kernel specific files as it is right now and > i am open to any suggestion that my come up during reviews. Ok. I'm not firmly tied to a specific naming scheme, and I'll defer to whatever the global maintainers prefer. > Also I was wondering if you can share details of kernel debugging > features which has implemented in your out of tree FreeBSD patches for > GDB. Also share some insight on ways to test this patchset. So I don't have any formalized testing. :-/ For testing cross-debug of crash dumps I modified the libkvm library in FreeBSD's base system to support reading the kernel crash dump format of other architectures and can then generate a crash dump in a VM / qemu instance and copy it to a 64-bit x86 host to examine against a sysroot. However, most of my testing is just by using the kernel target as most of my work is FreeBSD kernel and userland stuff, so I exercise it that way. I do have various gdb scripts (still written in the ancient gdb script language from gdb 6.x rather than python) at github.com/bsdjhb/kdbg/tree/master/gdb They aren't generally useful on other kernels though and are specific to FreeBSD. In terms of features, the current FreeBSD patches support most FreeBSD architectures (sparc64 is untested and I haven't gotten stack traces on ppc to work right, though I think I can reuse what I just did for risc-v to fix ppc at some point). A few years ago I reworked the libkvm library in FreeBSD to support cross-debugging of crashdumps, so you can generally cross-debug crashdumps on FreeBSD. Currently this only works on a FreeBSD host because I haven't pulled out a portable version of libkvm that could be compiled on a non-FreeBSD host yet. It does support kernel modules by reusing the shared library interface. This is implemented by having a custom set of solib_ops that get attached to the kernel-specific gdbarch during that OSABI's init handler. It has some custom commands that are wrappers around 'thread N' that let you switch to a thread by process ID or kernel thread ID. The main things it requires of each architecture are a method for populating a regcache from a kernel thread register state ('struct pcb' on FreeBSD), and custom frame unwinders to walk across exception frames on the stack. Currently these are marked as signal frames so that GDB will unwind across them ok when you have a page fault in the kernel due to a NULL function pointer (other frame unwinders stop unwinding when they see a NULL pc, but signal frames can keep unwinding past those). I gave a talk a couple of years ago at a BSD conference about GDB that includes some kgdb slides. You can see the slides at https://people.freebsd.org/~jhb/papers/bsdcan/2016/slides.pdf if that is helpful. -- John Baldwin
On Mon, 4 Feb 2019 at 22:52, John Baldwin <jhb@freebsd.org> wrote: > > On 2/4/19 1:12 AM, Omair Javaid wrote: > > On Tue, 29 Jan 2019 at 22:30, John Baldwin <jhb@freebsd.org> wrote: > >> > >> On 1/28/19 9:03 PM, Omair Javaid wrote: > >>> This patch series implements linux kernel target which allows linux kernel > >>> tasks to be viewed as GDB threads. We have had multiple set of patches > >>> submitted over last few years aiming to insert add-ons into GDB for debugging > >>> Linux kernel. This patch series builds on top of Linux kernel infrastructure > >>> submitted by Philipp Rudo in his various sets of patches aiming to debug > >>> Linux kernel dumps. > >> > >> I just have one minor suggestion / comment about file names. I maintain > >> FreeBSD kernel patches for gdb out-of-tree (for various reasons), and those > >> patches use some similar things (e.g. a different OSABI). My comment has > >> to do with the filenames. Other osabi-specific files tend to use more > >> verbose names such as 'linux-arm-nat.c'. I wonder if it makes sense to > >> spell out linux here as well. I have been using 'arm-fbsd-kern.c' as a > >> complement to 'arm-fbsd-tdep.c' for the kernel gdbarch. The architecture > >> independent files follow the patter 'fbsd-k*.c' (e.g. fbsd-kld.c for modules > >> and fbsd-kthr.c for thread enumeration), but I would be happy to move those > >> to something like 'fbsd-kern-ld.c' and 'fbsd-kern-thr.c'. For your current > >> patchset that might mean something like 'linux-kern-tdep.c' instead of > >> 'lk-tdep.c'. I would also be fine with 'arm-linux-kern-tdep.c' instead of > >> 'arm-linux-kern.c' perhaps if other folks feel like that is more consistent. > > > > Hi John, > > > > Andreas has aptly described the reason behind choosing the > > nomenclature of new linux kernel specific files as it is right now and > > i am open to any suggestion that my come up during reviews. > > Ok. I'm not firmly tied to a specific naming scheme, and I'll defer to > whatever the global maintainers prefer. > > > Also I was wondering if you can share details of kernel debugging > > features which has implemented in your out of tree FreeBSD patches for > > GDB. Also share some insight on ways to test this patchset. > > So I don't have any formalized testing. :-/ For testing cross-debug of > crash dumps I modified the libkvm library in FreeBSD's base system to > support reading the kernel crash dump format of other architectures and > can then generate a crash dump in a VM / qemu instance and copy it to a > 64-bit x86 host to examine against a sysroot. However, most of my testing > is just by using the kernel target as most of my work is FreeBSD kernel > and userland stuff, so I exercise it that way. I do have various gdb scripts > (still written in the ancient gdb script language from gdb 6.x rather than > python) at github.com/bsdjhb/kdbg/tree/master/gdb They aren't generally > useful on other kernels though and are specific to FreeBSD. > > In terms of features, the current FreeBSD patches support most FreeBSD > architectures (sparc64 is untested and I haven't gotten stack traces on > ppc to work right, though I think I can reuse what I just did for risc-v > to fix ppc at some point). A few years ago I reworked the libkvm library > in FreeBSD to support cross-debugging of crashdumps, so you can generally > cross-debug crashdumps on FreeBSD. Currently this only works on a FreeBSD > host because I haven't pulled out a portable version of libkvm that could > be compiled on a non-FreeBSD host yet. It does support kernel modules by > reusing the shared library interface. This is implemented by having a > custom set of solib_ops that get attached to the kernel-specific gdbarch > during that OSABI's init handler. It has some custom commands that are > wrappers around 'thread N' that let you switch to a thread by process ID > or kernel thread ID. The main things it requires of each architecture are > a method for populating a regcache from a kernel thread register state > ('struct pcb' on FreeBSD), and custom frame unwinders to walk across > exception frames on the stack. Currently these are marked as signal frames > so that GDB will unwind across them ok when you have a page fault in the > kernel due to a NULL function pointer (other frame unwinders stop unwinding > when they see a NULL pc, but signal frames can keep unwinding past those). > > I gave a talk a couple of years ago at a BSD conference about GDB that > includes some kgdb slides. You can see the slides at > https://people.freebsd.org/~jhb/papers/bsdcan/2016/slides.pdf if that is > helpful. Thanks John. This is helpful. > > -- > John Baldwin > >
On Fri, 8 Feb 2019 at 13:54, Omair Javaid <omair.javaid@linaro.org> wrote: > > On Mon, 4 Feb 2019 at 22:52, John Baldwin <jhb@freebsd.org> wrote: > > > > On 2/4/19 1:12 AM, Omair Javaid wrote: > > > On Tue, 29 Jan 2019 at 22:30, John Baldwin <jhb@freebsd.org> wrote: > > >> > > >> On 1/28/19 9:03 PM, Omair Javaid wrote: > > >>> This patch series implements linux kernel target which allows linux kernel > > >>> tasks to be viewed as GDB threads. We have had multiple set of patches > > >>> submitted over last few years aiming to insert add-ons into GDB for debugging > > >>> Linux kernel. This patch series builds on top of Linux kernel infrastructure > > >>> submitted by Philipp Rudo in his various sets of patches aiming to debug > > >>> Linux kernel dumps. > > >> > > >> I just have one minor suggestion / comment about file names. I maintain > > >> FreeBSD kernel patches for gdb out-of-tree (for various reasons), and those > > >> patches use some similar things (e.g. a different OSABI). My comment has > > >> to do with the filenames. Other osabi-specific files tend to use more > > >> verbose names such as 'linux-arm-nat.c'. I wonder if it makes sense to > > >> spell out linux here as well. I have been using 'arm-fbsd-kern.c' as a > > >> complement to 'arm-fbsd-tdep.c' for the kernel gdbarch. The architecture > > >> independent files follow the patter 'fbsd-k*.c' (e.g. fbsd-kld.c for modules > > >> and fbsd-kthr.c for thread enumeration), but I would be happy to move those > > >> to something like 'fbsd-kern-ld.c' and 'fbsd-kern-thr.c'. For your current > > >> patchset that might mean something like 'linux-kern-tdep.c' instead of > > >> 'lk-tdep.c'. I would also be fine with 'arm-linux-kern-tdep.c' instead of > > >> 'arm-linux-kern.c' perhaps if other folks feel like that is more consistent. > > > > > > Hi John, > > > > > > Andreas has aptly described the reason behind choosing the > > > nomenclature of new linux kernel specific files as it is right now and > > > i am open to any suggestion that my come up during reviews. > > > > Ok. I'm not firmly tied to a specific naming scheme, and I'll defer to > > whatever the global maintainers prefer. > > > > > Also I was wondering if you can share details of kernel debugging > > > features which has implemented in your out of tree FreeBSD patches for > > > GDB. Also share some insight on ways to test this patchset. > > > > So I don't have any formalized testing. :-/ For testing cross-debug of > > crash dumps I modified the libkvm library in FreeBSD's base system to > > support reading the kernel crash dump format of other architectures and > > can then generate a crash dump in a VM / qemu instance and copy it to a > > 64-bit x86 host to examine against a sysroot. However, most of my testing > > is just by using the kernel target as most of my work is FreeBSD kernel > > and userland stuff, so I exercise it that way. I do have various gdb scripts > > (still written in the ancient gdb script language from gdb 6.x rather than > > python) at github.com/bsdjhb/kdbg/tree/master/gdb They aren't generally > > useful on other kernels though and are specific to FreeBSD. > > > > In terms of features, the current FreeBSD patches support most FreeBSD > > architectures (sparc64 is untested and I haven't gotten stack traces on > > ppc to work right, though I think I can reuse what I just did for risc-v > > to fix ppc at some point). A few years ago I reworked the libkvm library > > in FreeBSD to support cross-debugging of crashdumps, so you can generally > > cross-debug crashdumps on FreeBSD. Currently this only works on a FreeBSD > > host because I haven't pulled out a portable version of libkvm that could > > be compiled on a non-FreeBSD host yet. It does support kernel modules by > > reusing the shared library interface. This is implemented by having a > > custom set of solib_ops that get attached to the kernel-specific gdbarch > > during that OSABI's init handler. It has some custom commands that are > > wrappers around 'thread N' that let you switch to a thread by process ID > > or kernel thread ID. The main things it requires of each architecture are > > a method for populating a regcache from a kernel thread register state > > ('struct pcb' on FreeBSD), and custom frame unwinders to walk across > > exception frames on the stack. Currently these are marked as signal frames > > so that GDB will unwind across them ok when you have a page fault in the > > kernel due to a NULL function pointer (other frame unwinders stop unwinding > > when they see a NULL pc, but signal frames can keep unwinding past those). > > > > I gave a talk a couple of years ago at a BSD conference about GDB that > > includes some kgdb slides. You can see the slides at > > https://people.freebsd.org/~jhb/papers/bsdcan/2016/slides.pdf if that is > > helpful. > > Thanks John. This is helpful. > > > > > -- > > John Baldwin > > > > Ping! I am going to be reworking the current patch-set accommodating the suggestions and comments I have received so far. Kindly post your comments in next week or so if there are any further other ideas/suggestions. Thanks!