diff mbox series

tty/sysrq: Dump kernel ring buffer messages via sysrq

Message ID 20231221133953.1507021-1-sreenath.vijayan@sony.com
State Superseded
Headers show
Series tty/sysrq: Dump kernel ring buffer messages via sysrq | expand

Commit Message

Sreenath Vijayan Dec. 21, 2023, 1:39 p.m. UTC 
When terminal is unresponsive, one cannot use dmesg to view kernel
ring buffer messages. Also, syslog services may be disabled,
to check them after a reboot, especially on embedded systems.
In this scenario, dump the kernel ring buffer messages via sysrq
by pressing sysrq+D.

Signed-off-by: Sreenath Vijayan <sreenath.vijayan@sony.com>
Signed-off-by: Shimoyashiki Taichi <taichi.shimoyashiki@sony.com>
---
 Documentation/admin-guide/sysrq.rst |  2 ++
 drivers/tty/sysrq.c                 | 43 ++++++++++++++++++++++++++++-
 2 files changed, 44 insertions(+), 1 deletion(-)

Comments

Greg Kroah-Hartman Dec. 21, 2023, 4:52 p.m. UTC | #1 
On Thu, Dec 21, 2023 at 07:09:53PM +0530, Sreenath Vijayan wrote:
> When terminal is unresponsive, one cannot use dmesg to view kernel
> ring buffer messages. Also, syslog services may be disabled,
> to check them after a reboot, especially on embedded systems.
> In this scenario, dump the kernel ring buffer messages via sysrq
> by pressing sysrq+D.
> 
> Signed-off-by: Sreenath Vijayan <sreenath.vijayan@sony.com>
> Signed-off-by: Shimoyashiki Taichi <taichi.shimoyashiki@sony.com>
> ---
>  Documentation/admin-guide/sysrq.rst |  2 ++
>  drivers/tty/sysrq.c                 | 43 ++++++++++++++++++++++++++++-
>  2 files changed, 44 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/admin-guide/sysrq.rst b/Documentation/admin-guide/sysrq.rst
> index 2f2e5bd440f9..464c4e138b9d 100644
> --- a/Documentation/admin-guide/sysrq.rst
> +++ b/Documentation/admin-guide/sysrq.rst
> @@ -161,6 +161,8 @@ Command	    Function
>              will be printed to your console. (``0``, for example would make
>              it so that only emergency messages like PANICs or OOPSes would
>              make it to your console.)
> +
> +``D``	    Dump the kernel ring buffer
>  =========== ===================================================================

Nit, this doesn't line up anymore :(
Randy Dunlap Dec. 21, 2023, 11:12 p.m. UTC | #2 
On 12/21/23 08:52, Greg KH wrote:
> On Thu, Dec 21, 2023 at 07:09:53PM +0530, Sreenath Vijayan wrote:
>> When terminal is unresponsive, one cannot use dmesg to view kernel
>> ring buffer messages. Also, syslog services may be disabled,
>> to check them after a reboot, especially on embedded systems.
>> In this scenario, dump the kernel ring buffer messages via sysrq
>> by pressing sysrq+D.
>>
>> Signed-off-by: Sreenath Vijayan <sreenath.vijayan@sony.com>
>> Signed-off-by: Shimoyashiki Taichi <taichi.shimoyashiki@sony.com>
>> ---
>>  Documentation/admin-guide/sysrq.rst |  2 ++
>>  drivers/tty/sysrq.c                 | 43 ++++++++++++++++++++++++++++-
>>  2 files changed, 44 insertions(+), 1 deletion(-)
>>
>> diff --git a/Documentation/admin-guide/sysrq.rst b/Documentation/admin-guide/sysrq.rst
>> index 2f2e5bd440f9..464c4e138b9d 100644
>> --- a/Documentation/admin-guide/sysrq.rst
>> +++ b/Documentation/admin-guide/sysrq.rst
>> @@ -161,6 +161,8 @@ Command	    Function
>>              will be printed to your console. (``0``, for example would make
>>              it so that only emergency messages like PANICs or OOPSes would
>>              make it to your console.)
>> +
>> +``D``	    Dump the kernel ring buffer
>>  =========== ===================================================================
> 
> Nit, this doesn't line up anymore :(

Yes, that will cause a docs build warning.

Also, can you be more explicit about which ring buffer this patch
is referring to, please.
kernel test robot Dec. 24, 2023, 11:36 a.m. UTC | #3 
Hi Sreenath,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tty/tty-testing]
[also build test WARNING on tty/tty-next tty/tty-linus linus/master v6.7-rc7 next-20231222]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sreenath-Vijayan/tty-sysrq-Dump-kernel-ring-buffer-messages-via-sysrq/20231222-172636
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty.git tty-testing
patch link:    https://lore.kernel.org/r/20231221133953.1507021-1-sreenath.vijayan%40sony.com
patch subject: [PATCH] tty/sysrq: Dump kernel ring buffer messages via sysrq
config: csky-defconfig (https://download.01.org/0day-ci/archive/20231224/202312241947.B2xJpFET-lkp@intel.com/config)
compiler: csky-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231224/202312241947.B2xJpFET-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202312241947.B2xJpFET-lkp@intel.com/

All warnings (new ones prefixed by >>):

   drivers/tty/sysrq.c: In function 'dmesg_dump_callback':
>> drivers/tty/sysrq.c:479:1: warning: the frame size of 1048 bytes is larger than 1024 bytes [-Wframe-larger-than=]
     479 | }
         | ^


vim +479 drivers/tty/sysrq.c

   454	
   455	static void dmesg_dump_callback(struct work_struct *work)
   456	{
   457		struct kmsg_dump_iter iter;
   458		size_t len;
   459		char buf[1024];
   460		struct console *con;
   461		int cookie;
   462	
   463		kmsg_dump_rewind(&iter);
   464		while (kmsg_dump_get_line(&iter, 1, buf, sizeof(buf), &len)) {
   465			/*
   466			 * Since using printk() or pr_*() will append the message to the
   467			 * kernel ring buffer, they cannot be used to display the retrieved
   468			 * message. Hence console_write() of serial drivers is used.
   469			 */
   470			console_lock();
   471			cookie = console_srcu_read_lock();
   472			for_each_console_srcu(con) {
   473				if ((console_srcu_read_flags(con) & CON_ENABLED) && con->write)
   474					con->write(con, buf, len);
   475			}
   476			console_srcu_read_unlock(cookie);
   477			console_unlock();
   478		}
 > 479	}
   480
kernel test robot Dec. 24, 2023, 2:22 p.m. UTC | #4 
Hi Sreenath,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tty/tty-testing]
[also build test WARNING on tty/tty-next tty/tty-linus linus/master v6.7-rc7 next-20231222]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sreenath-Vijayan/tty-sysrq-Dump-kernel-ring-buffer-messages-via-sysrq/20231222-172636
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty.git tty-testing
patch link:    https://lore.kernel.org/r/20231221133953.1507021-1-sreenath.vijayan%40sony.com
patch subject: [PATCH] tty/sysrq: Dump kernel ring buffer messages via sysrq
config: arm-defconfig (https://download.01.org/0day-ci/archive/20231224/202312242257.5c0xsTqe-lkp@intel.com/config)
compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project.git f28c006a5895fc0e329fe15fead81e37457cb1d1)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231224/202312242257.5c0xsTqe-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202312242257.5c0xsTqe-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> drivers/tty/sysrq.c:455:13: warning: stack frame size (1088) exceeds limit (1024) in 'dmesg_dump_callback' [-Wframe-larger-than]
   static void dmesg_dump_callback(struct work_struct *work)
               ^
   1 warning generated.


vim +/dmesg_dump_callback +455 drivers/tty/sysrq.c

   454	
 > 455	static void dmesg_dump_callback(struct work_struct *work)
   456	{
   457		struct kmsg_dump_iter iter;
   458		size_t len;
   459		char buf[1024];
   460		struct console *con;
   461		int cookie;
   462	
   463		kmsg_dump_rewind(&iter);
   464		while (kmsg_dump_get_line(&iter, 1, buf, sizeof(buf), &len)) {
   465			/*
   466			 * Since using printk() or pr_*() will append the message to the
   467			 * kernel ring buffer, they cannot be used to display the retrieved
   468			 * message. Hence console_write() of serial drivers is used.
   469			 */
   470			console_lock();
   471			cookie = console_srcu_read_lock();
   472			for_each_console_srcu(con) {
   473				if ((console_srcu_read_flags(con) & CON_ENABLED) && con->write)
   474					con->write(con, buf, len);
   475			}
   476			console_srcu_read_unlock(cookie);
   477			console_unlock();
   478		}
   479	}
   480
John Ogness Jan. 10, 2024, 10:20 a.m. UTC | #5 
On 2024-01-10, Sreenath Vijayan <sreenath.vijayan@sony.com> wrote:
> When terminal is unresponsive, one cannot use dmesg to view printk
> ring buffer messages. Also, syslog services may be disabled,
> to check them after a reboot, especially on embedded systems.
> In this scenario, dump the printk ring buffer messages via sysrq
> by pressing sysrq+D.

Generally speaking, I like this idea. I also like that you do not
re-flood the kernel buffers and instead just print directly to the
consoles. (I wish sysrq+z with ftrace did that as well.)

Relying on a workqueue will really limit the usefulness of this
feature. Safe efforts could be made to print directly from the interrupt
context. But maybe this approach can be accepted for now as being
"better than nothing", which can be improved upon in the future (for
example, when atomic consoles are available).

Some more comments from me below...

> diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c
> index 02217e3c916b..62b3911f03b5 100644
> --- a/drivers/tty/sysrq.c
> +++ b/drivers/tty/sysrq.c
> @@ -450,6 +452,51 @@ static const struct sysrq_key_op sysrq_unrt_op = {
>  	.enable_mask	= SYSRQ_ENABLE_RTNICE,
>  };
>  
> +static void dmesg_dump_callback(struct work_struct *work)
> +{
> +	struct kmsg_dump_iter iter;
> +	size_t len;
> +	char *buf;
> +	struct console *con;
> +	int cookie;
> +
> +	/* Size to be updated if PRINTK_MESSAGE_MAX changes */
> +	buf = kzalloc(2048, GFP_KERNEL);
> +	if (!buf)
> +		return;
> +
> +	kmsg_dump_rewind(&iter);
> +	while (kmsg_dump_get_line(&iter, 1, buf, 2048, &len)) {
> +		/*
> +		 * Since using printk() or pr_*() will append the message to the
> +		 * printk ring buffer, they cannot be used to display the retrieved
> +		 * message. Hence console_write() of serial drivers is used.
> +		 */
> +		console_lock();
> +		cookie = console_srcu_read_lock();
> +		for_each_console_srcu(con) {
> +			if ((console_srcu_read_flags(con) & CON_ENABLED) && con->write)
> +				con->write(con, buf, len);
> +		}
> +		console_srcu_read_unlock(cookie);
> +		console_unlock();
> +	}
> +	kfree(buf);
> +}

Rather than implementing all this in drivers/tty/sysrq.c it would
probably be better to just call a new function that is implemented in
kernel/printk/printk.c. Then you would have access to printk-private
items (such as the PRINTK_MESSAGE_MAX macro).

For example, sysrq+z just calls ftrace_dump(), which is implemented in
kernel/trace/trace.c.

John Ogness
diff mbox series

Patch

diff --git a/Documentation/admin-guide/sysrq.rst b/Documentation/admin-guide/sysrq.rst
index 2f2e5bd440f9..464c4e138b9d 100644
--- a/Documentation/admin-guide/sysrq.rst
+++ b/Documentation/admin-guide/sysrq.rst
@@ -161,6 +161,8 @@  Command	    Function
             will be printed to your console. (``0``, for example would make
             it so that only emergency messages like PANICs or OOPSes would
             make it to your console.)
+
+``D``	    Dump the kernel ring buffer
 =========== ===================================================================
 
 Okay, so what can I use them for?
diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c
index 02217e3c916b..aa43cb40c117 100644
--- a/drivers/tty/sysrq.c
+++ b/drivers/tty/sysrq.c
@@ -51,6 +51,8 @@ 
 #include <linux/syscalls.h>
 #include <linux/of.h>
 #include <linux/rcupdate.h>
+#include <linux/kmsg_dump.h>
+#include <linux/console.h>
 
 #include <asm/ptrace.h>
 #include <asm/irq_regs.h>
@@ -450,6 +452,45 @@  static const struct sysrq_key_op sysrq_unrt_op = {
 	.enable_mask	= SYSRQ_ENABLE_RTNICE,
 };
 
+static void dmesg_dump_callback(struct work_struct *work)
+{
+	struct kmsg_dump_iter iter;
+	size_t len;
+	char buf[1024];
+	struct console *con;
+	int cookie;
+
+	kmsg_dump_rewind(&iter);
+	while (kmsg_dump_get_line(&iter, 1, buf, sizeof(buf), &len)) {
+		/*
+		 * Since using printk() or pr_*() will append the message to the
+		 * kernel ring buffer, they cannot be used to display the retrieved
+		 * message. Hence console_write() of serial drivers is used.
+		 */
+		console_lock();
+		cookie = console_srcu_read_lock();
+		for_each_console_srcu(con) {
+			if ((console_srcu_read_flags(con) & CON_ENABLED) && con->write)
+				con->write(con, buf, len);
+		}
+		console_srcu_read_unlock(cookie);
+		console_unlock();
+	}
+}
+
+static DECLARE_WORK(sysrq_dmesg_work, dmesg_dump_callback);
+
+static void sysrq_handle_dmesg_dump(u8 key)
+{
+	queue_work(system_unbound_wq, &sysrq_dmesg_work);
+}
+static struct sysrq_key_op sysrq_dmesg_dump_op = {
+	.handler        = sysrq_handle_dmesg_dump,
+	.help_msg       = "dump-dmesg(D)",
+	.action_msg     = "Dump dmesg",
+	.enable_mask    = SYSRQ_ENABLE_DUMP,
+};
+
 /* Key Operations table and lock */
 static DEFINE_SPINLOCK(sysrq_key_table_lock);
 
@@ -505,7 +546,7 @@  static const struct sysrq_key_op *sysrq_key_table[62] = {
 	NULL,				/* A */
 	NULL,				/* B */
 	NULL,				/* C */
-	NULL,				/* D */
+	&sysrq_dmesg_dump_op,		/* D */
 	NULL,				/* E */
 	NULL,				/* F */
 	NULL,				/* G */