| Message ID | 20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com |
|---|---|
| State | New |
| Headers | show |
| Series | scsi: storvsc: Ratelimit warning logs to prevent VM denial of service | expand |
From: Easwar Hariharan <eahariha@linux.microsoft.com> Sent: Tuesday, January 7, 2025 9:29 AM > > If there's a persistent error in the hypervisor, the scsi warning for > failed IO can flood the kernel log and max out CPU utilization, > preventing troubleshooting from the VM side. Ratelimit the warning so > it doesn't DOS the VM. > > Closes: > https://github.com/microsoft/WSL/issues/9173 > Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com> > --- > If there's a persistent error in the hypervisor, the scsi warning for > failed IO can overwhelm the kernel log and max out CPU utilization, > preventing troubleshooting. Ratelimit the warning so it doesn't DOS the > VM. > > This is not super critical and can be deferred to an 6.14 rc since it > mostly occurs when there's a problem with the host disk or filesystem. > --- > drivers/scsi/storvsc_drv.c | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c > index > 45665d4aca925a10ce4293edf40cebdc4d4997f2..5a101ac06c478aad9b01072e7a6c > 5af48219f5c8 100644 > --- a/drivers/scsi/storvsc_drv.c > +++ b/drivers/scsi/storvsc_drv.c > @@ -171,6 +171,12 @@ do { > \ > dev_warn(&(dev)->device, fmt, ##__VA_ARGS__); \ > } while (0) > > +#define storvsc_log_ratelimited(dev, level, fmt, ...) \ > +do { \ > + if (do_logging(level)) \ > + dev_warn_ratelimited(&(dev)->device, fmt, ##__VA_ARGS__); \ > +} while (0) > + > struct vmscsi_request { > u16 length; > u8 srb_status; > @@ -1176,7 +1182,7 @@ static void storvsc_on_io_completion(struct storvsc_device *stor_device, > int loglevel = (stor_pkt->vm_srb.cdb[0] == TEST_UNIT_READY) ? > STORVSC_LOGGING_WARN : STORVSC_LOGGING_ERROR; > > - storvsc_log(device, loglevel, > + storvsc_log_ratelimited(device, loglevel, > "tag#%d cmd 0x%x status: scsi 0x%x srb 0x%x hv 0x%x\n", > scsi_cmd_to_rq(request->cmd)->tag, > stor_pkt->vm_srb.cdb[0], > > --- > base-commit: 4e16367cfe0ce395f29d0482b78970cce8e1db73 > change-id: 20241023-eahariha-ratelimit-storvsc-b3fe53f02a58 > > Best regards, > -- > Easwar Hariharan <eahariha@linux.microsoft.com> > Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Easwar, > If there's a persistent error in the hypervisor, the scsi warning for > failed IO can flood the kernel log and max out CPU utilization, > preventing troubleshooting from the VM side. Ratelimit the warning so > it doesn't DOS the VM. Applied to 6.14/scsi-staging, thanks!
diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c index 45665d4aca925a10ce4293edf40cebdc4d4997f2..5a101ac06c478aad9b01072e7a6c5af48219f5c8 100644 --- a/drivers/scsi/storvsc_drv.c +++ b/drivers/scsi/storvsc_drv.c @@ -171,6 +171,12 @@ do { \ dev_warn(&(dev)->device, fmt, ##__VA_ARGS__); \ } while (0) +#define storvsc_log_ratelimited(dev, level, fmt, ...) \ +do { \ + if (do_logging(level)) \ + dev_warn_ratelimited(&(dev)->device, fmt, ##__VA_ARGS__); \ +} while (0) + struct vmscsi_request { u16 length; u8 srb_status; @@ -1176,7 +1182,7 @@ static void storvsc_on_io_completion(struct storvsc_device *stor_device, int loglevel = (stor_pkt->vm_srb.cdb[0] == TEST_UNIT_READY) ? STORVSC_LOGGING_WARN : STORVSC_LOGGING_ERROR; - storvsc_log(device, loglevel, + storvsc_log_ratelimited(device, loglevel, "tag#%d cmd 0x%x status: scsi 0x%x srb 0x%x hv 0x%x\n", scsi_cmd_to_rq(request->cmd)->tag, stor_pkt->vm_srb.cdb[0],
If there's a persistent error in the hypervisor, the scsi warning for failed IO can flood the kernel log and max out CPU utilization, preventing troubleshooting from the VM side. Ratelimit the warning so it doesn't DOS the VM. Closes: https://github.com/microsoft/WSL/issues/9173 Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com> --- If there's a persistent error in the hypervisor, the scsi warning for failed IO can overwhelm the kernel log and max out CPU utilization, preventing troubleshooting. Ratelimit the warning so it doesn't DOS the VM. This is not super critical and can be deferred to an 6.14 rc since it mostly occurs when there's a problem with the host disk or filesystem. --- drivers/scsi/storvsc_drv.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) --- base-commit: 4e16367cfe0ce395f29d0482b78970cce8e1db73 change-id: 20241023-eahariha-ratelimit-storvsc-b3fe53f02a58 Best regards,