| Message ID | 09f87aed186f4c8cb56aee1d2409ae65@garmin.com |
|---|---|
| State | New |
| Headers | show |
| Series | USB Gadget EEM Suspend/Resume | expand |
On 20-11-16 23:11:20, Neuenschwander, Bowe wrote: > Hello, > > I was hoping for some input on how USB Ethernet Gadget drivers should handle USB suspend/resume events. We have a case where the USB suspend hook is being called for an EEM Gadget; but since that hook is currently unimplemented, the interface remains in an active/enabled state. I have attached a patch that seems to fix this issue for EEM devices by calling gether_disconnect() on suspend and gether_connect() on resume; but I do not if this is actually correct behavior, so I was looking for some advice on that. It seems that since the gadget driver cannot send data until the USB host resumes the USB link, that the interface should be considered disconnected. > Please wrap up to 80 characters per line, that's easy for reading. Would you please describe the real issue you have met? I have a simply ping test for EEM, it could work after resume, the packet is pending if host is suspended. 64 bytes from 192.168.1.1: icmp_seq=154 ttl=64 time=0.358 ms 64 bytes from 192.168.1.1: icmp_seq=155 ttl=64 time=0.375 ms 64 bytes from 192.168.1.1: icmp_seq=156 ttl=64 time=0.364 ms 64 bytes from 192.168.1.1: icmp_seq=157 ttl=64 time=0.359 ms 64 bytes from 192.168.1.1: icmp_seq=158 ttl=64 time=3223 ms 64 bytes from 192.168.1.1: icmp_seq=159 ttl=64 time=2199 ms 64 bytes from 192.168.1.1: icmp_seq=160 ttl=64 time=1175 ms 64 bytes from 192.168.1.1: icmp_seq=161 ttl=64 time=151 ms 64 bytes from 192.168.1.1: icmp_seq=162 ttl=64 time=0.407 ms 64 bytes from 192.168.1.1: icmp_seq=163 ttl=64 time=0.352 ms 64 bytes from 192.168.1.1: icmp_seq=164 ttl=64 time=0.351 ms 64 bytes from 192.168.1.1: icmp_seq=165 ttl=64 time=0.378 ms 64 bytes from 192.168.1.1: icmp_seq=166 ttl=64 time=0.351 ms 64 bytes from 192.168.1.1: icmp_seq=167 ttl=64 time=0.353 ms fg ^C --- 192.168.1.1 ping statistics --- 167 packets transmitted, 158 received, +9 errors, 5.38922% packet loss, time 791ms rtt min/avg/max/mdev = 0.330/1511.202/9886.157/2799.059 ms, pipe 10 root@imx8qmmek:~# dmesg -c [ 1620.382457] configfs-gadget gadget: suspend [ 1622.840007] configfs-gadget gadget: resume [ 1631.275452] configfs-gadget gadget: suspend [ 1633.839442] configfs-gadget gadget: resume [ 1648.257668] configfs-gadget gadget: suspend [ 1657.837955] configfs-gadget gadget: resume [ 1669.252874] configfs-gadget gadget: suspend [ 1679.836613] configfs-gadget gadget: resume [ 1692.245250] configfs-gadget gadget: suspend [ 1695.835642] configfs-gadget gadget: resume [ 1711.255216] configfs-gadget gadget: suspend Peter > From 7cdc1cebe4092393e1de33f59fd2f1cd4355d494 Mon Sep 17 00:00:00 2001 > From: Bowe Neuenschwander <bowe.neuenschwander@garmin.com> > Date: Tue, 10 Nov 2020 15:55:51 -0600 > Subject: [PATCH 1/2] usb: gadget: eem: Add suspend/resume handling > > Add suspend/resume handling to the USB Gadget EEM driver. On suspend, > disconnect the interface; on resume, attempt to reconnect the interface. > > Change-Id: I1c7a2bb1d68a99c774a415b13f6cdabb507ada92 > --- > drivers/usb/gadget/function/f_eem.c | 29 +++++++++++++++++++++++++++++ > 1 file changed, 29 insertions(+) > > diff --git a/drivers/usb/gadget/function/f_eem.c b/drivers/usb/gadget/function/f_eem.c > index cad35a502d3f..4fbdbb8ee295 100644 > --- a/drivers/usb/gadget/function/f_eem.c > +++ b/drivers/usb/gadget/function/f_eem.c > @@ -238,6 +238,33 @@ static void eem_disable(struct usb_function *f) > gether_disconnect(&eem->port); > } > > +static void eem_suspend(struct usb_function *f) > +{ > + struct f_eem *eem = func_to_eem(f); > + struct usb_composite_dev *cdev = f->config->cdev; > + > + DBG(cdev, "eem suspended\n"); > + > + if (eem->port.in_ep->enabled) > + gether_disconnect(&eem->port); > +} > + > +static void eem_resume(struct usb_function *f) > +{ > + struct f_eem *eem = func_to_eem(f); > + struct usb_composite_dev *cdev = f->config->cdev; > + struct net_device *net; > + > + DBG(cdev, "eem resumed\n"); > + > + if (!eem->port.in_ep->enabled) { > + net = gether_connect(&eem->port); > + if (IS_ERR(net)) > + ERROR(cdev, "%s: eem resume failed, err %d\n", > + f->name, PTR_ERR(net)); > + } > +} > + > /*-------------------------------------------------------------------------*/ > > /* EEM function driver setup/binding */ > @@ -636,6 +663,8 @@ static struct usb_function *eem_alloc(struct usb_function_instance *fi) > eem->port.func.set_alt = eem_set_alt; > eem->port.func.setup = eem_setup; > eem->port.func.disable = eem_disable; > + eem->port.func.suspend = eem_suspend; > + eem->port.func.resume = eem_resume; > eem->port.func.free_func = eem_free; > eem->port.wrap = eem_wrap; > eem->port.unwrap = eem_unwrap; > -- > 2.29.2 >
On 20-11-17 18:30:34, Neuenschwander, Bowe wrote: > Thanks for your quick reply. I think your ping test/example tells me quite a > bit about the expected behavior. I do have concerns though on what this would > do to TCP connections. Could that cause quite a bit of packet build up as > the connection attempts to re-transmit packets that were not ACKed (but still > sitting in the queue)? In the case that the suspend is long (say 10-60 min), > it seems this could cause quite a lot of packet build up, though I assume its > TX queue will fill up pretty quickly and send will start returning failure). > > The issue we are seeing is when USB is physically disconnected, the suspend > hooks are called, but the disconnect hooks are not. The device side of the > USB link (the one configured with Gadget EEM) is externally powered and has > a hub as well. The host is disconnected from the hub, but the link between > the hub and Gadget EEM remains intact, so the Gadget EEM processor does not > see VUSB go low. See below for a crude diagram (monospaced font needed): > ____________________________ > | Device | > | ________ ________ | ________ > | | | | | | | | > | | USB | | USB | | | USB | > | | Gadget |-----| HUB |--|---X---| Host | > | | EEM | | | | | | > | |________| |________| | |________| > | | > |____________________________| > > Therefore, it stays in that suspend state until USB is reconnected, at which > point the disconnect hook gets called and the connection is reset and set back > up. See below for dmesg (note that the USB Gadget EEM interface is vis0): > > USB Disconnected: > [ 4047.888922] g_ether gadget: suspend > [ 4048.442846] vis0: stop stats: rx/tx 13079/13406, errs 0/0 > [ 4048.442967] vis0: host still using in/out endpoints > > USB Reconnected: > [ 4054.891454] g_ether gadget: reset config > [ 4054.891487] g_ether gadget: eem deactivated > [ 4054.891500] vis0: gether_disconnect > [ 4054.897743] g_ether gadget: suspend > [ 4055.273258] g_ether gadget: suspend > [ 4055.662466] g_ether gadget: high-speed config #1: CDC Ethernet (EEM) > [ 4055.668899] g_ether gadget: reset eem > [ 4055.668912] vis0: gether_disconnect > [ 4055.668924] g_ether gadget: init eem > [ 4055.668934] g_ether gadget: activate eem > [ 4055.668974] vis0: qlen 10 > [ 4055.674126] g_ether gadget: reset eem > [ 4055.674161] vis0: gether_disconnect > [ 4055.674219] g_ether gadget: init eem > [ 4055.674230] g_ether gadget: activate eem > [ 4055.674267] vis0: qlen 10 > [ 4055.847697] vis0: eth_open > [ 4055.847729] vis0: eth_start > > The problem we have is when it goes into that suspend state, that interface > remains up but cannot actually send/receive. There is a process (for which > we do not have source code available) that starts consuming a large portion > of the CPU (according to top/htop), which in turn causes other issues. We > have have a little daemon to detect when USB gets disconnected and bring down > that interface (ifdown), and we have tried adjusting the offending process's > nice value, but these do not fix the issue (they help, but do not elminate > it). The only fix we have found so far to eliminate this issue is something > similar to the patch I previously sent; but again, I have questions of if it > is acceptable handling for USB suspend. Since suspend is one of USB bus state, the USB host may suspend the device at some situations, it seems it is your HUB's issue that does not disconnect its downstream ports that the host disconnects it. Could your daemon poll suspend state for gadget through /sys/class/udc/<YOUR UDC NAME>/state and bring down the interface? You need below change at kernel for that. diff --git a/drivers/usb/gadget/composite.c b/drivers/usb/gadget/composite.c index c6d455f2bb92..bf11488de93b 100644 --- a/drivers/usb/gadget/composite.c +++ b/drivers/usb/gadget/composite.c @@ -2330,6 +2330,7 @@ void composite_suspend(struct usb_gadget *gadget) usb_gadget_set_selfpowered(gadget); usb_gadget_vbus_draw(gadget, 2); + usb_gadget_set_state(gadget, USB_STATE_SUSPENDED); } void composite_resume(struct usb_gadget *gadget) -- Thanks, Peter Chen
I gave that a try and it works in general; but unfortunately, it is not quick enough. Even if our daemon is re-niced to be high priority, the problematic process still consumes the CPU and causes problems before it is able to take down the interface. Therefore, we are planning to move forward with the original patch for our local kernel, and will not submit it upstream. I am open to other suggestions if anybody has them; but at this point, I'm not convinced this is something wrong or lacking in kernel like I originally thought might be the case. Thanks, -Bowe From: Peter Chen <peter.chen@nxp.com> Sent: Tuesday, November 24, 2020 7:53 PM To: Neuenschwander, Bowe Cc: linux-usb@vger.kernel.org Subject: Re: USB Gadget EEM Suspend/Resume CAUTION - EXTERNAL EMAIL: Do not click any links or open any attachments unless you trust the sender and know the content is safe. On 20-11-17 18:30:34, Neuenschwander, Bowe wrote: > Thanks for your quick reply. I think your ping test/example tells me quite a > bit about the expected behavior. I do have concerns though on what this would > do to TCP connections. Could that cause quite a bit of packet build up as > the connection attempts to re-transmit packets that were not ACKed (but still > sitting in the queue)? In the case that the suspend is long (say 10-60 min), > it seems this could cause quite a lot of packet build up, though I assume its > TX queue will fill up pretty quickly and send will start returning failure). > > The issue we are seeing is when USB is physically disconnected, the suspend > hooks are called, but the disconnect hooks are not. The device side of the > USB link (the one configured with Gadget EEM) is externally powered and has > a hub as well. The host is disconnected from the hub, but the link between > the hub and Gadget EEM remains intact, so the Gadget EEM processor does not > see VUSB go low. See below for a crude diagram (monospaced font needed): > ____________________________ > | Device | > | ________ ________ | ________ > | | | | | | | | > | | USB | | USB | | | USB | > | | Gadget |-----| HUB |--|---X---| Host | > | | EEM | | | | | | > | |________| |________| | |________| > | | > |____________________________| > > Therefore, it stays in that suspend state until USB is reconnected, at which > point the disconnect hook gets called and the connection is reset and set back > up. See below for dmesg (note that the USB Gadget EEM interface is vis0): > > USB Disconnected: > [ 4047.888922] g_ether gadget: suspend > [ 4048.442846] vis0: stop stats: rx/tx 13079/13406, errs 0/0 > [ 4048.442967] vis0: host still using in/out endpoints > > USB Reconnected: > [ 4054.891454] g_ether gadget: reset config > [ 4054.891487] g_ether gadget: eem deactivated > [ 4054.891500] vis0: gether_disconnect > [ 4054.897743] g_ether gadget: suspend > [ 4055.273258] g_ether gadget: suspend > [ 4055.662466] g_ether gadget: high-speed config #1: CDC Ethernet (EEM) > [ 4055.668899] g_ether gadget: reset eem > [ 4055.668912] vis0: gether_disconnect > [ 4055.668924] g_ether gadget: init eem > [ 4055.668934] g_ether gadget: activate eem > [ 4055.668974] vis0: qlen 10 > [ 4055.674126] g_ether gadget: reset eem > [ 4055.674161] vis0: gether_disconnect > [ 4055.674219] g_ether gadget: init eem > [ 4055.674230] g_ether gadget: activate eem > [ 4055.674267] vis0: qlen 10 > [ 4055.847697] vis0: eth_open > [ 4055.847729] vis0: eth_start > > The problem we have is when it goes into that suspend state, that interface > remains up but cannot actually send/receive. There is a process (for which > we do not have source code available) that starts consuming a large portion > of the CPU (according to top/htop), which in turn causes other issues. We > have have a little daemon to detect when USB gets disconnected and bring down > that interface (ifdown), and we have tried adjusting the offending process's > nice value, but these do not fix the issue (they help, but do not elminate > it). The only fix we have found so far to eliminate this issue is something > similar to the patch I previously sent; but again, I have questions of if it > is acceptable handling for USB suspend. Since suspend is one of USB bus state, the USB host may suspend the device at some situations, it seems it is your HUB's issue that does not disconnect its downstream ports that the host disconnects it. Could your daemon poll suspend state for gadget through /sys/class/udc/<YOUR UDC NAME>/state and bring down the interface? You need below change at kernel for that. diff --git a/drivers/usb/gadget/composite.c b/drivers/usb/gadget/composite.c index c6d455f2bb92..bf11488de93b 100644 --- a/drivers/usb/gadget/composite.c +++ b/drivers/usb/gadget/composite.c @@ -2330,6 +2330,7 @@ void composite_suspend(struct usb_gadget *gadget) usb_gadget_set_selfpowered(gadget); usb_gadget_vbus_draw(gadget, 2); + usb_gadget_set_state(gadget, USB_STATE_SUSPENDED); } void composite_resume(struct usb_gadget *gadget) -- Thanks, Peter Chen ________________________________ CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.
From 7cdc1cebe4092393e1de33f59fd2f1cd4355d494 Mon Sep 17 00:00:00 2001 From: Bowe Neuenschwander <bowe.neuenschwander@garmin.com> Date: Tue, 10 Nov 2020 15:55:51 -0600 Subject: [PATCH 1/2] usb: gadget: eem: Add suspend/resume handling Add suspend/resume handling to the USB Gadget EEM driver. On suspend, disconnect the interface; on resume, attempt to reconnect the interface. Change-Id: I1c7a2bb1d68a99c774a415b13f6cdabb507ada92 --- drivers/usb/gadget/function/f_eem.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/drivers/usb/gadget/function/f_eem.c b/drivers/usb/gadget/function/f_eem.c index cad35a502d3f..4fbdbb8ee295 100644 --- a/drivers/usb/gadget/function/f_eem.c +++ b/drivers/usb/gadget/function/f_eem.c @@ -238,6 +238,33 @@ static void eem_disable(struct usb_function *f) gether_disconnect(&eem->port); } +static void eem_suspend(struct usb_function *f) +{ + struct f_eem *eem = func_to_eem(f); + struct usb_composite_dev *cdev = f->config->cdev; + + DBG(cdev, "eem suspended\n"); + + if (eem->port.in_ep->enabled) + gether_disconnect(&eem->port); +} + +static void eem_resume(struct usb_function *f) +{ + struct f_eem *eem = func_to_eem(f); + struct usb_composite_dev *cdev = f->config->cdev; + struct net_device *net; + + DBG(cdev, "eem resumed\n"); + + if (!eem->port.in_ep->enabled) { + net = gether_connect(&eem->port); + if (IS_ERR(net)) + ERROR(cdev, "%s: eem resume failed, err %d\n", + f->name, PTR_ERR(net)); + } +} + /*-------------------------------------------------------------------------*/ /* EEM function driver setup/binding */ @@ -636,6 +663,8 @@ static struct usb_function *eem_alloc(struct usb_function_instance *fi) eem->port.func.set_alt = eem_set_alt; eem->port.func.setup = eem_setup; eem->port.func.disable = eem_disable; + eem->port.func.suspend = eem_suspend; + eem->port.func.resume = eem_resume; eem->port.func.free_func = eem_free; eem->port.wrap = eem_wrap; eem->port.unwrap = eem_unwrap; -- 2.29.2