mbox series

[0/8] usb: dwc3: qcom: fix wakeup implementation

Message ID 20220802151404.1797-1-johan+linaro@kernel.org
Headers show
Series usb: dwc3: qcom: fix wakeup implementation | expand

Message

Johan Hovold Aug. 2, 2022, 3:13 p.m. UTC
This series fixes some of the fallout after the recently merged series
that added wakeup support to the Qualcomm dwc3 driver:

	https://lore.kernel.org/all/1655094654-24052-1-git-send-email-quic_kriskura@quicinc.com/

The first patch fixes a long standing PHY power sequencing issue in dwc3
core.

The second patch reverts a power-domain hack that was added by the above
series. There are other ways to implement this which doesn't violate the
genpd interface, and if for some reason those are not sufficient, the
genpd implementation needs to be extended, not hacked around.

The third patch fixes a couple of NULL-pointer dereferences when
suspending controllers in peripheral or OTG mode due to a hack that was
added to suspend path. Unfortunately, it seems the hack needs to stay
for now if we want functioning suspend on some Qualcomm platforms.

The fourth patch fixes another issue in the Qualcomm dwc3 implementation
that has been added a while back and which breaks runtime PM.

The remaining patches moves the wakeup-source property over from the
core node to the glue node in the binding and instead propagates the
wakeup capability to the former during probe.

Note that this incidentally also avoids adding probe-deferral hacks to
the driver as was recently proposed to deal with another problem with
the current implementation:

	https://lore.kernel.org/all/1657891312-21748-1-git-send-email-quic_kriskura@quicinc.com/

With this series I have functioning USB system suspend and wakeup as
well as somewhat functioning runtime PM in host mode on sc8280xp. Note
that the PHYs apparently do not need to be shutdown for wakeup on this
platform.

Some issues remain such as that the controller needs to be suspended to
handle remote wakeup during runtime PM (the wakeup interrupts probably
needs to be enabled whenever there's a wakeup-capable device connected
to the bus) and that root hub connect/disconnect events cannot
selectively be disabled.

And of course, the suspend speed hack needs to be replaced at some
point but that likely requires some more heavy lifting in the dwc3
implementation.

Johan


Johan Hovold (8):
  usb: dwc3: fix PHY disable sequence
  Revert "usb: dwc3: qcom: Keep power domain on to retain controller
    status"
  usb: dwc3: qcom: fix broken non-host-mode suspend
  usb: dwc3: qcom: fix runtime PM wakeup
  Revert "dt-bindings: usb: dwc3: Add wakeup-source property support"
  dt-bindings: usb: qcom,dwc3: add wakeup-source property
  usb: dwc3: qcom: fix wakeup implementation
  usb: dwc3: qcom: clean up suspend callbacks

 .../devicetree/bindings/usb/qcom,dwc3.yaml    |  2 +
 .../devicetree/bindings/usb/snps,dwc3.yaml    |  5 --
 drivers/usb/dwc3/core.c                       | 24 +++---
 drivers/usb/dwc3/dwc3-qcom.c                  | 77 ++++++++++---------
 4 files changed, 55 insertions(+), 53 deletions(-)

Comments

Krishna Kurapati Aug. 2, 2022, 6 p.m. UTC | #1
On 8/2/2022 8:43 PM, Johan Hovold wrote:
> This reverts commit d9be8d5c5b032e5383ff5c404ff4155e9c705429.
>
> Generic power-domain flags must be set before the power-domain is
> initialised and must specifically not be modified by drivers for devices
> that happen to be in the domain.
>
> To make sure that USB power-domains are left enabled during system
> suspend when a device in the domain is in the wakeup path, the
> GENPD_FLAG_ACTIVE_WAKEUP flag should instead be set for the domain
> unconditionally when it is registered.
Hi Johan, Thanks for the review and suggestion.

In case we need the genpd framework to set the GENPD_FLAG_ACTIVE_WAKEUP
flag for a particular domain that will be used by a device (which is
wakeup capable) and hasn't been probed yet , can you help me understand if
there any dt-flags we  can add to convey the same the genpd  framework
so that it will set that flag during domain registration ?

Regards,
Krishna,
>
> Note that this also avoids keeping power-domains on during suspend when
> wakeup has not been enabled (e.g. through sysfs).
>
> For the runtime PM case, making sure that the PHYs are not suspended and
> that they are in the same domain as the controller prevents the domain
> from being suspended. If there are cases where this is not possible or
> desirable, the genpd implementation may need to be extended.
>
> Fixes: d9be8d5c5b03 ("usb: dwc3: qcom: Keep power domain on to retain controller status")
> Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
> ---
>   drivers/usb/dwc3/dwc3-qcom.c | 28 +++++++---------------------
>   1 file changed, 7 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/usb/dwc3/dwc3-qcom.c b/drivers/usb/dwc3/dwc3-qcom.c
> index c5e482f53e9d..be2e3dd36440 100644
> --- a/drivers/usb/dwc3/dwc3-qcom.c
> +++ b/drivers/usb/dwc3/dwc3-qcom.c
> @@ -17,7 +17,6 @@
>   #include <linux/of_platform.h>
>   #include <linux/platform_device.h>
>   #include <linux/phy/phy.h>
> -#include <linux/pm_domain.h>
>   #include <linux/usb/of.h>
>   #include <linux/reset.h>
>   #include <linux/iopoll.h>
> @@ -757,13 +756,12 @@ dwc3_qcom_create_urs_usb_platdev(struct device *dev)
>   
>   static int dwc3_qcom_probe(struct platform_device *pdev)
>   {
> -	struct device_node *np = pdev->dev.of_node;
> -	struct device *dev = &pdev->dev;
> -	struct dwc3_qcom *qcom;
> -	struct resource	*res, *parent_res = NULL;
> -	int ret, i;
> -	bool ignore_pipe_clk;
> -	struct generic_pm_domain *genpd;
> +	struct device_node	*np = pdev->dev.of_node;
> +	struct device		*dev = &pdev->dev;
> +	struct dwc3_qcom	*qcom;
> +	struct resource		*res, *parent_res = NULL;
> +	int			ret, i;
> +	bool			ignore_pipe_clk;
>   
>   	qcom = devm_kzalloc(&pdev->dev, sizeof(*qcom), GFP_KERNEL);
>   	if (!qcom)
> @@ -772,8 +770,6 @@ static int dwc3_qcom_probe(struct platform_device *pdev)
>   	platform_set_drvdata(pdev, qcom);
>   	qcom->dev = &pdev->dev;
>   
> -	genpd = pd_to_genpd(qcom->dev->pm_domain);
> -
>   	if (has_acpi_companion(dev)) {
>   		qcom->acpi_pdata = acpi_device_get_match_data(dev);
>   		if (!qcom->acpi_pdata) {
> @@ -881,17 +877,7 @@ static int dwc3_qcom_probe(struct platform_device *pdev)
>   	if (ret)
>   		goto interconnect_exit;
>   
> -	if (device_can_wakeup(&qcom->dwc3->dev)) {
> -		/*
> -		 * Setting GENPD_FLAG_ALWAYS_ON flag takes care of keeping
> -		 * genpd on in both runtime suspend and system suspend cases.
> -		 */
> -		genpd->flags |= GENPD_FLAG_ALWAYS_ON;
> -		device_init_wakeup(&pdev->dev, true);
> -	} else {
> -		genpd->flags |= GENPD_FLAG_RPM_ALWAYS_ON;
> -	}
> -
> +	device_init_wakeup(&pdev->dev, 1);
>   	qcom->is_suspended = false;
>   	pm_runtime_set_active(dev);
>   	pm_runtime_enable(dev);
Johan Hovold Aug. 3, 2022, 7:42 a.m. UTC | #2
On Tue, Aug 02, 2022 at 11:30:34PM +0530, Krishna Kurapati PSSNV wrote:
> 
> On 8/2/2022 8:43 PM, Johan Hovold wrote:
> > This reverts commit d9be8d5c5b032e5383ff5c404ff4155e9c705429.
> >
> > Generic power-domain flags must be set before the power-domain is
> > initialised and must specifically not be modified by drivers for devices
> > that happen to be in the domain.
> >
> > To make sure that USB power-domains are left enabled during system
> > suspend when a device in the domain is in the wakeup path, the
> > GENPD_FLAG_ACTIVE_WAKEUP flag should instead be set for the domain
> > unconditionally when it is registered.

> In case we need the genpd framework to set the GENPD_FLAG_ACTIVE_WAKEUP
> flag for a particular domain that will be used by a device (which is
> wakeup capable) and hasn't been probed yet , can you help me understand if
> there any dt-flags we  can add to convey the same the genpd  framework
> so that it will set that flag during domain registration ?

This can't be expressed in DT (currently), so what you need is something
like the below:

diff --git a/drivers/clk/qcom/gcc-sc7280.c b/drivers/clk/qcom/gcc-sc7280.c
index 7ff64d4d5920..4ff855269467 100644
--- a/drivers/clk/qcom/gcc-sc7280.c
+++ b/drivers/clk/qcom/gcc-sc7280.c
@@ -3125,6 +3125,7 @@ static struct gdsc gcc_usb30_prim_gdsc = {
        .gdscr = 0xf004,
        .pd = {
                .name = "gcc_usb30_prim_gdsc",
+               .flags = GENPD_FLAG_ACTIVE_WAKEUP,
        },
        .pwrsts = PWRSTS_OFF_ON,
        .flags = VOTABLE,
@@ -3134,6 +3135,7 @@ static struct gdsc gcc_usb30_sec_gdsc = {
        .gdscr = 0x9e004,
        .pd = {
                .name = "gcc_usb30_sec_gdsc",
+               .flags = GENPD_FLAG_ACTIVE_WAKEUP,
        },
        .pwrsts = PWRSTS_OFF_ON,
        .flags = VOTABLE,

to make sure that genpd keeps these domains powered during system
suspend if they are used by devices that are in the wakeup path.

Johan
Matthias Kaehlcke Aug. 3, 2022, 7:11 p.m. UTC | #3
On Tue, Aug 02, 2022 at 05:13:59PM +0200, Johan Hovold wrote:
> A recent commit implementing wakeup support in host mode instead broke
> suspend for peripheral and OTG mode.
> 
> The hack that was added in the suspend path to determine the speed of
> any device connected to the USB2 bus not only accesses internal driver
> data for a child device, but also dereferences a NULL pointer when not
> in host mode and there is no HCD.
> 
> As the controller can switch role at any time when in OTG mode, there's
> no quick fix to this, and since reverting would leave us with broken
> suspend in host-mode (wakeup triggers immediately), keep the hack for
> now and only fix the NULL-pointer dereference.
> 
> Fixes: 6895ea55c385 ("usb: dwc3: qcom: Configure wakeup interrupts during suspend")
> Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
> ---
>  drivers/usb/dwc3/dwc3-qcom.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/usb/dwc3/dwc3-qcom.c b/drivers/usb/dwc3/dwc3-qcom.c
> index be2e3dd36440..b75ff40f75a2 100644
> --- a/drivers/usb/dwc3/dwc3-qcom.c
> +++ b/drivers/usb/dwc3/dwc3-qcom.c
> @@ -301,8 +301,17 @@ static void dwc3_qcom_interconnect_exit(struct dwc3_qcom *qcom)
>  static enum usb_device_speed dwc3_qcom_read_usb2_speed(struct dwc3_qcom *qcom)
>  {
>  	struct dwc3 *dwc = platform_get_drvdata(qcom->dwc3);
> -	struct usb_hcd *hcd = platform_get_drvdata(dwc->xhci);
>  	struct usb_device *udev;
> +	struct usb_hcd *hcd;
> +
> +	if (qcom->mode != USB_DR_MODE_HOST)
> +		return USB_SPEED_UNKNOWN;

Couldn't instead the below block in dwc3_qcom_suspend() be conditional on
the controller being in host mode?

	if (device_may_wakeup(qcom->dev)) {
		qcom->usb2_speed = dwc3_qcom_read_usb2_speed(qcom);
		dwc3_qcom_enable_interrupts(qcom);
	}

I see, the problem is that the role switch could happen at any time as the
commit message says. With this patch there is also a race though, the role
switch could happen just after the check and before obtaining 'hcd'.
Matthias Kaehlcke Aug. 3, 2022, 9:58 p.m. UTC | #4
On Tue, Aug 02, 2022 at 05:14:00PM +0200, Johan Hovold wrote:
> A device must enable wakeups during runtime suspend regardless of
> whether it is capable and allowed to wake the system up from system
> suspend.
> 
> Fixes: 2664deb09306 ("usb: dwc3: qcom: Honor wakeup enabled/disabled state")
> Signed-off-by: Johan Hovold <johan+linaro@kernel.org>

Ah, I wasn't aware that the same wakeup mechanism is used in runtime suspend.

In how far is runtime PM actually supported/used by this driver? The device is
set 'active' in _probe(), and there are no other pm_runtime_* calls, except
in dwc3_qcom_remove() and qcom_dwc3_resume_irq(). How does the device get from
'active' into 'suspended'?
Johan Hovold Aug. 4, 2022, 6:12 a.m. UTC | #5
On Wed, Aug 03, 2022 at 12:11:41PM -0700, Matthias Kaehlcke wrote:
> On Tue, Aug 02, 2022 at 05:13:59PM +0200, Johan Hovold wrote:
> > A recent commit implementing wakeup support in host mode instead broke
> > suspend for peripheral and OTG mode.
> > 
> > The hack that was added in the suspend path to determine the speed of
> > any device connected to the USB2 bus not only accesses internal driver
> > data for a child device, but also dereferences a NULL pointer when not
> > in host mode and there is no HCD.
> > 
> > As the controller can switch role at any time when in OTG mode, there's
> > no quick fix to this, and since reverting would leave us with broken
> > suspend in host-mode (wakeup triggers immediately), keep the hack for
> > now and only fix the NULL-pointer dereference.
> > 
> > Fixes: 6895ea55c385 ("usb: dwc3: qcom: Configure wakeup interrupts during suspend")
> > Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
> > ---
> >  drivers/usb/dwc3/dwc3-qcom.c | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/usb/dwc3/dwc3-qcom.c b/drivers/usb/dwc3/dwc3-qcom.c
> > index be2e3dd36440..b75ff40f75a2 100644
> > --- a/drivers/usb/dwc3/dwc3-qcom.c
> > +++ b/drivers/usb/dwc3/dwc3-qcom.c
> > @@ -301,8 +301,17 @@ static void dwc3_qcom_interconnect_exit(struct dwc3_qcom *qcom)
> >  static enum usb_device_speed dwc3_qcom_read_usb2_speed(struct dwc3_qcom *qcom)
> >  {
> >  	struct dwc3 *dwc = platform_get_drvdata(qcom->dwc3);
> > -	struct usb_hcd *hcd = platform_get_drvdata(dwc->xhci);
> >  	struct usb_device *udev;
> > +	struct usb_hcd *hcd;
> > +
> > +	if (qcom->mode != USB_DR_MODE_HOST)
> > +		return USB_SPEED_UNKNOWN;
> 
> Couldn't instead the below block in dwc3_qcom_suspend() be conditional on
> the controller being in host mode?
> 
> 	if (device_may_wakeup(qcom->dev)) {
> 		qcom->usb2_speed = dwc3_qcom_read_usb2_speed(qcom);
> 		dwc3_qcom_enable_interrupts(qcom);
> 	}

Yeah, the authors clearly didn't consider non-host mode when
implementing this and keeping wakeups disabled in that case probably
doesn't break anything that was ever working.

> I see, the problem is that the role switch could happen at any time as the
> commit message says. With this patch there is also a race though, the role
> switch could happen just after the check and before obtaining 'hcd'.

No, there's no race here as I'm checking the static configuration that
comes from DT. Specifically, I'm not trying to add support for wakeup in
OTG mode, but just prevent suspend from crashing.

I may be possible address also the host-role in OTG mode, but that means
continuing to build on this layer violation.

Note that we're in the suspend callback of the parent so as long as the
drivers for the descendant devices has disabled role switching at this
stage during suspend, we should be good.

But I'm torn about simply ripping this patch out and trying to fix it
up. I want the feature, but the patch adding this clearly wasn't ready
for merging.

I'll take another look at this.

Johan
Johan Hovold Aug. 4, 2022, 7:35 a.m. UTC | #6
On Wed, Aug 03, 2022 at 02:58:33PM -0700, Matthias Kaehlcke wrote:
> On Tue, Aug 02, 2022 at 05:14:00PM +0200, Johan Hovold wrote:
> > A device must enable wakeups during runtime suspend regardless of
> > whether it is capable and allowed to wake the system up from system
> > suspend.
> > 
> > Fixes: 2664deb09306 ("usb: dwc3: qcom: Honor wakeup enabled/disabled state")
> > Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
> 
> Ah, I wasn't aware that the same wakeup mechanism is used in runtime suspend.
> 
> In how far is runtime PM actually supported/used by this driver? The device is
> set 'active' in _probe(), and there are no other pm_runtime_* calls, except
> in dwc3_qcom_remove() and qcom_dwc3_resume_irq(). How does the device get from
> 'active' into 'suspended'?

It will be runtime suspended when the child (core) device suspends, but
you need to enable runtime PM through sysfs first.

And the controller is resumed in the wakeup-interrupt handler for the
runtime PM case.

It seems to work ok, and it looks like the driver has supported this
since it was first merged.

Johan
Matthias Kaehlcke Aug. 4, 2022, 3:35 p.m. UTC | #7
On Thu, Aug 04, 2022 at 09:35:16AM +0200, Johan Hovold wrote:
> On Wed, Aug 03, 2022 at 02:58:33PM -0700, Matthias Kaehlcke wrote:
> > On Tue, Aug 02, 2022 at 05:14:00PM +0200, Johan Hovold wrote:
> > > A device must enable wakeups during runtime suspend regardless of
> > > whether it is capable and allowed to wake the system up from system
> > > suspend.
> > > 
> > > Fixes: 2664deb09306 ("usb: dwc3: qcom: Honor wakeup enabled/disabled state")
> > > Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
> > 
> > Ah, I wasn't aware that the same wakeup mechanism is used in runtime suspend.
> > 
> > In how far is runtime PM actually supported/used by this driver? The device is
> > set 'active' in _probe(), and there are no other pm_runtime_* calls, except
> > in dwc3_qcom_remove() and qcom_dwc3_resume_irq(). How does the device get from
> > 'active' into 'suspended'?
> 
> It will be runtime suspended when the child (core) device suspends, but
> you need to enable runtime PM through sysfs first.

Thanks for the clarification.

After enabling runtime suspend for the dwc3 core, dwc3 glue and the xHCI
the dwc3-qcom enters autosuspend when the delay expires.

> And the controller is resumed in the wakeup-interrupt handler for the
> runtime PM case.
>
> It seems to work ok, and it looks like the driver has supported this
> since it was first merged.

With and without your patch dwc3-qcom enters autosuspend and stays there.
USB devices like a mouse or a USB to Ethernet adapter keep working while
the glue is suspended.

How is the runtime resume triggered for the dwc3 glue?

Sorry if my questions are very basic, so far I haven't dealt much with
autosuspend and I'm trying to get a better understanding in the context
of the dwc3 and why it is currently broken.
Johan Hovold Aug. 4, 2022, 4:04 p.m. UTC | #8
On Thu, Aug 04, 2022 at 08:35:10AM -0700, Matthias Kaehlcke wrote:
> On Thu, Aug 04, 2022 at 09:35:16AM +0200, Johan Hovold wrote:

> After enabling runtime suspend for the dwc3 core, dwc3 glue and the xHCI
> the dwc3-qcom enters autosuspend when the delay expires.
> 
> > And the controller is resumed in the wakeup-interrupt handler for the
> > runtime PM case.
> >
> > It seems to work ok, and it looks like the driver has supported this
> > since it was first merged.
> 
> With and without your patch dwc3-qcom enters autosuspend and stays there.
> USB devices like a mouse or a USB to Ethernet adapter keep working while
> the glue is suspended.

Are you sure you're looking at the right controller? And that it is
actually suspended?

If you plug in a keyboard, enable autosuspend for all devices in the
path (from glue to the keyboard device) and type away, then the
controller must remain active. Stop typing, and all devices in the chain
should suspend.

> How is the runtime resume triggered for the dwc3 glue?

Either by the host driver when it needs to access the device, or by the
device if it is remote-wakeup capable (e.g. a keyboard, but not
necessarily a speaker).

Note that the latter part is what is broken currently as the wakeup
interrupts were not enabled and those are needed to wake up sc8280xp 
when the dwc3 glue has been runtime suspended.

Johan
Matthias Kaehlcke Aug. 4, 2022, 4:25 p.m. UTC | #9
On Thu, Aug 04, 2022 at 06:04:53PM +0200, Johan Hovold wrote:
> On Thu, Aug 04, 2022 at 08:35:10AM -0700, Matthias Kaehlcke wrote:
> > On Thu, Aug 04, 2022 at 09:35:16AM +0200, Johan Hovold wrote:
> 
> > After enabling runtime suspend for the dwc3 core, dwc3 glue and the xHCI
> > the dwc3-qcom enters autosuspend when the delay expires.
> > 
> > > And the controller is resumed in the wakeup-interrupt handler for the
> > > runtime PM case.
> > >
> > > It seems to work ok, and it looks like the driver has supported this
> > > since it was first merged.
> > 
> > With and without your patch dwc3-qcom enters autosuspend and stays there.
> > USB devices like a mouse or a USB to Ethernet adapter keep working while
> > the glue is suspended.
> 
> Are you sure you're looking at the right controller? And that it is
> actually suspended?

Good point! My mind was set on a SC7180 system, which has a single dwc3
controller, but this time I was tinkering on a SC7280 board, which has
two dwc3, and obviously I was looking at the wrong one (-‸ლ)

> If you plug in a keyboard, enable autosuspend for all devices in the
> path (from glue to the keyboard device) and type away, then the
> controller must remain active. Stop typing, and all devices in the chain
> should suspend.

That's what I expected, and now that I'm looking at the right controller
I'm seeing it. I wondered whether the glue device was somehow special.

> > How is the runtime resume triggered for the dwc3 glue?
> 
> Either by the host driver when it needs to access the device, or by the
> device if it is remote-wakeup capable (e.g. a keyboard, but not
> necessarily a speaker).
> 
> Note that the latter part is what is broken currently as the wakeup
> interrupts were not enabled and those are needed to wake up sc8280xp 
> when the dwc3 glue has been runtime suspended.

Thanks for helping me to get a better understanding!