| Message ID | 20220902233543.390890-1-helgaas@kernel.org |
|---|---|
| Headers | show |
| Series | PCI/PM: Always disable PTM for all devices during suspend | expand |
On 9/2/22 4:35 PM, Bjorn Helgaas wrote: > From: Bjorn Helgaas <bhelgaas@google.com> > > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> > --- > drivers/pci/pcie/ptm.c | 34 +++++++++++++++++++++++++++------- > 1 file changed, 27 insertions(+), 7 deletions(-) > > diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c > index b6a417247ce3..ad283818f37b 100644 > --- a/drivers/pci/pcie/ptm.c > +++ b/drivers/pci/pcie/ptm.c > @@ -167,11 +167,11 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) > if (!pos) > return -EINVAL; > > - pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > - if (!(cap & PCI_PTM_CAP_REQ)) > - return -EINVAL; > - May be saving PCI_PTM_CAP_REQ enabled state here and using it below only for endpoints checks will reduce the code duplication? > /* > + * Root Ports and Switch Upstream Ports have been configured > + * by pci_ptm_init(), so preserve their PCI_PTM_CTRL_ROOT and > + * granularity. > + * > * For a PCIe Endpoint, PTM is only useful if the endpoint can > * issue PTM requests to upstream devices that have PTM enabled. > * > @@ -179,19 +179,39 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) > * device, so there must be some implementation-specific way to > * associate the endpoint with a time source. > */ > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { > + if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT || > + pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM) { > + if (pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM) { > + ups = pci_upstream_bridge(dev); > + if (!ups || !ups->ptm_enabled) > + return -EINVAL; > + } > + > + pci_read_config_dword(dev, pos + PCI_PTM_CTRL, &ctrl); Why read PCI_PTM_CTRL state only for root and upstream ports? The same logic will work for endpoints and RC endpoints right? What not use dev->ptm_granularity for root ports? > + ctrl |= PCI_PTM_CTRL_ENABLE; > + } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { > + pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > + if (!(cap & PCI_PTM_CAP_REQ)) > + return -EINVAL; > + > ups = pci_upstream_bridge(dev); > if (!ups || !ups->ptm_enabled) > return -EINVAL; > > dev->ptm_granularity = ups->ptm_granularity; > + ctrl = PCI_PTM_CTRL_ENABLE; > + ctrl |= dev->ptm_granularity << 8; > } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END) { > + pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > + if (!(cap & PCI_PTM_CAP_REQ)) > + return -EINVAL; > + > dev->ptm_granularity = 0; > + ctrl = PCI_PTM_CTRL_ENABLE; > + ctrl |= dev->ptm_granularity << 8; > } else > return -EINVAL; > > - ctrl = PCI_PTM_CTRL_ENABLE; > - ctrl |= dev->ptm_granularity << 8; > pci_write_config_dword(dev, pos + PCI_PTM_CTRL, ctrl); > dev->ptm_enabled = 1; >
On 9/2/22 4:35 PM, Bjorn Helgaas wrote: > From: Bjorn Helgaas <bhelgaas@google.com> > > We want to disable PTM on Root Ports because that allows some chips, e.g., > Intel mobile chips since Coffee Lake, to enter a lower-power PM state. > > That means we also have to disable PTM on downstream devices. PCIe r6.0, > sec 2.2.8, recommends that functions support generation of messages in > non-D0 states, so we have to assume Switch Upstream Ports or Endpoints may > send PTM Requests while in D1, D2, and D3hot. A PTM message received by a > Downstream Port (including a Root Port) with PTM disabled must be treated > as an Unsupported Request (sec 6.21.3). > > PTM was previously disabled only for Root Ports, and it was disabled in > pci_prepare_to_sleep(), which is not called at all if a driver supports > legacy PM or does its own state saving. > > Instead, disable PTM early in pci_pm_suspend() and pci_pm_runtime_suspend() > so we do it in all cases. > > Previously PTM was disabled *after* saving device state, so the state > restore on resume automatically re-enabled it. Since we now disable PTM > *before* saving state, we must explicitly re-enable it. > > Here's a sample of errors that occur when PTM is disabled only on the Root > Port. With this topology: > > 0000:00:1d.0 Root Port to [bus 08-71] > 0000:08:00.0 Switch Upstream Port to [bus 09-71] > > Kai-Heng reported errors like this: > > pcieport 0000:00:1d.0: AER: Uncorrected (Non-Fatal) error received: 0000:00:1d.0 > pcieport 0000:00:1d.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID) > pcieport 0000:00:1d.0: device [8086:7ab0] error status/mask=00100000/00004000 > pcieport 0000:00:1d.0: [20] UnsupReq (First) > pcieport 0000:00:1d.0: AER: TLP Header: 34000000 08000052 00000000 00000000 > > Decoding TLP header 0x34...... (0011 0100b) and 0x08000052: > > Fmt 001b 4 DW header, no data > Type 1 0100b Msg (Local - Terminate at Receiver) > Requester ID 0x0800 Bus 08 Devfn 00.0 > Message Code 0x52 0101 0010b PTM Request > > The 00:1d.0 Root Port logged an Unsupported Request error when it received > a PTM Request with Requester ID 08:00.0. > > Fixes: a697f072f5da ("PCI: Disable PTM during suspend to save power") > Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215453 > Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216210 > Based-on: https://lore.kernel.org/r/20220706123244.18056-1-kai.heng.feng@canonical.com > Based-on-patch-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > Reported-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> > --- > drivers/pci/pci-driver.c | 14 ++++++++++++++ > drivers/pci/pci.c | 20 -------------------- > 2 files changed, 14 insertions(+), 20 deletions(-) > > diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c > index 2815922ac525..115febaa7e0b 100644 > --- a/drivers/pci/pci-driver.c > +++ b/drivers/pci/pci-driver.c > @@ -772,6 +772,12 @@ static int pci_pm_suspend(struct device *dev) > struct pci_dev *pci_dev = to_pci_dev(dev); > const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL; > > + /* > + * Disabling PTM allows some systems, e.g., Intel mobile chips > + * since Coffee Lake, to enter a lower-power PM state. > + */ > + pci_disable_ptm(pci_dev); I think you can use "if (pci_dev->ptm_enabled)" check for pci_disable_ptm() as well. This will avoid unnecessary checks in pci_disable_ptm(). > + > pci_dev->skip_bus_pm = false; > > if (pci_has_legacy_pm_support(pci_dev)) > @@ -982,6 +988,9 @@ static int pci_pm_resume(struct device *dev) > if (pci_dev->state_saved) > pci_restore_standard_config(pci_dev); > > + if (pci_dev->ptm_enabled) > + pci_enable_ptm(pci_dev, NULL); > + > if (pci_has_legacy_pm_support(pci_dev)) > return pci_legacy_resume(dev); > > @@ -1269,6 +1278,8 @@ static int pci_pm_runtime_suspend(struct device *dev) > pci_power_t prev = pci_dev->current_state; > int error; > > + pci_disable_ptm(pci_dev); > + > /* > * If pci_dev->driver is not set (unbound), we leave the device in D0, > * but it may go to D3cold when the bridge above it runtime suspends. > @@ -1331,6 +1342,9 @@ static int pci_pm_runtime_resume(struct device *dev) > */ > pci_pm_default_resume_early(pci_dev); > > + if (pci_dev->ptm_enabled) > + pci_enable_ptm(pci_dev, NULL); > + > if (!pci_dev->driver) > return 0; > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 95bc329e74c0..b0e2968c8cca 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -2706,16 +2706,6 @@ int pci_prepare_to_sleep(struct pci_dev *dev) > if (target_state == PCI_POWER_ERROR) > return -EIO; > > - /* > - * There are systems (for example, Intel mobile chips since Coffee > - * Lake) where the power drawn while suspended can be significantly > - * reduced by disabling PTM on PCIe root ports as this allows the > - * port to enter a lower-power PM state and the SoC to reach a > - * lower-power idle state as a whole. > - */ > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) > - pci_disable_ptm(dev); > - > pci_enable_wake(dev, target_state, wakeup); > > error = pci_set_power_state(dev, target_state); > @@ -2764,16 +2754,6 @@ int pci_finish_runtime_suspend(struct pci_dev *dev) > if (target_state == PCI_POWER_ERROR) > return -EIO; > > - /* > - * There are systems (for example, Intel mobile chips since Coffee > - * Lake) where the power drawn while suspended can be significantly > - * reduced by disabling PTM on PCIe root ports as this allows the > - * port to enter a lower-power PM state and the SoC to reach a > - * lower-power idle state as a whole. > - */ > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) > - pci_disable_ptm(dev); > - > __pci_enable_wake(dev, target_state, pci_dev_run_wake(dev)); > > error = pci_set_power_state(dev, target_state);
On Sat, Sep 3, 2022 at 1:59 AM Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@linux.intel.com> wrote: > > > > On 9/2/22 4:35 PM, Bjorn Helgaas wrote: > > From: Bjorn Helgaas <bhelgaas@google.com> > > > > We want to disable PTM on Root Ports because that allows some chips, e.g., > > Intel mobile chips since Coffee Lake, to enter a lower-power PM state. > > > > That means we also have to disable PTM on downstream devices. PCIe r6.0, > > sec 2.2.8, recommends that functions support generation of messages in > > non-D0 states, so we have to assume Switch Upstream Ports or Endpoints may > > send PTM Requests while in D1, D2, and D3hot. A PTM message received by a > > Downstream Port (including a Root Port) with PTM disabled must be treated > > as an Unsupported Request (sec 6.21.3). > > > > PTM was previously disabled only for Root Ports, and it was disabled in > > pci_prepare_to_sleep(), which is not called at all if a driver supports > > legacy PM or does its own state saving. > > > > Instead, disable PTM early in pci_pm_suspend() and pci_pm_runtime_suspend() > > so we do it in all cases. > > > > Previously PTM was disabled *after* saving device state, so the state > > restore on resume automatically re-enabled it. Since we now disable PTM > > *before* saving state, we must explicitly re-enable it. > > > > Here's a sample of errors that occur when PTM is disabled only on the Root > > Port. With this topology: > > > > 0000:00:1d.0 Root Port to [bus 08-71] > > 0000:08:00.0 Switch Upstream Port to [bus 09-71] > > > > Kai-Heng reported errors like this: > > > > pcieport 0000:00:1d.0: AER: Uncorrected (Non-Fatal) error received: 0000:00:1d.0 > > pcieport 0000:00:1d.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID) > > pcieport 0000:00:1d.0: device [8086:7ab0] error status/mask=00100000/00004000 > > pcieport 0000:00:1d.0: [20] UnsupReq (First) > > pcieport 0000:00:1d.0: AER: TLP Header: 34000000 08000052 00000000 00000000 > > > > Decoding TLP header 0x34...... (0011 0100b) and 0x08000052: > > > > Fmt 001b 4 DW header, no data > > Type 1 0100b Msg (Local - Terminate at Receiver) > > Requester ID 0x0800 Bus 08 Devfn 00.0 > > Message Code 0x52 0101 0010b PTM Request > > > > The 00:1d.0 Root Port logged an Unsupported Request error when it received > > a PTM Request with Requester ID 08:00.0. > > > > Fixes: a697f072f5da ("PCI: Disable PTM during suspend to save power") > > Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215453 > > Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216210 > > Based-on: https://lore.kernel.org/r/20220706123244.18056-1-kai.heng.feng@canonical.com > > Based-on-patch-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > > Reported-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> > > --- > > drivers/pci/pci-driver.c | 14 ++++++++++++++ > > drivers/pci/pci.c | 20 -------------------- > > 2 files changed, 14 insertions(+), 20 deletions(-) > > > > diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c > > index 2815922ac525..115febaa7e0b 100644 > > --- a/drivers/pci/pci-driver.c > > +++ b/drivers/pci/pci-driver.c > > @@ -772,6 +772,12 @@ static int pci_pm_suspend(struct device *dev) > > struct pci_dev *pci_dev = to_pci_dev(dev); > > const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL; > > > > + /* > > + * Disabling PTM allows some systems, e.g., Intel mobile chips > > + * since Coffee Lake, to enter a lower-power PM state. > > + */ > > + pci_disable_ptm(pci_dev); > > I think you can use "if (pci_dev->ptm_enabled)" check for pci_disable_ptm() > as well. This will avoid unnecessary checks in pci_disable_ptm(). Or use that check in pci_disable_ptm() instead of the pci_is_pcie() one. Also, I would remae pci_disable_ptm() to pci_suspend_ptm() (because its role is to temporarily disable PTM for system-wide suspend) and introduc pci_resume_ptm() that will do if (pci_dev->ptm_enabled) pci_enable_ptm(pci_dev, NULL); > > + > > pci_dev->skip_bus_pm = false; > > > > if (pci_has_legacy_pm_support(pci_dev)) > > @@ -982,6 +988,9 @@ static int pci_pm_resume(struct device *dev) > > if (pci_dev->state_saved) > > pci_restore_standard_config(pci_dev); > > > > + if (pci_dev->ptm_enabled) > > + pci_enable_ptm(pci_dev, NULL); > > + > > if (pci_has_legacy_pm_support(pci_dev)) > > return pci_legacy_resume(dev); > > > > @@ -1269,6 +1278,8 @@ static int pci_pm_runtime_suspend(struct device *dev) > > pci_power_t prev = pci_dev->current_state; > > int error; > > > > + pci_disable_ptm(pci_dev); > > + > > /* > > * If pci_dev->driver is not set (unbound), we leave the device in D0, > > * but it may go to D3cold when the bridge above it runtime suspends. > > @@ -1331,6 +1342,9 @@ static int pci_pm_runtime_resume(struct device *dev) > > */ > > pci_pm_default_resume_early(pci_dev); > > > > + if (pci_dev->ptm_enabled) > > + pci_enable_ptm(pci_dev, NULL); > > + > > if (!pci_dev->driver) > > return 0; > > > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > > index 95bc329e74c0..b0e2968c8cca 100644 > > --- a/drivers/pci/pci.c > > +++ b/drivers/pci/pci.c > > @@ -2706,16 +2706,6 @@ int pci_prepare_to_sleep(struct pci_dev *dev) > > if (target_state == PCI_POWER_ERROR) > > return -EIO; > > > > - /* > > - * There are systems (for example, Intel mobile chips since Coffee > > - * Lake) where the power drawn while suspended can be significantly > > - * reduced by disabling PTM on PCIe root ports as this allows the > > - * port to enter a lower-power PM state and the SoC to reach a > > - * lower-power idle state as a whole. > > - */ > > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) > > - pci_disable_ptm(dev); > > - > > pci_enable_wake(dev, target_state, wakeup); > > > > error = pci_set_power_state(dev, target_state); > > @@ -2764,16 +2754,6 @@ int pci_finish_runtime_suspend(struct pci_dev *dev) > > if (target_state == PCI_POWER_ERROR) > > return -EIO; > > > > - /* > > - * There are systems (for example, Intel mobile chips since Coffee > > - * Lake) where the power drawn while suspended can be significantly > > - * reduced by disabling PTM on PCIe root ports as this allows the > > - * port to enter a lower-power PM state and the SoC to reach a > > - * lower-power idle state as a whole. > > - */ > > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) > > - pci_disable_ptm(dev); > > - > > __pci_enable_wake(dev, target_state, pci_dev_run_wake(dev)); > > > > error = pci_set_power_state(dev, target_state); > > -- > Sathyanarayanan Kuppuswamy > Linux Kernel Developer
On Sat, Sep 3, 2022 at 1:35 AM Bjorn Helgaas <helgaas@kernel.org> wrote: > > From: Bjorn Helgaas <bhelgaas@google.com> > > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> > --- > drivers/pci/pcie/ptm.c | 34 +++++++++++++++++++++++++++------- > 1 file changed, 27 insertions(+), 7 deletions(-) > > diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c > index b6a417247ce3..ad283818f37b 100644 > --- a/drivers/pci/pcie/ptm.c > +++ b/drivers/pci/pcie/ptm.c > @@ -167,11 +167,11 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) > if (!pos) > return -EINVAL; > > - pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > - if (!(cap & PCI_PTM_CAP_REQ)) > - return -EINVAL; > - > /* > + * Root Ports and Switch Upstream Ports have been configured > + * by pci_ptm_init(), so preserve their PCI_PTM_CTRL_ROOT and > + * granularity. > + * > * For a PCIe Endpoint, PTM is only useful if the endpoint can > * issue PTM requests to upstream devices that have PTM enabled. > * > @@ -179,19 +179,39 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) > * device, so there must be some implementation-specific way to > * associate the endpoint with a time source. > */ > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { > + if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT || > + pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM) { > + if (pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM) { > + ups = pci_upstream_bridge(dev); > + if (!ups || !ups->ptm_enabled) > + return -EINVAL; > + } > + > + pci_read_config_dword(dev, pos + PCI_PTM_CTRL, &ctrl); > + ctrl |= PCI_PTM_CTRL_ENABLE; > + } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { > + pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > + if (!(cap & PCI_PTM_CAP_REQ)) > + return -EINVAL; > + > ups = pci_upstream_bridge(dev); > if (!ups || !ups->ptm_enabled) > return -EINVAL; > > dev->ptm_granularity = ups->ptm_granularity; > + ctrl = PCI_PTM_CTRL_ENABLE; > + ctrl |= dev->ptm_granularity << 8; > } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END) { > + pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > + if (!(cap & PCI_PTM_CAP_REQ)) > + return -EINVAL; > + > dev->ptm_granularity = 0; > + ctrl = PCI_PTM_CTRL_ENABLE; > + ctrl |= dev->ptm_granularity << 8; > } else > return -EINVAL; I would do if ((pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM || pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT)) { ups = pci_upstream_bridge(dev); if (!ups || !ups->ptm_enabled) return -EINVAL; dev->ptm_granularity = ups->ptm_granularity; } switch(pci_pcie_type(dev)) { case PCI_EXP_TYPE_ROOT_PORT: case PCI_EXP_TYPE_UPSTREAM: pci_read_config_dword(dev, pos + PCI_PTM_CTRL, &ctrl); ctrl |= PCI_PTM_CTRL_ENABLE; break; case PCI_EXP_TYPE_ENDPOINT: case PCI_EXP_TYPE_RC_END: ctrl = PCI_PTM_CTRL_ENABLE; break; default: return -EINVAL; } > > - ctrl = PCI_PTM_CTRL_ENABLE; > - ctrl |= dev->ptm_granularity << 8; And I wouldn't remove the line above. Note that for root ports dev->ptm_granularity must be set and reflect the register setting or else the code wouldn't have worked for downstream components. > pci_write_config_dword(dev, pos + PCI_PTM_CTRL, ctrl); > dev->ptm_enabled = 1; > > --
On Sat, Sep 3, 2022 at 7:40 PM Rafael J. Wysocki <rafael@kernel.org> wrote: > > On Sat, Sep 3, 2022 at 1:35 AM Bjorn Helgaas <helgaas@kernel.org> wrote: > > > > From: Bjorn Helgaas <bhelgaas@google.com> > > > > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> > > --- > > drivers/pci/pcie/ptm.c | 34 +++++++++++++++++++++++++++------- > > 1 file changed, 27 insertions(+), 7 deletions(-) > > > > diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c > > index b6a417247ce3..ad283818f37b 100644 > > --- a/drivers/pci/pcie/ptm.c > > +++ b/drivers/pci/pcie/ptm.c > > @@ -167,11 +167,11 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) > > if (!pos) > > return -EINVAL; > > > > - pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > > - if (!(cap & PCI_PTM_CAP_REQ)) > > - return -EINVAL; > > - > > /* > > + * Root Ports and Switch Upstream Ports have been configured > > + * by pci_ptm_init(), so preserve their PCI_PTM_CTRL_ROOT and > > + * granularity. > > + * > > * For a PCIe Endpoint, PTM is only useful if the endpoint can > > * issue PTM requests to upstream devices that have PTM enabled. > > * > > @@ -179,19 +179,39 @@ int pci_enable_ptm(struct pci_dev *dev, u8 *granularity) > > * device, so there must be some implementation-specific way to > > * associate the endpoint with a time source. > > */ > > - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { > > + if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT || > > + pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM) { > > + if (pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM) { > > + ups = pci_upstream_bridge(dev); > > + if (!ups || !ups->ptm_enabled) > > + return -EINVAL; > > + } > > + > > + pci_read_config_dword(dev, pos + PCI_PTM_CTRL, &ctrl); > > + ctrl |= PCI_PTM_CTRL_ENABLE; > > + } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT) { > > + pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > > + if (!(cap & PCI_PTM_CAP_REQ)) > > + return -EINVAL; > > + > > ups = pci_upstream_bridge(dev); > > if (!ups || !ups->ptm_enabled) > > return -EINVAL; > > > > dev->ptm_granularity = ups->ptm_granularity; > > + ctrl = PCI_PTM_CTRL_ENABLE; > > + ctrl |= dev->ptm_granularity << 8; > > } else if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END) { > > + pci_read_config_dword(dev, pos + PCI_PTM_CAP, &cap); > > + if (!(cap & PCI_PTM_CAP_REQ)) > > + return -EINVAL; > > + > > dev->ptm_granularity = 0; > > + ctrl = PCI_PTM_CTRL_ENABLE; > > + ctrl |= dev->ptm_granularity << 8; > > } else > > return -EINVAL; > > I would do > > if ((pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM || pci_pcie_type(dev) > == PCI_EXP_TYPE_ENDPOINT)) { > ups = pci_upstream_bridge(dev); > if (!ups || !ups->ptm_enabled) > return -EINVAL; > > dev->ptm_granularity = ups->ptm_granularity; > } > > switch(pci_pcie_type(dev)) { > case PCI_EXP_TYPE_ROOT_PORT: > case PCI_EXP_TYPE_UPSTREAM: > pci_read_config_dword(dev, pos + PCI_PTM_CTRL, &ctrl); > ctrl |= PCI_PTM_CTRL_ENABLE; > break; > case PCI_EXP_TYPE_ENDPOINT: > case PCI_EXP_TYPE_RC_END: I missed the cap check here, sorry. > ctrl = PCI_PTM_CTRL_ENABLE; > break; > default: > return -EINVAL; > } > > > > > - ctrl = PCI_PTM_CTRL_ENABLE; > > - ctrl |= dev->ptm_granularity << 8; > > And I wouldn't remove the line above. > > Note that for root ports dev->ptm_granularity must be set and reflect > the register setting or else the code wouldn't have worked for > downstream components. > > > pci_write_config_dword(dev, pos + PCI_PTM_CTRL, ctrl); > > dev->ptm_enabled = 1; > > > > --
From: Bjorn Helgaas <bhelgaas@google.com> We currently disable PTM for Root Ports during suspend. Leaving PTM enabled for downstream devices causes UR errors if they send PTM Requests. The intent of this series is to: - Unconditionally disable PTM during suspend (even if the driver saves its own state) by moving the disable from pci_prepare_to_sleep() to pci_pm_suspend(). - Disable PTM for all devices by removing the Root Port condition and doing it early in the suspend paths. - Explicitly re-enable PTM during resume, which requires new support in pci_enable_ptm() for Root Ports and Switch Upstream Ports. Bjorn Helgaas (3): PCI/PTM: Preserve PTM Root Select PCI/PTM: Implement pci_enable_ptm() for Root Ports, Switch Upstream Ports PCI/PM: Always disable PTM for all devices during suspend drivers/pci/pci-driver.c | 14 ++++++++++++++ drivers/pci/pci.c | 20 -------------------- drivers/pci/pcie/ptm.c | 36 ++++++++++++++++++++++++++++-------- 3 files changed, 42 insertions(+), 28 deletions(-)