| Message ID | 4419002.LvFx2qVVIh@kreacher |
|---|---|
| Headers | show |
| Series | PCI/PM: Improvements related to device transitions into D0 | expand |
On Mon, Apr 11, 2022 at 04:21:04PM +0200, Rafael J. Wysocki wrote: > From: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > Save one config space access in pci_update_current_state() by > testing the retireved PCI_PM_CTRL register value against ^^^^^^^^^ retrieved
On Tue, Apr 12, 2022 at 12:54 PM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > On Mon, Apr 11, 2022 at 04:21:04PM +0200, Rafael J. Wysocki wrote: > > From: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > Save one config space access in pci_update_current_state() by > > testing the retireved PCI_PM_CTRL register value against > ^^^^^^^^^ > retrieved Yup, thanks!
Hi Mika, On Tue, Apr 12, 2022 at 1:24 PM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > Hi Rafael, > > On Mon, Apr 11, 2022 at 04:17:41PM +0200, Rafael J. Wysocki wrote: > > Hi All, > > > > On Saturday, April 9, 2022 3:03:14 PM CEST Rafael J. Wysocki wrote: > > > Hi All, > > > > > > This series supersedes the one at > > > > > > https://lore.kernel.org/linux-pm/4198163.ejJDZkT8p0@kreacher > > > > > > It addresses some potential issues related to PCI device transitions from > > > low-power states into D0 and makes the related code more straightforward > > > and so easier to follow. > > > > > > Please refer to the patch changelogs for details. > > > > Here's a v2 of this patch series which is being sent, because I realized that > > one of the checks in pci_power_up() added by patch [4/7] in v1 was redundant > > and can be dropped, but that affected the last 3 patches in the series and > > then I noticed that more improvements were possible and hence the new patches > > [2/9]. > > I sent a few minor nits separately. The series looks good to me in > general and certainly improves readability :) > > Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com> Thank you! I'll address the issues that you have pointed out and send a v3 of this series later this week.
On Wednesday, May 4, 2022 6:36:00 PM CEST Nathan Chancellor wrote: > On Wed, May 04, 2022 at 02:59:17PM +0200, Rafael J. Wysocki wrote: > > On Tue, May 3, 2022 at 7:59 PM Nathan Chancellor <nathan@kernel.org> wrote: > > > > > > Hi Rafael, > > > > > > On Thu, Apr 14, 2022 at 03:11:21PM +0200, Rafael J. Wysocki wrote: > > > > From: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > > > > > There are some issues related to changing power states of PCI > > > > devices, mostly related to carrying out unnecessary actions in some > > > > places, and the code is generally hard to follow. > > > > > > > > 1. pci_power_up() has two callers, pci_set_power_state() and > > > > pci_pm_default_resume_early(). The latter updates the current > > > > power state of the device right after calling pci_power_up() > > > > and it restores the entire config space of the device right > > > > after that, so pci_power_up() itself need not read the > > > > PCI_PM_CTRL register or restore the BARs after programming the > > > > device into D0 in that case. > > > > > > > > 2. It is generally hard to get a clear view of the pci_power_up() > > > > code flow, especially in some corner cases, due to all of the > > > > involved PCI_PM_CTRL register reads and writes occurring in > > > > pci_platform_power_transition() and in pci_raw_set_power_state(), > > > > some of which are redundant. > > > > > > > > 3. The transitions from low-power states to D0 and the other way > > > > around are unnecessarily tangled in pci_raw_set_power_state() > > > > which causes it to use a redundant local variable and makes it > > > > rather hard to follow. > > > > > > > > To address the above shortcomings, make the following changes: > > > > > > > > a. Remove the code handling transitions into D0 > > > > from pci_raw_set_power_state() and rename it as > > > > pci_set_low_power_state(). > > > > > > > > b. Add the code handling transitions into D0 directly > > > > to pci_power_up() and to a new wrapper function > > > > pci_set_full_power_state() calling it internally that is > > > > only used in pci_set_power_state(). > > > > > > > > c. Make pci_power_up() avoid redundant PCI_PM_CTRL register reads > > > > and make it work in the same way for transitions from any > > > > low-power states (transitions from D1 and D2 are handled > > > > slightly differently before the change). > > > > > > > > d. Put the restoration of the BARs and the PCI_PM_CTRL > > > > register read confirming the power state change into > > > > pci_set_full_power_state() to avoid doing that in > > > > pci_pm_default_resume_early() unnecessarily. > > > > > > > > Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > > > > > This change as commit 5bffe4c611f5 ("PCI/PM: Rework changing power > > > states of PCI devices") causes my AMD-based system to fail to fully > > > boot. As far as I can tell, this might be NVMe related, which might make > > > getting a full log difficult, as journalctl won't have anywhere to save > > > it. I see: > > > > > > nvme nvme0: I/O 8 QID 0 timeout, completion polled > > > > > > then shortly afterwards: > > > > > > nvme nvme0: I/O 24 QID 0 timeout, completion polled > > > nvme nvme0: missing or invalid SUBNQN field > > > > > > then I am dropped into an emergency shell. > > > > Thanks for the report! > > > > > This is a log from the previous commit, which may give some hints about > > > the configuration of this particular system. > > > > > > https://gist.github.com/nathanchance/8a56f0939410cb187896e904c72e41e7/raw/b47b2620bdd32d43c7a3b209fcfd9e3d4668f058/good-boot.log > > > > > > If there is any additional debugging information I can provide or > > > patches I can try, please let me know! > > > > Please see what happens if the "if (dev->current_state == PCI_D0)" > > check and the following "return 0" statement in pci_power_up() are > > commented out. > > If I understand you correctly, this? Unfortunately, that does not help. Thanks for testing. Please check if the patch below makes any difference. --- drivers/pci/pci.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) Index: linux-pm/drivers/pci/pci.c =================================================================== --- linux-pm.orig/drivers/pci/pci.c +++ linux-pm/drivers/pci/pci.c @@ -1245,7 +1245,7 @@ int pci_power_up(struct pci_dev *dev) /* There's nothing more to do if current_state is D0 at this point. */ if (dev->current_state == PCI_D0) - return 0; + goto done; /* * Program the device into PCI_D0 by forcing the entire word to 0 (this @@ -1260,6 +1260,11 @@ int pci_power_up(struct pci_dev *dev) udelay(PCI_PM_D2_DELAY); dev->current_state = PCI_D0; + +done: + if (dev->bus->self) + pcie_aspm_pm_state_change(dev->bus->self); + return 1; fail: @@ -1339,9 +1344,6 @@ static int pci_set_full_power_state(stru pci_restore_bars(dev); } - if (dev->bus->self) - pcie_aspm_pm_state_change(dev->bus->self); - return 0; }
On Wed, May 04, 2022 at 08:00:33PM +0200, Rafael J. Wysocki wrote: > On Wednesday, May 4, 2022 6:36:00 PM CEST Nathan Chancellor wrote: > > On Wed, May 04, 2022 at 02:59:17PM +0200, Rafael J. Wysocki wrote: > > > On Tue, May 3, 2022 at 7:59 PM Nathan Chancellor <nathan@kernel.org> wrote: > > > > > > > > Hi Rafael, > > > > > > > > On Thu, Apr 14, 2022 at 03:11:21PM +0200, Rafael J. Wysocki wrote: > > > > > From: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > > > > > > > There are some issues related to changing power states of PCI > > > > > devices, mostly related to carrying out unnecessary actions in some > > > > > places, and the code is generally hard to follow. > > > > > > > > > > 1. pci_power_up() has two callers, pci_set_power_state() and > > > > > pci_pm_default_resume_early(). The latter updates the current > > > > > power state of the device right after calling pci_power_up() > > > > > and it restores the entire config space of the device right > > > > > after that, so pci_power_up() itself need not read the > > > > > PCI_PM_CTRL register or restore the BARs after programming the > > > > > device into D0 in that case. > > > > > > > > > > 2. It is generally hard to get a clear view of the pci_power_up() > > > > > code flow, especially in some corner cases, due to all of the > > > > > involved PCI_PM_CTRL register reads and writes occurring in > > > > > pci_platform_power_transition() and in pci_raw_set_power_state(), > > > > > some of which are redundant. > > > > > > > > > > 3. The transitions from low-power states to D0 and the other way > > > > > around are unnecessarily tangled in pci_raw_set_power_state() > > > > > which causes it to use a redundant local variable and makes it > > > > > rather hard to follow. > > > > > > > > > > To address the above shortcomings, make the following changes: > > > > > > > > > > a. Remove the code handling transitions into D0 > > > > > from pci_raw_set_power_state() and rename it as > > > > > pci_set_low_power_state(). > > > > > > > > > > b. Add the code handling transitions into D0 directly > > > > > to pci_power_up() and to a new wrapper function > > > > > pci_set_full_power_state() calling it internally that is > > > > > only used in pci_set_power_state(). > > > > > > > > > > c. Make pci_power_up() avoid redundant PCI_PM_CTRL register reads > > > > > and make it work in the same way for transitions from any > > > > > low-power states (transitions from D1 and D2 are handled > > > > > slightly differently before the change). > > > > > > > > > > d. Put the restoration of the BARs and the PCI_PM_CTRL > > > > > register read confirming the power state change into > > > > > pci_set_full_power_state() to avoid doing that in > > > > > pci_pm_default_resume_early() unnecessarily. > > > > > > > > > > Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > > Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > > > > > > > This change as commit 5bffe4c611f5 ("PCI/PM: Rework changing power > > > > states of PCI devices") causes my AMD-based system to fail to fully > > > > boot. As far as I can tell, this might be NVMe related, which might make > > > > getting a full log difficult, as journalctl won't have anywhere to save > > > > it. I see: > > > > > > > > nvme nvme0: I/O 8 QID 0 timeout, completion polled > > > > > > > > then shortly afterwards: > > > > > > > > nvme nvme0: I/O 24 QID 0 timeout, completion polled > > > > nvme nvme0: missing or invalid SUBNQN field > > > > > > > > then I am dropped into an emergency shell. > > > > > > Thanks for the report! > > > > > > > This is a log from the previous commit, which may give some hints about > > > > the configuration of this particular system. > > > > > > > > https://gist.github.com/nathanchance/8a56f0939410cb187896e904c72e41e7/raw/b47b2620bdd32d43c7a3b209fcfd9e3d4668f058/good-boot.log > > > > > > > > If there is any additional debugging information I can provide or > > > > patches I can try, please let me know! > > > > > > Please see what happens if the "if (dev->current_state == PCI_D0)" > > > check and the following "return 0" statement in pci_power_up() are > > > commented out. > > > > If I understand you correctly, this? Unfortunately, that does not help. > > Thanks for testing. > > Please check if the patch below makes any difference. Unfortunately, there is still no difference. Even worse, I thought I might be able to get some information from the emergency shell but I don't think the HID driver is loaded yet so my keyboard does not work. I am not sure of how to get any further information from the problematic kernel; if anyone has any ideas, I am happy to test them! I am more than happy to continue to test patches or provide information, I just don't want to be a waste of time :) Cheers, Nathan > --- > drivers/pci/pci.c | 10 ++++++---- > 1 file changed, 6 insertions(+), 4 deletions(-) > > Index: linux-pm/drivers/pci/pci.c > =================================================================== > --- linux-pm.orig/drivers/pci/pci.c > +++ linux-pm/drivers/pci/pci.c > @@ -1245,7 +1245,7 @@ int pci_power_up(struct pci_dev *dev) > > /* There's nothing more to do if current_state is D0 at this point. */ > if (dev->current_state == PCI_D0) > - return 0; > + goto done; > > /* > * Program the device into PCI_D0 by forcing the entire word to 0 (this > @@ -1260,6 +1260,11 @@ int pci_power_up(struct pci_dev *dev) > udelay(PCI_PM_D2_DELAY); > > dev->current_state = PCI_D0; > + > +done: > + if (dev->bus->self) > + pcie_aspm_pm_state_change(dev->bus->self); > + > return 1; > > fail: > @@ -1339,9 +1344,6 @@ static int pci_set_full_power_state(stru > pci_restore_bars(dev); > } > > - if (dev->bus->self) > - pcie_aspm_pm_state_change(dev->bus->self); > - > return 0; > } > > > >
On Wed, May 4, 2022 at 9:35 PM Nathan Chancellor <nathan@kernel.org> wrote: > > On Wed, May 04, 2022 at 08:00:33PM +0200, Rafael J. Wysocki wrote: > > On Wednesday, May 4, 2022 6:36:00 PM CEST Nathan Chancellor wrote: > > > On Wed, May 04, 2022 at 02:59:17PM +0200, Rafael J. Wysocki wrote: > > > > On Tue, May 3, 2022 at 7:59 PM Nathan Chancellor <nathan@kernel.org> wrote: > > > > > > > > > > Hi Rafael, > > > > > > > > > > On Thu, Apr 14, 2022 at 03:11:21PM +0200, Rafael J. Wysocki wrote: > > > > > > From: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > > > > > > > > > There are some issues related to changing power states of PCI > > > > > > devices, mostly related to carrying out unnecessary actions in some > > > > > > places, and the code is generally hard to follow. > > > > > > > > > > > > 1. pci_power_up() has two callers, pci_set_power_state() and > > > > > > pci_pm_default_resume_early(). The latter updates the current > > > > > > power state of the device right after calling pci_power_up() > > > > > > and it restores the entire config space of the device right > > > > > > after that, so pci_power_up() itself need not read the > > > > > > PCI_PM_CTRL register or restore the BARs after programming the > > > > > > device into D0 in that case. > > > > > > > > > > > > 2. It is generally hard to get a clear view of the pci_power_up() > > > > > > code flow, especially in some corner cases, due to all of the > > > > > > involved PCI_PM_CTRL register reads and writes occurring in > > > > > > pci_platform_power_transition() and in pci_raw_set_power_state(), > > > > > > some of which are redundant. > > > > > > > > > > > > 3. The transitions from low-power states to D0 and the other way > > > > > > around are unnecessarily tangled in pci_raw_set_power_state() > > > > > > which causes it to use a redundant local variable and makes it > > > > > > rather hard to follow. > > > > > > > > > > > > To address the above shortcomings, make the following changes: > > > > > > > > > > > > a. Remove the code handling transitions into D0 > > > > > > from pci_raw_set_power_state() and rename it as > > > > > > pci_set_low_power_state(). > > > > > > > > > > > > b. Add the code handling transitions into D0 directly > > > > > > to pci_power_up() and to a new wrapper function > > > > > > pci_set_full_power_state() calling it internally that is > > > > > > only used in pci_set_power_state(). > > > > > > > > > > > > c. Make pci_power_up() avoid redundant PCI_PM_CTRL register reads > > > > > > and make it work in the same way for transitions from any > > > > > > low-power states (transitions from D1 and D2 are handled > > > > > > slightly differently before the change). > > > > > > > > > > > > d. Put the restoration of the BARs and the PCI_PM_CTRL > > > > > > register read confirming the power state change into > > > > > > pci_set_full_power_state() to avoid doing that in > > > > > > pci_pm_default_resume_early() unnecessarily. > > > > > > > > > > > > Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > > > Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > > > > > > > > > This change as commit 5bffe4c611f5 ("PCI/PM: Rework changing power > > > > > states of PCI devices") causes my AMD-based system to fail to fully > > > > > boot. As far as I can tell, this might be NVMe related, which might make > > > > > getting a full log difficult, as journalctl won't have anywhere to save > > > > > it. I see: > > > > > > > > > > nvme nvme0: I/O 8 QID 0 timeout, completion polled > > > > > > > > > > then shortly afterwards: > > > > > > > > > > nvme nvme0: I/O 24 QID 0 timeout, completion polled > > > > > nvme nvme0: missing or invalid SUBNQN field > > > > > > > > > > then I am dropped into an emergency shell. > > > > > > > > Thanks for the report! > > > > > > > > > This is a log from the previous commit, which may give some hints about > > > > > the configuration of this particular system. > > > > > > > > > > https://gist.github.com/nathanchance/8a56f0939410cb187896e904c72e41e7/raw/b47b2620bdd32d43c7a3b209fcfd9e3d4668f058/good-boot.log > > > > > > > > > > If there is any additional debugging information I can provide or > > > > > patches I can try, please let me know! > > > > > > > > Please see what happens if the "if (dev->current_state == PCI_D0)" > > > > check and the following "return 0" statement in pci_power_up() are > > > > commented out. > > > > > > If I understand you correctly, this? Unfortunately, that does not help. > > > > Thanks for testing. > > > > Please check if the patch below makes any difference. > > Unfortunately, there is still no difference. Even worse, I thought I > might be able to get some information from the emergency shell but I > don't think the HID driver is loaded yet so my keyboard does not work. I > am not sure of how to get any further information from the problematic > kernel; if anyone has any ideas, I am happy to test them! I am more than > happy to continue to test patches or provide information, I just don't > want to be a waste of time :) It's not a waste of time if you run tests I ask for. Anyway, I'm going to change the approach, because we're looking for a subtle change in behavior that breaks your system and there are quite a few of these in the problematic patch. I'll post a new series of patches to replace the commits dropped by Bjorn later today. Thanks!