diff mbox series

[5.4,49/62] PCI/MSI: Enable and mask MSI-X early

Message ID 20210816125429.897761686@linuxfoundation.org
State Superseded
Headers show
Series None | expand

Commit Message

Greg KH Aug. 16, 2021, 1:02 p.m. UTC 
From: Thomas Gleixner <tglx@linutronix.de>

commit 438553958ba19296663c6d6583d208dfb6792830 upstream.

The ordering of MSI-X enable in hardware is dysfunctional:

 1) MSI-X is disabled in the control register
 2) Various setup functions
 3) pci_msi_setup_msi_irqs() is invoked which ends up accessing
    the MSI-X table entries
 4) MSI-X is enabled and masked in the control register with the
    comment that enabling is required for some hardware to access
    the MSI-X table

Step #4 obviously contradicts #3. The history of this is an issue with the
NIU hardware. When #4 was introduced the table access actually happened in
msix_program_entries() which was invoked after enabling and masking MSI-X.

This was changed in commit d71d6432e105 ("PCI/MSI: Kill redundant call of
irq_set_msi_desc() for MSI-X interrupts") which removed the table write
from msix_program_entries().

Interestingly enough nobody noticed and either NIU still works or it did
not get any testing with a kernel 3.19 or later.

Nevertheless this is inconsistent and there is no reason why MSI-X can't be
enabled and masked in the control register early on, i.e. move step #4
above to step #1. This preserves the NIU workaround and has no side effects
on other hardware.

Fixes: d71d6432e105 ("PCI/MSI: Kill redundant call of irq_set_msi_desc() for MSI-X interrupts")
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Ashok Raj <ashok.raj@intel.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20210729222542.344136412@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/pci/msi.c |   28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

Comments

Pavel Machek Aug. 17, 2021, 7:36 a.m. UTC | #1 
Hi!

I'm sorry to report here, but 4.4 patches were not yet sent to the
lists (and it may be worth correcting before release).

> +++ b/drivers/pci/msi.c

> @@ -778,18 +778,25 @@ static int msix_capability_init(struct p

...
>  

>  	pci_read_config_word(dev, dev->msix_cap + PCI_MSIX_FLAGS, &control);

>  	/* Request & Map MSI-X table region */

>  	base = msix_map_region(dev, msix_table_size(control));

> -	if (!base)

> -		return -ENOMEM;

> +	if (!base) {

> +		ret = -ENOMEM;

> +		goto out_disable;

> +	}

>  

>  	ret = msix_setup_entries(dev, base, entries, nvec, affd);

>  	if (ret)


This one is correct, and so is the version queued for 4.19, but 4.4
version (9da69496e86237e94c4ffa2a5b375a4d2ee7c482) has:

 /* Request & Map MSI-X table region */
    	  base = msix_map_region(dev, msix_table_size(control));
 -       if (!base)
 +       if (!base) {
                 return -ENOMEM;
 +               goto out_disable;
 +       }
 
	ret = msix_setup_entries(dev, base, entries, nvec);
	

That return is misplaced, it should be ret = -ENOMEM, similar to 4.19
and newer.

Best regards,
								Pavel
--
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Greg KH Aug. 18, 2021, 6:53 a.m. UTC | #2 
On Tue, Aug 17, 2021 at 09:36:55AM +0200, Pavel Machek wrote:
> Hi!

> 

> I'm sorry to report here, but 4.4 patches were not yet sent to the

> lists (and it may be worth correcting before release).


Yes, they are known to not be complete and incorrect at the moment,
others have reported this to me.  I will be working on these later
today, thanks.

greg k-h
Greg KH Aug. 18, 2021, 9:19 a.m. UTC | #3 
On Wed, Aug 18, 2021 at 08:53:45AM +0200, Greg Kroah-Hartman wrote:
> On Tue, Aug 17, 2021 at 09:36:55AM +0200, Pavel Machek wrote:

> > Hi!

> > 

> > I'm sorry to report here, but 4.4 patches were not yet sent to the

> > lists (and it may be worth correcting before release).

> 

> Yes, they are known to not be complete and incorrect at the moment,

> others have reported this to me.  I will be working on these later

> today, thanks.


Now should be all fixed up thanks to some patches sent by Thomas.

greg k-h
Pavel Machek Aug. 18, 2021, 7:30 p.m. UTC | #4 
On Wed 2021-08-18 11:19:58, Greg Kroah-Hartman wrote:
> On Wed, Aug 18, 2021 at 08:53:45AM +0200, Greg Kroah-Hartman wrote:

> > On Tue, Aug 17, 2021 at 09:36:55AM +0200, Pavel Machek wrote:

> > > Hi!

> > > 

> > > I'm sorry to report here, but 4.4 patches were not yet sent to the

> > > lists (and it may be worth correcting before release).

> > 

> > Yes, they are known to not be complete and incorrect at the moment,

> > others have reported this to me.  I will be working on these later

> > today, thanks.

> 

> Now should be all fixed up thanks to some patches sent by Thomas.


I can't spot a bug any more; thank you,
								Pavel
-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
diff mbox series

Patch

--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -778,18 +778,25 @@  static int msix_capability_init(struct p
 	u16 control;
 	void __iomem *base;
 
-	/* Ensure MSI-X is disabled while it is set up */
-	pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_ENABLE, 0);
+	/*
+	 * Some devices require MSI-X to be enabled before the MSI-X
+	 * registers can be accessed.  Mask all the vectors to prevent
+	 * interrupts coming in before they're fully set up.
+	 */
+	pci_msix_clear_and_set_ctrl(dev, 0, PCI_MSIX_FLAGS_MASKALL |
+				    PCI_MSIX_FLAGS_ENABLE);
 
 	pci_read_config_word(dev, dev->msix_cap + PCI_MSIX_FLAGS, &control);
 	/* Request & Map MSI-X table region */
 	base = msix_map_region(dev, msix_table_size(control));
-	if (!base)
-		return -ENOMEM;
+	if (!base) {
+		ret = -ENOMEM;
+		goto out_disable;
+	}
 
 	ret = msix_setup_entries(dev, base, entries, nvec, affd);
 	if (ret)
-		return ret;
+		goto out_disable;
 
 	ret = pci_msi_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSIX);
 	if (ret)
@@ -800,14 +807,6 @@  static int msix_capability_init(struct p
 	if (ret)
 		goto out_free;
 
-	/*
-	 * Some devices require MSI-X to be enabled before we can touch the
-	 * MSI-X registers.  We need to mask all the vectors to prevent
-	 * interrupts coming in before they're fully set up.
-	 */
-	pci_msix_clear_and_set_ctrl(dev, 0,
-				PCI_MSIX_FLAGS_MASKALL | PCI_MSIX_FLAGS_ENABLE);
-
 	msix_program_entries(dev, entries);
 
 	ret = populate_msi_sysfs(dev);
@@ -842,6 +841,9 @@  out_avail:
 out_free:
 	free_msi_irqs(dev);
 
+out_disable:
+	pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_ENABLE, 0);
+
 	return ret;
 }