From patchwork Thu Jun 22 12:15:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 106203 Delivered-To: patch@linaro.org Received: by 10.140.91.2 with SMTP id y2csp83646qgd; Thu, 22 Jun 2017 05:16:35 -0700 (PDT) X-Received: by 10.84.224.207 with SMTP id k15mr2612426pln.15.1498133795691; Thu, 22 Jun 2017 05:16:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1498133795; cv=none; d=google.com; s=arc-20160816; b=TIZu9q8Zu2k5O/B57Clws3C8WwQlUhkDuIMT+cHFqne6ejbbnaruvV0EWl3GEX+R4N KGnlHeBUQarRyaZpe7p3Uo0EneCRzJlnRDI0K0YYlE7hIFETHbCK4OmTZnhj95rWb1Dg hIpkzicamBx++XCGzGwKbtSmGOXZLuVoNMG1dqKcyEq0b6UX2nsXDKcM9VlrBLCkfFdK bY9owlzu4t754TYksfe1HLgdrHjB5ZOOpQ43W4BlhJHUI2eclJPHxML25dqY0BFTARF/ cv0MA4kVY0EO0uL8pN0mVG2Z+SP8pa3XDdMB89anFcNRD23QitIhlXHh31gzk/rmB28B qxtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=3hEFr3fTvX/Gf17m88pQ3cHmaFAD+KbppIoX/NFZBUk=; b=XgTxnLtxGyYILfPYgr+9MPUF/1uIwEegubrlonNSrERqiwhai5tYLC+WB86OkBX1+z vIL3S9qzFO054SybxuxXQ8ze+F2izFyPYwyu0tvlncFc5oOxzGvsFgc7LcbX0tQZsKn9 UkKDWVj5sf8oOns1Q8+e7EtMUzpXgXzonXIfSpHq7hBF2UXcwc2XDQmL3K0S5/ge/uZv hjoN+JHAr5p+dB19cUwt8FgcUrnN4ZoIq+kDD5JT3pQsq1iR8S01CiSC9w3R7t2W030E DXWvoMRCiPdeSfispXWU8CAvnYkyFTIwfc6QkCfiGwfG76P7KBpCQwF9922Cq+UCu1ye CxKg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g3si1062430pgu.85.2017.06.22.05.16.34; Thu, 22 Jun 2017 05:16:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753246AbdFVMQM (ORCPT + 25 others); Thu, 22 Jun 2017 08:16:12 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:8815 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751893AbdFVMQJ (ORCPT ); Thu, 22 Jun 2017 08:16:09 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id APV26033; Thu, 22 Jun 2017 20:15:39 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Thu, 22 Jun 2017 20:15:26 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v6 1/3] PCI: Add new PCIe Fabric End Node flag, PCI_DEV_FLAGS_NO_RELAXED_ORDERING Date: Thu, 22 Jun 2017 20:15:19 +0800 Message-ID: <1498133721-21152-2-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1498133721-21152-1-git-send-email-dingtianhong@huawei.com> References: <1498133721-21152-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0205.594BB4EE.0096, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: f05a93592527ccb9b35125b1472050d2 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Casey Leedom The new flag PCI_DEV_FLAGS_NO_RELAXED_ORDERING indicates that the Relaxed Ordering Attribute should not be used on Transaction Layer Packets destined for the PCIe End Node so flagged. Initially flagged this way are Intel E5-26xx Root Complex Ports which suffer from a Flow Control Credit Performance Problem and AMD A1100 ARM ("SEATTLE") Root Complex Ports which don't obey PCIe 3.0 ordering rules which can lead to Data Corruption. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong --- drivers/pci/quirks.c | 38 ++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 2 files changed, 40 insertions(+) -- 1.9.0 diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 085fb78..58bdd23 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -3999,6 +3999,44 @@ static void quirk_tw686x_class(struct pci_dev *pdev) quirk_tw686x_class); /* + * Some devices have problems with Transaction Layer Packets with the Relaxed + * Ordering Attribute set. Such devices should mark themselves and other + * Device Drivers should check before sending TLPs with RO set. + */ +static void quirk_relaxedordering_disable(struct pci_dev *dev) +{ + dev->dev_flags |= PCI_DEV_FLAGS_NO_RELAXED_ORDERING; +} + +/* + * Intel E5-26xx Root Complex has a Flow Control Credit issue which can + * cause performance problems with Upstream Transaction Layer Packets with + * Relaxed Ordering set. + */ +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f02, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f04, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f08, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); + +/* + * The AMD ARM A1100 (AKA "SEATTLE") SoC has a bug in its PCIe Root Complex + * where Upstream Transaction Layer Packets with the Relaxed Ordering + * Attribute clear are allowed to bypass earlier TLPs with Relaxed Ordering + * set. This is a violation of the PCIe 3.0 Transaction Ordering Rules + * outlined in Section 2.4.1 (PCI Express(r) Base Specification Revision 3.0 + * November 10, 2010). As a result, on this platform we can't use Relaxed + * Ordering for Upstream TLPs. + */ +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a00, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a01, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a02, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); + +/* * Per PCIe r3.0, sec 2.2.9, "Completion headers must supply the same * values for the Attribute as were supplied in the header of the * corresponding Request, except as explicitly allowed when IDO is used." diff --git a/include/linux/pci.h b/include/linux/pci.h index 8039f9f..ce77690 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -188,6 +188,8 @@ enum pci_dev_flags { * the direct_complete optimization. */ PCI_DEV_FLAGS_NEEDS_RESUME = (__force pci_dev_flags_t) (1 << 11), + /* Don't use Relaxed Ordering for TLPs directed at this device */ + PCI_DEV_FLAGS_NO_RELAXED_ORDERING = (__force pci_dev_flags_t) (1 << 12), }; enum pci_irq_reroute_variant { From patchwork Thu Jun 22 12:15:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 106205 Delivered-To: patch@linaro.org Received: by 10.140.91.2 with SMTP id y2csp83969qgd; Thu, 22 Jun 2017 05:17:16 -0700 (PDT) X-Received: by 10.99.189.18 with SMTP id a18mr2357912pgf.195.1498133836552; Thu, 22 Jun 2017 05:17:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1498133836; cv=none; d=google.com; s=arc-20160816; b=Uu+y98AJK9749MwVr8ZpKgeOhHNPxKASGRpaFeH2SIyb0B0fEawmmRMTZqiUrrgxJD ijSUatBSIUrZ72ICJrBDHJm82mhNhe8S413mqg1EkfpPENhrDrX1MRKxWzdQmbisYxvT QrSXV3a2LYXIz4fyfbvz/hclZFOzB0aF8Qw/U0xoJPFJ7lLrGfdYHFFNzdvCFQaZQJ3C v+0LK9LHDtXF0unz9azafzEZ+ZxBchgzeLRTyIcimbxhM6SpgR2Xjru4KdOH58ycY0h7 w0FGMH7AGS41ftjsVUF+l6+AgnTHqgoifapstV0dHfUD0eL+BLIYP+LDFLiuD2vPfctF G7tQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=4fJnk7fBE9W+0iw3uy7Sz1CjBoBvbYNUwXwkvI4ic6A=; b=wceMVaPSDdar/nCyas8DtQUCbLv6UHOaT+5X/QKlGDSJOM+bVlfa6jazfBPVx1Vh9Z qL7kLCIiZQynRiiXE6I4Iwlv+SyqplXI9pI4ZU6pekcmkPmeprdFLndEbm20gAfcLunf mMuT13b7K0wMj8LTpQ3i+Eos1Md4zd3XZEMvnbRijl/czAZXWXpELQctTgqYdEsZVi6p y076dF/d7hAdW+noFsyvv6Ubnh7QgarlelBottmGAE5SZ72CYR8hXohvE578TJaujHbH 2OBRzBr4oMhYvjNwfzbYMj6NQJW3oD3i31yDM4kg1kwc3StMlapWuGNrx6cHriAY1xoQ dSsg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g8si1107368plm.564.2017.06.22.05.17.16; Thu, 22 Jun 2017 05:17:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753266AbdFVMQo (ORCPT + 25 others); Thu, 22 Jun 2017 08:16:44 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:8814 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752988AbdFVMQA (ORCPT ); Thu, 22 Jun 2017 08:16:00 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id APV26036; Thu, 22 Jun 2017 20:15:40 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Thu, 22 Jun 2017 20:15:27 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v6 2/3] PCI: Enable PCIe Relaxed Ordering if supported Date: Thu, 22 Jun 2017 20:15:20 +0800 Message-ID: <1498133721-21152-3-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1498133721-21152-1-git-send-email-dingtianhong@huawei.com> References: <1498133721-21152-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0205.594BB4ED.01CE, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 6ae1e3f60a0ae7e2f6773a630d846e58 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The PCIe Device Control Register use the bit 4 to indicate that whether the device is permitted to enable relaxed ordering or not. But relaxed ordering is not safe for some platform which could only use strong write ordering, so devices are allowed (but not required) to enable relaxed ordering bit by default. If a PCIe device didn't enable the relaxed ordering attribute default, we should not do anything in the PCIe configuration, otherwise we should check if any of the devices above us do not support relaxed ordering by the PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag, then base on the result if we get a return that indicate that the relaxed ordering is not supported we should update our device to disable relaxed ordering in configuration space. If the device above us doesn't exist or isn't the PCIe device, we shouldn't do anything and skip updating relaxed ordering because we are probably running in a guest machine. Signed-off-by: Ding Tianhong --- drivers/pci/pci.c | 29 +++++++++++++++++++++++++++++ drivers/pci/probe.c | 37 +++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 3 files changed, 68 insertions(+) -- 1.9.0 diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 563901c..c773e0d 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4879,6 +4879,35 @@ int pcie_set_mps(struct pci_dev *dev, int mps) EXPORT_SYMBOL(pcie_set_mps); /** + * pcie_clear_relaxed_ordering - clear PCI Express relaxed ordering bit + * @dev: PCI device to query + * + * If possible clear relaxed ordering + */ +int pcie_clear_relaxed_ordering(struct pci_dev *dev) +{ + return pcie_capability_clear_word(dev, PCI_EXP_DEVCTL, + PCI_EXP_DEVCTL_RELAX_EN); +} +EXPORT_SYMBOL(pcie_clear_relaxed_ordering); + +/** + * pcie_relaxed_ordering_supported - Probe for PCIe relexed ordering support + * @dev: PCI device to query + * + * Returns true if the device support relaxed ordering attribute. + */ +bool pcie_relaxed_ordering_supported(struct pci_dev *dev) +{ + u16 v; + + pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &v); + + return !!(v & PCI_EXP_DEVCTL_RELAX_EN); +} +EXPORT_SYMBOL(pcie_relaxed_ordering_supported); + +/** * pcie_get_minimum_link - determine minimum link settings of a PCI device * @dev: PCI device to query * @speed: storage for minimum speed diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 19c8950..2615299 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1701,6 +1701,42 @@ static void pci_configure_extended_tags(struct pci_dev *dev) PCI_EXP_DEVCTL_EXT_TAG); } +/** + * pci_dev_should_disable_relaxed_ordering - check if the PCI device + * should disable the relaxed ordering attribute. + * @dev: PCI device + * + * Return true if any of the PCI devices above us do not support + * relaxed ordering. + */ +static bool pci_dev_should_disable_relaxed_ordering(struct pci_dev *dev) +{ + while (dev) { + if (dev->dev_flags & PCI_DEV_FLAGS_NO_RELAXED_ORDERING) + return true; + + dev = dev->bus->self; + } + + return false; +} + +static void pci_configure_relaxed_ordering(struct pci_dev *dev) +{ + /* We should not alter the relaxed ordering bit for the VF */ + if (dev->is_virtfn) + return; + + /* If the releaxed ordering enable bit is not set, do nothing. */ + if (!pcie_relaxed_ordering_supported(dev)) + return; + + if (pci_dev_should_disable_relaxed_ordering(dev)) { + pcie_clear_relaxed_ordering(dev); + dev_info(&dev->dev, "Disable Relaxed Ordering\n"); + } +} + static void pci_configure_device(struct pci_dev *dev) { struct hotplug_params hpp; @@ -1708,6 +1744,7 @@ static void pci_configure_device(struct pci_dev *dev) pci_configure_mps(dev); pci_configure_extended_tags(dev); + pci_configure_relaxed_ordering(dev); memset(&hpp, 0, sizeof(hpp)); ret = pci_get_hp_params(dev, &hpp); diff --git a/include/linux/pci.h b/include/linux/pci.h index ce77690..f5f200f 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1110,6 +1110,8 @@ int __pci_enable_wake(struct pci_dev *dev, pci_power_t state, void pci_pme_wakeup_bus(struct pci_bus *bus); void pci_d3cold_enable(struct pci_dev *dev); void pci_d3cold_disable(struct pci_dev *dev); +int pcie_clear_relaxed_ordering(struct pci_dev *dev); +bool pcie_relaxed_ordering_supported(struct pci_dev *dev); static inline int pci_enable_wake(struct pci_dev *dev, pci_power_t state, bool enable) From patchwork Thu Jun 22 12:15:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 106202 Delivered-To: patch@linaro.org Received: by 10.140.91.2 with SMTP id y2csp83481qgd; Thu, 22 Jun 2017 05:16:13 -0700 (PDT) X-Received: by 10.99.127.17 with SMTP id a17mr2304977pgd.213.1498133773762; Thu, 22 Jun 2017 05:16:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1498133773; cv=none; d=google.com; s=arc-20160816; b=SlsrMTzmX7/isGSzngCU1meDc6dTK0uaCr9cKpPXFROF7QXH6Ix6Gg0rkGe2Zb6EHc b61txF82/sEEuqu2QKKyZrZn1NhICxbpH2u1WWixWfApmsTgg756vQsG/tcKKkawiZqx spghYfuKKMHQNVjLRd07EKTy6IBCixaA2G6SoIt+GIVnpb4ysvHNFqvFzAns4kdqiPbq Z7SE1emMTskG3GCgRr2bqdbobzoM92rxfwo03IYoSRsjlvRbVOeLHUc5OkLrhAq1aRgl cJjdoU6Y2rqXGe1mMRdrfv+xF9PINXhw/TRVbhPK+jIgGdaTSHQZuEzfZB8vD1o4Gl7A FiHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=sfzn7sXTp6qYauQo0xbzqkFvhZRQOpbq9Mqx12swldc=; b=ifJK4/yiuIkd45HnmRa/Rcr+iWZUvsMJJb2EFEFsi+yhg7UET8uvOoKcF8tv6Hao0K /bquEjaEnjRM1M1DVIXPAuN7AWZ+qgsNSskXu0QSmrVbXMpy3zPyWplDCxvMEzHp2lwy Jj+wlGfBZGw0ZDvRpPxQDiAuIGi4gECXeS7Fhvg/nJlVtnLMckIi7M7W/p0bch2/6s3h /KxHOPYbaMM4rqiy7PlQuFSwYdVwJQQW/PTU1K0EXFa3+zLedz1NzNQBUZOqFPSEsvHk viOlcs5Kjf569PbR9DYpufCvM+5sjE2fnSyLiYMeAhDD5qFY/SlrLcgbR9ZKDBRVNAps Mz1A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g13si836854plm.355.2017.06.22.05.16.13; Thu, 22 Jun 2017 05:16:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753213AbdFVMQC (ORCPT + 25 others); Thu, 22 Jun 2017 08:16:02 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:8813 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751920AbdFVMP6 (ORCPT ); Thu, 22 Jun 2017 08:15:58 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id APV26035; Thu, 22 Jun 2017 20:15:40 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Thu, 22 Jun 2017 20:15:29 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v6 3/3] net/cxgb4: Use new PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag Date: Thu, 22 Jun 2017 20:15:21 +0800 Message-ID: <1498133721-21152-4-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1498133721-21152-1-git-send-email-dingtianhong@huawei.com> References: <1498133721-21152-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020202.594BB4ED.0241, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 40dada7c9e4d676c39e21d9948ad07dd Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Casey Leedom cxgb4 Ethernet driver now queries PCIe configuration space to determine if it can send TLPs to it with the Relaxed Ordering Attribute set. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong --- drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 1 + drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 17 +++++++++++++++++ drivers/net/ethernet/chelsio/cxgb4/sge.c | 5 +++-- 3 files changed, 21 insertions(+), 2 deletions(-) -- 1.9.0 diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h index e88c180..478f25a 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h @@ -521,6 +521,7 @@ enum { /* adapter flags */ USING_SOFT_PARAMS = (1 << 6), MASTER_PF = (1 << 7), FW_OFLD_CONN = (1 << 9), + ROOT_NO_RELAXED_ORDERING = (1 << 10), }; enum { diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c index ea1bfcf..0c4a6e9 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c @@ -4735,6 +4735,23 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent) adapter->msg_enable = DFLT_MSG_ENABLE; memset(adapter->chan_map, 0xff, sizeof(adapter->chan_map)); + /* If possible, we use PCIe Relaxed Ordering Attribute to deliver + * Ingress Packet Data to Free List Buffers in order to allow for + * chipset performance optimizations between the Root Complex and + * Memory Controllers. (Messages to the associated Ingress Queue + * notifying new Packet Placement in the Free Lists Buffers will be + * send without the Relaxed Ordering Attribute thus guaranteeing that + * all preceding PCIe Transaction Layer Packets will be processed + * first.) But some Root Complexes have various issues with Upstream + * Transaction Layer Packets with the Relaxed Ordering Attribute set. + * The PCIe devices which under the Root Complexes will be cleared the + * Relaxed Ordering bit in the configuration space, So we check our + * PCIe configuration space to see if it's flagged with advice against + * using Relaxed Ordering. + */ + if (!pcie_relaxed_ordering_supported(pdev)) + adapter->flags |= ROOT_NO_RELAXED_ORDERING; + spin_lock_init(&adapter->stats_lock); spin_lock_init(&adapter->tid_release_lock); spin_lock_init(&adapter->win0_lock); diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c index f05f0d4..ac229a3 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c @@ -2571,6 +2571,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, struct fw_iq_cmd c; struct sge *s = &adap->sge; struct port_info *pi = netdev_priv(dev); + int relaxed = !(adap->flags & ROOT_NO_RELAXED_ORDERING); /* Size needs to be multiple of 16, including status entry. */ iq->size = roundup(iq->size, 16); @@ -2624,8 +2625,8 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, flsz = fl->size / 8 + s->stat_len / sizeof(struct tx_desc); c.iqns_to_fl0congen |= htonl(FW_IQ_CMD_FL0PACKEN_F | - FW_IQ_CMD_FL0FETCHRO_F | - FW_IQ_CMD_FL0DATARO_F | + FW_IQ_CMD_FL0FETCHRO_V(relaxed) | + FW_IQ_CMD_FL0DATARO_V(relaxed) | FW_IQ_CMD_FL0PADEN_F); if (cong >= 0) c.iqns_to_fl0congen |=