From patchwork Mon Jun 19 06:53:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 105798 Delivered-To: patch@linaro.org Received: by 10.140.91.2 with SMTP id y2csp742342qgd; Sun, 18 Jun 2017 23:54:59 -0700 (PDT) X-Received: by 10.84.232.5 with SMTP id h5mr1678962plk.261.1497855298947; Sun, 18 Jun 2017 23:54:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497855298; cv=none; d=google.com; s=arc-20160816; b=RgykSiVGmlWBU8bqvY57ktX6ZfOiKlyHcZ2jHHgPfwYoZUKXbFuuEnpggAZoAtfvAt RRASEopTvszpvc5CeTGhO8+fpvli1e/Ul7/JpSCIKdrf1spLLlPEJHzAMnb4A6OuO3xX FyWJOFeR7QuJBLE2fDjOaF9GEfEfqaJnZIk0olRhhnoptIMTrbOqWlhbJHybC7RcIlXJ tT8LvvSsCZMglkiHDsutW32q2ekmg66AuFHs3ngkuyg/CZMsOy03XY2p+oRQ8d0y/AAO IgHkwEgTborqiR/Zx2KOsTwrGhbhLxI29xiqSq3CKkGJsyAmzjPkWhl7joEirDMGaLLJ Q1Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=3hEFr3fTvX/Gf17m88pQ3cHmaFAD+KbppIoX/NFZBUk=; b=UINDX18msIYGUmkt5RSgsWMxk4cdZdgQRzyvtI4E/W8Ksjwy8l1O9P0ylUF8LxtJLy n3VTSMooUGgVmmSO34BWAFEIz3r7KF7VD1LukxCQEW/YdkbCEN7wmeh6Bv7efgSA8VcO EDWelP85xjtNXTqE99ANojzrx6j/pK6PmToyCWuo/7JUiLd/Zauc1Zqw/F+0ZoDuvT68 ne/pV6PKLJxHMIyylvCZQyfeYipRZnvjQYSDoTyBBSnTpTtEE3ZItKIOEdMYR9ahg5th vPAU2UJwYhK6A6ZhD19gjRHU2Cd3Ps+2L9hB2Qvb5Up/x7M8vHdaWRg7p4PceghXFLdf H6lQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h17si7324908pgn.513.2017.06.18.23.54.58; Sun, 18 Jun 2017 23:54:58 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753742AbdFSGyw (ORCPT + 25 others); Mon, 19 Jun 2017 02:54:52 -0400 Received: from szxga03-in.huawei.com ([45.249.212.189]:7890 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753324AbdFSGyt (ORCPT ); Mon, 19 Jun 2017 02:54:49 -0400 Received: from 172.30.72.55 (EHLO DGGEML401-HUB.china.huawei.com) ([172.30.72.55]) by dggrg03-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id APP89445; Mon, 19 Jun 2017 14:54:19 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEML401-HUB.china.huawei.com (10.3.17.32) with Microsoft SMTP Server id 14.3.301.0; Mon, 19 Jun 2017 14:54:07 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v5 1/3] PCI: Add new PCIe Fabric End Node flag, PCI_DEV_FLAGS_NO_RELAXED_ORDERING Date: Mon, 19 Jun 2017 14:53:57 +0800 Message-ID: <1497855239-18560-2-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1497855239-18560-1-git-send-email-dingtianhong@huawei.com> References: <1497855239-18560-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020205.5947751E.00FC, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 396219e948d26301079b14bbdaf72f47 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Casey Leedom The new flag PCI_DEV_FLAGS_NO_RELAXED_ORDERING indicates that the Relaxed Ordering Attribute should not be used on Transaction Layer Packets destined for the PCIe End Node so flagged. Initially flagged this way are Intel E5-26xx Root Complex Ports which suffer from a Flow Control Credit Performance Problem and AMD A1100 ARM ("SEATTLE") Root Complex Ports which don't obey PCIe 3.0 ordering rules which can lead to Data Corruption. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong --- drivers/pci/quirks.c | 38 ++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 2 files changed, 40 insertions(+) -- 1.9.0 diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 085fb78..58bdd23 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -3999,6 +3999,44 @@ static void quirk_tw686x_class(struct pci_dev *pdev) quirk_tw686x_class); /* + * Some devices have problems with Transaction Layer Packets with the Relaxed + * Ordering Attribute set. Such devices should mark themselves and other + * Device Drivers should check before sending TLPs with RO set. + */ +static void quirk_relaxedordering_disable(struct pci_dev *dev) +{ + dev->dev_flags |= PCI_DEV_FLAGS_NO_RELAXED_ORDERING; +} + +/* + * Intel E5-26xx Root Complex has a Flow Control Credit issue which can + * cause performance problems with Upstream Transaction Layer Packets with + * Relaxed Ordering set. + */ +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f02, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f04, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f08, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); + +/* + * The AMD ARM A1100 (AKA "SEATTLE") SoC has a bug in its PCIe Root Complex + * where Upstream Transaction Layer Packets with the Relaxed Ordering + * Attribute clear are allowed to bypass earlier TLPs with Relaxed Ordering + * set. This is a violation of the PCIe 3.0 Transaction Ordering Rules + * outlined in Section 2.4.1 (PCI Express(r) Base Specification Revision 3.0 + * November 10, 2010). As a result, on this platform we can't use Relaxed + * Ordering for Upstream TLPs. + */ +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a00, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a01, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a02, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); + +/* * Per PCIe r3.0, sec 2.2.9, "Completion headers must supply the same * values for the Attribute as were supplied in the header of the * corresponding Request, except as explicitly allowed when IDO is used." diff --git a/include/linux/pci.h b/include/linux/pci.h index 8039f9f..ce77690 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -188,6 +188,8 @@ enum pci_dev_flags { * the direct_complete optimization. */ PCI_DEV_FLAGS_NEEDS_RESUME = (__force pci_dev_flags_t) (1 << 11), + /* Don't use Relaxed Ordering for TLPs directed at this device */ + PCI_DEV_FLAGS_NO_RELAXED_ORDERING = (__force pci_dev_flags_t) (1 << 12), }; enum pci_irq_reroute_variant { From patchwork Mon Jun 19 06:53:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 105799 Delivered-To: patch@linaro.org Received: by 10.140.91.2 with SMTP id y2csp742404qgd; Sun, 18 Jun 2017 23:55:16 -0700 (PDT) X-Received: by 10.99.44.19 with SMTP id s19mr24669077pgs.257.1497855316079; Sun, 18 Jun 2017 23:55:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497855316; cv=none; d=google.com; s=arc-20160816; b=QnMHWP/zNyZLu0tSsjdeEf79osa95NqpmbTsiaiUUar+Wgrr3oG1g7XCHwhnAB+W8+ wN4K7kDk4vrztfx/sdULvTg0DAdOx1PwrcD7bVMgFqzlUpGE6QUPI8J6em5TV1FrqFg4 BMDGWzExjxZiWvHcttAGUNFYvRklbbMFbOLBzyeVKSZGdC9O8zFyyiMjBS3LFsi5NH2Y ePDBpFM8Q1i2sFV/a16Uacf0Wp5k6CfA/gGAP/RW6M/BxzAbRQeuskoOf2dySGBPgTo7 kHjy5Jdasqh851oxni1MnI4FXmfuXHPzL3+m/4gwnsx+f9wAejxs2d4Uj/XNNhAqJgBw uKCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=4fJnk7fBE9W+0iw3uy7Sz1CjBoBvbYNUwXwkvI4ic6A=; b=mIW3o923x0jb6aRmW4vrmSDEIlC9uNGLVRIdC4hb/5cmAgBKOnGxY3EpNN1W7BmqMx cCyIrrR6C7HMRzCP+rACnI5f1FPUAzNlHMU4Bt9FcZIJn8VINVTfGbfBmCiPyB97EeJk 4aL6P/3Vdv0DCmBkuUIKlFAVrGXyxBeLrVb7B5GY5ccYzNCBhfF199X4i/poz9P0wuHh MPykt03UrgxbDEZ565UBD9pB/ejYbgI6MoGIbVETLWIv1F+++9uW/hyrRT7fY230HFRV 14kJnSHI0VLYam5XKUeE5SOK8r+8xe+v0RPL8lI4bYbmddoaNxWKYaWw3Ws1kTXF0/VE JclA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i127si5278237pgc.170.2017.06.18.23.55.15; Sun, 18 Jun 2017 23:55:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753645AbdFSGym (ORCPT + 25 others); Mon, 19 Jun 2017 02:54:42 -0400 Received: from szxga03-in.huawei.com ([45.249.212.189]:7891 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753348AbdFSGyi (ORCPT ); Mon, 19 Jun 2017 02:54:38 -0400 Received: from 172.30.72.55 (EHLO DGGEML401-HUB.china.huawei.com) ([172.30.72.55]) by dggrg03-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id APP89461; Mon, 19 Jun 2017 14:54:21 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEML401-HUB.china.huawei.com (10.3.17.32) with Microsoft SMTP Server id 14.3.301.0; Mon, 19 Jun 2017 14:54:09 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v5 2/3] PCI: Enable PCIe Relaxed Ordering if supported Date: Mon, 19 Jun 2017 14:53:58 +0800 Message-ID: <1497855239-18560-3-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1497855239-18560-1-git-send-email-dingtianhong@huawei.com> References: <1497855239-18560-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020201.5947751E.00F6, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 9fcde18034790ce85695d87fb335930b Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The PCIe Device Control Register use the bit 4 to indicate that whether the device is permitted to enable relaxed ordering or not. But relaxed ordering is not safe for some platform which could only use strong write ordering, so devices are allowed (but not required) to enable relaxed ordering bit by default. If a PCIe device didn't enable the relaxed ordering attribute default, we should not do anything in the PCIe configuration, otherwise we should check if any of the devices above us do not support relaxed ordering by the PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag, then base on the result if we get a return that indicate that the relaxed ordering is not supported we should update our device to disable relaxed ordering in configuration space. If the device above us doesn't exist or isn't the PCIe device, we shouldn't do anything and skip updating relaxed ordering because we are probably running in a guest machine. Signed-off-by: Ding Tianhong --- drivers/pci/pci.c | 29 +++++++++++++++++++++++++++++ drivers/pci/probe.c | 37 +++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 3 files changed, 68 insertions(+) -- 1.9.0 diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 563901c..c773e0d 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4879,6 +4879,35 @@ int pcie_set_mps(struct pci_dev *dev, int mps) EXPORT_SYMBOL(pcie_set_mps); /** + * pcie_clear_relaxed_ordering - clear PCI Express relaxed ordering bit + * @dev: PCI device to query + * + * If possible clear relaxed ordering + */ +int pcie_clear_relaxed_ordering(struct pci_dev *dev) +{ + return pcie_capability_clear_word(dev, PCI_EXP_DEVCTL, + PCI_EXP_DEVCTL_RELAX_EN); +} +EXPORT_SYMBOL(pcie_clear_relaxed_ordering); + +/** + * pcie_relaxed_ordering_supported - Probe for PCIe relexed ordering support + * @dev: PCI device to query + * + * Returns true if the device support relaxed ordering attribute. + */ +bool pcie_relaxed_ordering_supported(struct pci_dev *dev) +{ + u16 v; + + pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &v); + + return !!(v & PCI_EXP_DEVCTL_RELAX_EN); +} +EXPORT_SYMBOL(pcie_relaxed_ordering_supported); + +/** * pcie_get_minimum_link - determine minimum link settings of a PCI device * @dev: PCI device to query * @speed: storage for minimum speed diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 19c8950..2615299 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1701,6 +1701,42 @@ static void pci_configure_extended_tags(struct pci_dev *dev) PCI_EXP_DEVCTL_EXT_TAG); } +/** + * pci_dev_should_disable_relaxed_ordering - check if the PCI device + * should disable the relaxed ordering attribute. + * @dev: PCI device + * + * Return true if any of the PCI devices above us do not support + * relaxed ordering. + */ +static bool pci_dev_should_disable_relaxed_ordering(struct pci_dev *dev) +{ + while (dev) { + if (dev->dev_flags & PCI_DEV_FLAGS_NO_RELAXED_ORDERING) + return true; + + dev = dev->bus->self; + } + + return false; +} + +static void pci_configure_relaxed_ordering(struct pci_dev *dev) +{ + /* We should not alter the relaxed ordering bit for the VF */ + if (dev->is_virtfn) + return; + + /* If the releaxed ordering enable bit is not set, do nothing. */ + if (!pcie_relaxed_ordering_supported(dev)) + return; + + if (pci_dev_should_disable_relaxed_ordering(dev)) { + pcie_clear_relaxed_ordering(dev); + dev_info(&dev->dev, "Disable Relaxed Ordering\n"); + } +} + static void pci_configure_device(struct pci_dev *dev) { struct hotplug_params hpp; @@ -1708,6 +1744,7 @@ static void pci_configure_device(struct pci_dev *dev) pci_configure_mps(dev); pci_configure_extended_tags(dev); + pci_configure_relaxed_ordering(dev); memset(&hpp, 0, sizeof(hpp)); ret = pci_get_hp_params(dev, &hpp); diff --git a/include/linux/pci.h b/include/linux/pci.h index ce77690..f5f200f 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1110,6 +1110,8 @@ int __pci_enable_wake(struct pci_dev *dev, pci_power_t state, void pci_pme_wakeup_bus(struct pci_bus *bus); void pci_d3cold_enable(struct pci_dev *dev); void pci_d3cold_disable(struct pci_dev *dev); +int pcie_clear_relaxed_ordering(struct pci_dev *dev); +bool pcie_relaxed_ordering_supported(struct pci_dev *dev); static inline int pci_enable_wake(struct pci_dev *dev, pci_power_t state, bool enable) From patchwork Mon Jun 19 06:53:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 105796 Delivered-To: patch@linaro.org Received: by 10.140.91.2 with SMTP id y2csp742336qgd; Sun, 18 Jun 2017 23:54:58 -0700 (PDT) X-Received: by 10.84.133.162 with SMTP id f31mr27376197plf.83.1497855297955; Sun, 18 Jun 2017 23:54:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497855297; cv=none; d=google.com; s=arc-20160816; b=m9MMiWNvlNKIs4f2aoirdl/Fr+b2z9LIarfpeZr5d/dQjzYJdBhZZtuKrBJ92dPU8g 5QYlSe2iKEsAE/BYsyu5wE+tGmBFTT6HfiXdKYiiulFZod+c2jiuWO2o9ILyhqJvmlu2 nU1oFGXRN+E4XjKVAbnBINmHV/vcuZeiHIaZQ6LUlYqu+VhMmv63Q1PQ3P76ZzFPZ7tz T3NglG1SK3/U7RkQrOZWIUK1mEo4UwfKMZkKFAkPjuGQb67FnxFGRRl/HbBNJWcecPyH A+CQ/xFLsyS41OWdcoLvDIIOq2dRDmCKwAOWyLwjB06xVuqiSJlGWtnreondn21gCwPr gPBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=2/2HerUEvBgf+ZYXtFh5RP8gku7NgITX0Af2Yi52V7c=; b=REKu/yk29OuGV5IDvApuuR9rnQoqL7Z2bc0YDLFO1fcPFlVF3DoBJaAiDCnkUGUSJF iM+eSu0KYl2kuUjadx0qAsyAdkq53XkGPwSEVIy83uYdlJKPvwWUtZW225InkvSpSVHG qVzsUByV+o3RgKMwSnCh5j6JKLEoVvfu+3/f55Fg2cNRuH4iXjl8T15kpnCqzjssXaRK mCjSNKfdFMxiBPxbT6V0UP0jxFbrKDaVHsU3vrxdr8VkZ5AyrJuY8v59F8Wvb8itBLRr p9zdBB/cuAn+YIcGRH6itXZ5M+26GwkkC0u9dH1qrYc7nMqhbb2tEqBPU+Ma7vTV+5ny JWJw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h17si7324908pgn.513.2017.06.18.23.54.57; Sun, 18 Jun 2017 23:54:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753537AbdFSGyi (ORCPT + 25 others); Mon, 19 Jun 2017 02:54:38 -0400 Received: from szxga03-in.huawei.com ([45.249.212.189]:7888 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753324AbdFSGyg (ORCPT ); Mon, 19 Jun 2017 02:54:36 -0400 Received: from 172.30.72.55 (EHLO DGGEML401-HUB.china.huawei.com) ([172.30.72.55]) by dggrg03-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id APP89457; Mon, 19 Jun 2017 14:54:20 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEML401-HUB.china.huawei.com (10.3.17.32) with Microsoft SMTP Server id 14.3.301.0; Mon, 19 Jun 2017 14:54:10 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v5 3/3] net/cxgb4: Use new PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag Date: Mon, 19 Jun 2017 14:53:59 +0800 Message-ID: <1497855239-18560-4-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1497855239-18560-1-git-send-email-dingtianhong@huawei.com> References: <1497855239-18560-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020206.5947751E.00B9, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: b02b4d4ac73303cc01e52fc555ed232a Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Casey Leedom cxgb4 Ethernet driver now queries PCIe configuration space to determine if it can send TLPs to it with the Relaxed Ordering Attribute set. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong --- drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 1 + drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 17 +++++++++++++++++ drivers/net/ethernet/chelsio/cxgb4/sge.c | 5 +++-- 3 files changed, 21 insertions(+), 2 deletions(-) -- 1.9.0 diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h index e88c180..478f25a 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h @@ -521,6 +521,7 @@ enum { /* adapter flags */ USING_SOFT_PARAMS = (1 << 6), MASTER_PF = (1 << 7), FW_OFLD_CONN = (1 << 9), + ROOT_NO_RELAXED_ORDERING = (1 << 10), }; enum { diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c index ea1bfcf..7cd4e1b 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c @@ -4735,6 +4735,23 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent) adapter->msg_enable = DFLT_MSG_ENABLE; memset(adapter->chan_map, 0xff, sizeof(adapter->chan_map)); + /* If possible, we use PCIe Relaxed Ordering Attribute to deliver + * Ingress Packet Data to Free List Buffers in order to allow for + * chipset performance optimizations between the Root Complex and + * Memory Controllers. (Messages to the associated Ingress Queue + * notifying new Packet Placement in the Free Lists Buffers will be + * send without the Relaxed Ordering Attribute thus guaranteeing that + * all preceding PCIe Transaction Layer Packets will be processed + * first.) But some Root Complexes have various issues with Upstream + * Transaction Layer Packets with the Relaxed Ordering Attribute set. + * The PCIe devices which under the Root Complexes will be cleared the + * Relaxed Ordering bit in the configuration space, So we check our + * PCIe configuration space to see if it's flagged with advice against + * using Relaxed Ordering. + */ + if (pcie_relaxed_ordering_supported(pdev)) + adapter->flags |= ROOT_NO_RELAXED_ORDERING; + spin_lock_init(&adapter->stats_lock); spin_lock_init(&adapter->tid_release_lock); spin_lock_init(&adapter->win0_lock); diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c index f05f0d4..ac229a3 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c @@ -2571,6 +2571,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, struct fw_iq_cmd c; struct sge *s = &adap->sge; struct port_info *pi = netdev_priv(dev); + int relaxed = !(adap->flags & ROOT_NO_RELAXED_ORDERING); /* Size needs to be multiple of 16, including status entry. */ iq->size = roundup(iq->size, 16); @@ -2624,8 +2625,8 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, flsz = fl->size / 8 + s->stat_len / sizeof(struct tx_desc); c.iqns_to_fl0congen |= htonl(FW_IQ_CMD_FL0PACKEN_F | - FW_IQ_CMD_FL0FETCHRO_F | - FW_IQ_CMD_FL0DATARO_F | + FW_IQ_CMD_FL0FETCHRO_V(relaxed) | + FW_IQ_CMD_FL0DATARO_V(relaxed) | FW_IQ_CMD_FL0PADEN_F); if (cong >= 0) c.iqns_to_fl0congen |=