From patchwork Mon Jun 12 11:05:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 103612 Delivered-To: patch@linaro.org Received: by 10.182.29.35 with SMTP id g3csp832198obh; Mon, 12 Jun 2017 04:07:03 -0700 (PDT) X-Received: by 10.98.41.130 with SMTP id p124mr51606292pfp.220.1497265623690; Mon, 12 Jun 2017 04:07:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497265623; cv=none; d=google.com; s=arc-20160816; b=r8VkRDFzOyWIW/UIdoND+6S2zEJloZ4NwMS6uiiJf3cPzTAZNimnH5wdlkT4vTZsVf wiTrLXa4G1oLQZQG5WSYCIbNITeiJcazprJWj/VOAHI8RiKxptzvkMYQOcI3G8ZXvO1l 4D+77BmGLOz0DD9ZU+4j5LbqutV8Vvr0u71H1upugN6ivUdcuuQQYB74j9N19HQEc7sf qN48NrTeSwCco19APqtew1ymJmOZs2+UgOJ/W/D9lhFKbTClpRTi7x05txz+pZdRBRBe /bDLWX1On04LO8WjblLLrD7dm4zXTon8OzKfm7r10ExzdW1G0XceGz3Czk2xerCmCgMv kGNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=44Bg9U8BvSrRj6nBK+KhCZq/21LLygqY4TTP9eCfKbk=; b=J2XeNFbYI5CmN8dyqVx82Snn3RbYggL4Ju2ypdov47MsftywbxlBXZRDJXUodBkdVz nUydKcuNNroEtEPrma9C5kmJDNSXm8gtpVHwC5wPfXi5+qd0Xzto7CB8j/LV4rpqlJso SvMo46atVwuMpLWo+Fepb0Ue89+RhYw8NhZJl5pVDq9t/mk4iVx4S0DcRrQyFc0hZg3Z k6eiFxWsJMbfhmrXFc4lAGQd3l6flvLdZt+3Ujfd41S6b2UZPMFWMYkNaOEYaanVXdIb 6ywkPpFulbHNAgQhDLrDR/a+2azHKbJjG0dsRtyrrBmES81xcnIJNEkGvq7OndqsNY2W UE4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k9si6786842pfk.323.2017.06.12.04.07.03; Mon, 12 Jun 2017 04:07:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752711AbdFLLGo (ORCPT + 25 others); Mon, 12 Jun 2017 07:06:44 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:7374 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752683AbdFLLGm (ORCPT ); Mon, 12 Jun 2017 07:06:42 -0400 Received: from 172.30.72.55 (EHLO dggeml406-hub.china.huawei.com) ([172.30.72.55]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id APF17482; Mon, 12 Jun 2017 19:06:08 +0800 (CST) Received: from localhost (10.177.23.32) by dggeml406-hub.china.huawei.com (10.3.17.50) with Microsoft SMTP Server id 14.3.301.0; Mon, 12 Jun 2017 19:05:58 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v4 1/3] PCI: Add new PCIe Fabric End Node flag, PCI_DEV_FLAGS_NO_RELAXED_ORDERING Date: Mon, 12 Jun 2017 19:05:23 +0800 Message-ID: <1497265525-4752-2-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1497265525-4752-1-git-send-email-dingtianhong@huawei.com> References: <1497265525-4752-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020204.593E75A3.006B, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 91a2672d5572691b85ea7cae3fecd1d1 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Casey Leedom The new flag PCI_DEV_FLAGS_NO_RELAXED_ORDERING indicates that the Relaxed Ordering Attribute should not be used on Transaction Layer Packets destined for the PCIe End Node so flagged. Initially flagged this way are Intel E5-26xx Root Complex Ports which suffer from a Flow Control Credit Performance Problem and AMD A1100 ARM ("SEATTLE") Root Complex Ports which don't obey PCIe 3.0 ordering rules which can lead to Data Corruption. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong --- drivers/pci/quirks.c | 38 ++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 2 files changed, 40 insertions(+) -- 1.9.0 diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 085fb78..58bdd23 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -3999,6 +3999,44 @@ static void quirk_tw686x_class(struct pci_dev *pdev) quirk_tw686x_class); /* + * Some devices have problems with Transaction Layer Packets with the Relaxed + * Ordering Attribute set. Such devices should mark themselves and other + * Device Drivers should check before sending TLPs with RO set. + */ +static void quirk_relaxedordering_disable(struct pci_dev *dev) +{ + dev->dev_flags |= PCI_DEV_FLAGS_NO_RELAXED_ORDERING; +} + +/* + * Intel E5-26xx Root Complex has a Flow Control Credit issue which can + * cause performance problems with Upstream Transaction Layer Packets with + * Relaxed Ordering set. + */ +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f02, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f04, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_INTEL, 0x6f08, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); + +/* + * The AMD ARM A1100 (AKA "SEATTLE") SoC has a bug in its PCIe Root Complex + * where Upstream Transaction Layer Packets with the Relaxed Ordering + * Attribute clear are allowed to bypass earlier TLPs with Relaxed Ordering + * set. This is a violation of the PCIe 3.0 Transaction Ordering Rules + * outlined in Section 2.4.1 (PCI Express(r) Base Specification Revision 3.0 + * November 10, 2010). As a result, on this platform we can't use Relaxed + * Ordering for Upstream TLPs. + */ +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a00, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a01, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a02, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); + +/* * Per PCIe r3.0, sec 2.2.9, "Completion headers must supply the same * values for the Attribute as were supplied in the header of the * corresponding Request, except as explicitly allowed when IDO is used." diff --git a/include/linux/pci.h b/include/linux/pci.h index 33c2b0b..e1e8428 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -183,6 +183,8 @@ enum pci_dev_flags { PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT = (__force pci_dev_flags_t) (1 << 9), /* Do not use FLR even if device advertises PCI_AF_CAP */ PCI_DEV_FLAGS_NO_FLR_RESET = (__force pci_dev_flags_t) (1 << 10), + /* Don't use Relaxed Ordering for TLPs directed at this device */ + PCI_DEV_FLAGS_NO_RELAXED_ORDERING = (__force pci_dev_flags_t) (1 << 11), }; enum pci_irq_reroute_variant { From patchwork Mon Jun 12 11:05:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 103611 Delivered-To: patch@linaro.org Received: by 10.182.29.35 with SMTP id g3csp832193obh; Mon, 12 Jun 2017 04:07:03 -0700 (PDT) X-Received: by 10.84.131.1 with SMTP id 1mr55463269pld.232.1497265623320; Mon, 12 Jun 2017 04:07:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497265623; cv=none; d=google.com; s=arc-20160816; b=IbBAHKMdJdDqyhsvWvJwRpX2CTCxyhlTKaFHPnF1ye2McP4JHb+MjwXATLqbqAMLY+ 9rGY4MYxc66pMJOCALtTRsHYTRnnH2vjNcLUD9yDI74ugM4mWTChQAzNlo9STaLLSTew 6muWtEIzM8IHj0fbPWm3MXS2GceeGSxTuxJR/LdqyZkZSFdizBH1k6QL1NUc37IRcCVv RLe9xjxPLiI4zSatrZysVwUlhgmrdR0Nx0do1Xm0e3JdZOorYoqZIYaC6E+M+d4Okswm DiBvOi5BUW2wcDHxNu118bYdvAcMbzckz1laWKUmzPKjLbB+9Ws9JBlCuqByn7O2cLgz ReiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=a2Hc7rIW58uHkBmyX/Y88myOhUSzvS/tgGNRybPXHfE=; b=WpaJiSqknnUReJWmNQaq3q+g9Ag/x26zW/ulTNTb1c0KQkgxk4dm4f83g00q3Gd8Cm gunOVsYdaoykCRqMHb7A+KpI8shGOVY03yX65mksPwBh3zSIFFnDsSXlbL8+lkqVKL3G tqZebu9jVwYSeEvWwWnOUKCfdkopzyiZ2cxTRgxE+uXAWBiUUy/tE10bkFT1lAciLpkb sULjsCEWbcSsqiJdFxMZZQgTIywFyH0/ezzcSnqlMxIpXwv8e6CEJOXcTIFjRbm1hRV3 iGzdI3S7hOUX1pj9w0O098Dbp/i6VCqFY/UzEFqolb7KvU0zAlyD36XMkzvO8chnzTyq pljA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k9si6786842pfk.323.2017.06.12.04.07.02; Mon, 12 Jun 2017 04:07:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752687AbdFLLGf (ORCPT + 25 others); Mon, 12 Jun 2017 07:06:35 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:8262 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752123AbdFLLGd (ORCPT ); Mon, 12 Jun 2017 07:06:33 -0400 Received: from 172.30.72.57 (EHLO dggeml406-hub.china.huawei.com) ([172.30.72.57]) by dggrg01-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AQE17775; Mon, 12 Jun 2017 19:06:10 +0800 (CST) Received: from localhost (10.177.23.32) by dggeml406-hub.china.huawei.com (10.3.17.50) with Microsoft SMTP Server id 14.3.301.0; Mon, 12 Jun 2017 19:06:00 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v4 2/3] PCI: Enable PCIe Relaxed Ordering if supported Date: Mon, 12 Jun 2017 19:05:24 +0800 Message-ID: <1497265525-4752-3-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1497265525-4752-1-git-send-email-dingtianhong@huawei.com> References: <1497265525-4752-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020201.593E75A4.012A, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: c5f040871a2fa1d8528684eb2ffb7102 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The PCIe Device Control Register use the bit 4 to indicate that whether the device is permitted to enable relaxed ordering or not. But relaxed ordering is not safe for some platform which could only use strong write ordering, so devices are allowed (but not required) to enable relaxed ordering bit by default. If a PCIe device didn't enable the relaxed ordering attribute default, we should not do anything in the PCIe configuration, otherwise we should check if any of the devices above us do not support relaxed ordering by the PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag, then base on the result if we get a return that indicate that the relaxed ordering is not supported we should update our device to disable relaxed ordering in configuration space. If the device above us doesn't exist or isn't the PCIe device, we shouldn't do anything and skip updating relaxed ordering because we are probably running in a guest machine. Signed-off-by: Ding Tianhong --- drivers/pci/pci.c | 32 ++++++++++++++++++++++++++++++++ drivers/pci/probe.c | 41 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 3 files changed, 75 insertions(+) -- 1.9.0 diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index b01bd5b..b44f34c 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4878,6 +4878,38 @@ int pcie_set_mps(struct pci_dev *dev, int mps) EXPORT_SYMBOL(pcie_set_mps); /** + * pcie_clear_relaxed_ordering - clear PCI Express relaxed ordering bit + * @dev: PCI device to query + * + * If possible clear relaxed ordering + */ +int pcie_clear_relaxed_ordering(struct pci_dev *dev) +{ + return pcie_capability_clear_word(dev, PCI_EXP_DEVCTL, + PCI_EXP_DEVCTL_RELAX_EN); +} +EXPORT_SYMBOL(pcie_clear_relaxed_ordering); + +/** + * pcie_relaxed_ordering_supported - Probe for PCIe relexed ordering support + * @dev: PCI device to query + * + * Returns true if the device support relaxed ordering attribute. + */ +bool pcie_relaxed_ordering_supported(struct pci_dev *dev) +{ + bool ro_supported = false; + u16 v; + + pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &v); + if ((v & PCI_EXP_DEVCTL_RELAX_EN) >> 4) + ro_supported = true; + + return ro_supported; +} +EXPORT_SYMBOL(pcie_relaxed_ordering_supported); + +/** * pcie_get_minimum_link - determine minimum link settings of a PCI device * @dev: PCI device to query * @speed: storage for minimum speed diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 19c8950..ed1f717 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1701,6 +1701,46 @@ static void pci_configure_extended_tags(struct pci_dev *dev) PCI_EXP_DEVCTL_EXT_TAG); } +/** + * pci_dev_should_disable_relaxed_ordering - check if the PCI device + * should disable the relaxed ordering attribute. + * @dev: PCI device + * + * Return true if any of the PCI devices above us do not support + * relaxed ordering. + */ +static bool pci_dev_should_disable_relaxed_ordering(struct pci_dev *dev) +{ + bool ro_disabled = false; + + while (dev) { + if (dev->dev_flags & PCI_DEV_FLAGS_NO_RELAXED_ORDERING) { + ro_disabled = true; + break; + } + dev = dev->bus->self; + } + + return ro_disabled; +} + +static void pci_configure_relaxed_ordering(struct pci_dev *dev) +{ + struct pci_dev *bridge = pci_upstream_bridge(dev); + + if (!pci_is_pcie(dev) || !bridge || !pci_is_pcie(bridge)) + return; + + /* If the releaxed ordering enable bit is not set, do nothing. */ + if (!pcie_relaxed_ordering_supported(dev)) + return; + + if (pci_dev_should_disable_relaxed_ordering(dev)) { + pcie_clear_relaxed_ordering(dev); + dev_info(&dev->dev, "Disable Relaxed Ordering\n"); + } +} + static void pci_configure_device(struct pci_dev *dev) { struct hotplug_params hpp; @@ -1708,6 +1748,7 @@ static void pci_configure_device(struct pci_dev *dev) pci_configure_mps(dev); pci_configure_extended_tags(dev); + pci_configure_relaxed_ordering(dev); memset(&hpp, 0, sizeof(hpp)); ret = pci_get_hp_params(dev, &hpp); diff --git a/include/linux/pci.h b/include/linux/pci.h index e1e8428..9870781 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1105,6 +1105,8 @@ int __pci_enable_wake(struct pci_dev *dev, pci_power_t state, void pci_pme_wakeup_bus(struct pci_bus *bus); void pci_d3cold_enable(struct pci_dev *dev); void pci_d3cold_disable(struct pci_dev *dev); +int pcie_clear_relaxed_ordering(struct pci_dev *dev); +bool pcie_relaxed_ordering_supported(struct pci_dev *dev); static inline int pci_enable_wake(struct pci_dev *dev, pci_power_t state, bool enable) From patchwork Mon Jun 12 11:05:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 103613 Delivered-To: patch@linaro.org Received: by 10.182.29.35 with SMTP id g3csp832343obh; Mon, 12 Jun 2017 04:07:25 -0700 (PDT) X-Received: by 10.84.143.162 with SMTP id 31mr51063597plz.277.1497265645316; Mon, 12 Jun 2017 04:07:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497265645; cv=none; d=google.com; s=arc-20160816; b=G3aQTwTOUbkLEIch0puZ+R7Vgz/eI5TMuZLA+Vcl6NXtlkysS0A3RU/N1WQB8sERZf eo9Q3+gmfmpT8++mfs5ldyzqCKiPVrqnWD17KxfNSAO5VGU3IT8NVdtyKff1czIDX7XE iXuboSjn8XiLDfpqIxEvXX6D9QTN/Xalr3vFXxLAHvQQvBTj7tpCR0LjfgM/AwZeEYtz k4ZdgMtU2vXrIxVF9TeTZAqJ5wYzRqxq8YcdMfaNHxhSty2V3aRhBqwWz5EV2mvpIR/1 AfOAvtEpR74R8lGFj4RAVGc8AfMtIHAs6gqmF2LOFi/dyoZ1kNkcy4M88x/nM4ytf+YB b+DA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=XLz0beNQ9yngrBwM9Jf2SfzVRXEqZT9dzpc7Azk5JIY=; b=uFERF0n/K+NZhjBU3L4b4EWufr02aOBkkLoTAsZGkBXaQDl53Ae66ybnqrzrVPvZgH UWo/X5Zb/S7appW2phOQ4GYhI5KAs/gwg5k4gSmSd6472R/t2QjNqQiWRdDfMpn9g7aQ 9BK9Re4nq3BymtKtzus4eM/8QAfgkZ6yMMHz77lyofdn6FwaLtTL1BJ/hx1EveXnpOEW Ji9YuPbCrP728llyPs17fmSJsRG2mPPdXoGWGfllO2D68Zrutp0stkvQrLjJwNx4DIty X135+LQNIQIWyT6jyqVzzhhKgYFOE2io+cFiz83W7Jvw1nBxleQXHpHy5Fv4lcLq7knK h6sQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v2si12921232pgn.193.2017.06.12.04.07.25; Mon, 12 Jun 2017 04:07:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752734AbdFLLHF (ORCPT + 25 others); Mon, 12 Jun 2017 07:07:05 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:8263 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752688AbdFLLGm (ORCPT ); Mon, 12 Jun 2017 07:06:42 -0400 Received: from 172.30.72.57 (EHLO dggeml406-hub.china.huawei.com) ([172.30.72.57]) by dggrg01-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AQE17777; Mon, 12 Jun 2017 19:06:11 +0800 (CST) Received: from localhost (10.177.23.32) by dggeml406-hub.china.huawei.com (10.3.17.50) with Microsoft SMTP Server id 14.3.301.0; Mon, 12 Jun 2017 19:06:01 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v4 3/3] net/cxgb4: Use new PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag Date: Mon, 12 Jun 2017 19:05:25 +0800 Message-ID: <1497265525-4752-4-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1497265525-4752-1-git-send-email-dingtianhong@huawei.com> References: <1497265525-4752-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020201.593E75A4.0120, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 8b6d0fc8b52ce49dedbfdf9cbeed99dd Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Casey Leedom cxgb4 Ethernet driver now queries PCIe configuration space to determine if it can send TLPs to it with the Relaxed Ordering Attribute set. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong --- drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 1 + drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 17 +++++++++++++++++ drivers/net/ethernet/chelsio/cxgb4/sge.c | 5 +++-- 3 files changed, 21 insertions(+), 2 deletions(-) -- 1.9.0 diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h index e88c180..478f25a 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h @@ -521,6 +521,7 @@ enum { /* adapter flags */ USING_SOFT_PARAMS = (1 << 6), MASTER_PF = (1 << 7), FW_OFLD_CONN = (1 << 9), + ROOT_NO_RELAXED_ORDERING = (1 << 10), }; enum { diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c index 38a5c67..1dd093d 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c @@ -4726,6 +4726,23 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent) adapter->msg_enable = DFLT_MSG_ENABLE; memset(adapter->chan_map, 0xff, sizeof(adapter->chan_map)); + /* If possible, we use PCIe Relaxed Ordering Attribute to deliver + * Ingress Packet Data to Free List Buffers in order to allow for + * chipset performance optimizations between the Root Complex and + * Memory Controllers. (Messages to the associated Ingress Queue + * notifying new Packet Placement in the Free Lists Buffers will be + * send without the Relaxed Ordering Attribute thus guaranteeing that + * all preceding PCIe Transaction Layer Packets will be processed + * first.) But some Root Complexes have various issues with Upstream + * Transaction Layer Packets with the Relaxed Ordering Attribute set. + * The PCIe devices which under the Root Complexes will be cleared the + * Relaxed Ordering bit in the configuration space, So we check our + * PCIe configuration space to see if it's flagged with advice against + * using Relaxed Ordering. + */ + if (pcie_relaxed_ordering_supported(pdev)) + adapter->flags |= ROOT_NO_RELAXED_ORDERING; + spin_lock_init(&adapter->stats_lock); spin_lock_init(&adapter->tid_release_lock); spin_lock_init(&adapter->win0_lock); diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c index f05f0d4..ac229a3 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c @@ -2571,6 +2571,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, struct fw_iq_cmd c; struct sge *s = &adap->sge; struct port_info *pi = netdev_priv(dev); + int relaxed = !(adap->flags & ROOT_NO_RELAXED_ORDERING); /* Size needs to be multiple of 16, including status entry. */ iq->size = roundup(iq->size, 16); @@ -2624,8 +2625,8 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, flsz = fl->size / 8 + s->stat_len / sizeof(struct tx_desc); c.iqns_to_fl0congen |= htonl(FW_IQ_CMD_FL0PACKEN_F | - FW_IQ_CMD_FL0FETCHRO_F | - FW_IQ_CMD_FL0DATARO_F | + FW_IQ_CMD_FL0FETCHRO_V(relaxed) | + FW_IQ_CMD_FL0DATARO_V(relaxed) | FW_IQ_CMD_FL0PADEN_F); if (cong >= 0) c.iqns_to_fl0congen |=