From patchwork Mon Aug 14 15:44:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 110053 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp4505662qge; Mon, 14 Aug 2017 08:47:51 -0700 (PDT) X-Received: by 10.98.2.146 with SMTP id 140mr13096655pfc.207.1502725671237; Mon, 14 Aug 2017 08:47:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502725671; cv=none; d=google.com; s=arc-20160816; b=EIzeTb/z42Tjqr5HhhITU75Sy1Btpq05vHeU9Y2Xqm2qdfZIJyAF22y93kSVMznPrk UGhmS7Jz1CwrfIy01PvH8sGig/AHiTTXB6bdh15US3x6OM63pQAIQEFp7u8Ccm+LvIyy byFmBsZHuYJOdRb4ubJ+wXWDLtDe/5wWlmWdzPswdRtINeenQMcXZcZiLpxFPX7tEk5h nHqp9wp26eyxDR/rr2oHne/oOD0J8hg5r6YALV8nWWTFZfIWEouQt7C+R61TaF7Uss52 d1Y0AYjR1YWQRqS5y3jnIyVAM0HwyUdlDM8iymCt34cN0pfbRBU4xAvMC9EbXYMcmjec Tj5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=Tvw2aoHTNt/zpbHFmgbucgjJLnxgnvUS4rsp1LMKR84=; b=xhPeD6I0cIKGf/zZNtrCSGBOXyoA8DTR1lpfEUJxOsQrEAHMKeRT9yRfqtjWESnoFQ t5whnRYoluBMvxVKOQaVIe8YlXvB4AsKaUSSyn86W2+fHOVZatwlKrS/k9G2a7Hm80Ca mYVIXxdMCxGUAE+FZdJLG9JAmHYmDMwBUETkkIyH7NkUDshRzA0ChiI/ZPKVDaTeYWnV 2dfYNcFn03Zpq5HDFS552h4b5pBMVzVAxtP9yhJKeGm2ABY/Vm5AMHjO558paWBdY+vb pUh3Jw+UoNjWyFw2IDGPoyWnMsujt3xqAbl1PykxfT3lkIVRDYaPTAclGwnrzPA42Kkk 4mcA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g15si4791744pli.354.2017.08.14.08.47.50; Mon, 14 Aug 2017 08:47:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753277AbdHNPrs (ORCPT + 25 others); Mon, 14 Aug 2017 11:47:48 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:3080 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751441AbdHNPpb (ORCPT ); Mon, 14 Aug 2017 11:45:31 -0400 Received: from 172.30.72.60 (EHLO DGGEMS408-HUB.china.huawei.com) ([172.30.72.60]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DFF55872; Mon, 14 Aug 2017 23:45:16 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.301.0; Mon, 14 Aug 2017 23:45:06 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v10 3/5] PCI: Disable Relaxed Ordering Attributes for AMD A1100 Date: Mon, 14 Aug 2017 23:44:57 +0800 Message-ID: <1502725499-11276-4-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1502725499-11276-1-git-send-email-dingtianhong@huawei.com> References: <1502725499-11276-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A090206.5991C58C.008D, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 788d57bf3a183630211b9dd3dd73331c Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Casey reported that the AMD ARM A1100 SoC has a bug in its PCIe Root Port where Upstream Transaction Layer Packets with the Relaxed Ordering Attribute clear are allowed to bypass earlier TLPs with Relaxed Ordering set, it would cause Data Corruption, so we need to disable Relaxed Ordering Attribute when Upstream TLPs to the Root Port. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong Acked-by: Alexander Duyck Acked-by: Ashok Raj --- drivers/pci/quirks.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) -- 1.8.3.1 diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 1272f7e..1407604 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -4089,6 +4089,22 @@ static void quirk_relaxedordering_disable(struct pci_dev *dev) quirk_relaxedordering_disable); /* + * The AMD ARM A1100 (AKA "SEATTLE") SoC has a bug in its PCIe Root Complex + * where Upstream Transaction Layer Packets with the Relaxed Ordering + * Attribute clear are allowed to bypass earlier TLPs with Relaxed Ordering + * set. This is a violation of the PCIe 3.0 Transaction Ordering Rules + * outlined in Section 2.4.1 (PCI Express(r) Base Specification Revision 3.0 + * November 10, 2010). As a result, on this platform we can't use Relaxed + * Ordering for Upstream TLPs. + */ +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a00, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a01, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); +DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMD, 0x1a02, PCI_CLASS_NOT_DEFINED, 8, + quirk_relaxedordering_disable); + +/* * Per PCIe r3.0, sec 2.2.9, "Completion headers must supply the same * values for the Attribute as were supplied in the header of the * corresponding Request, except as explicitly allowed when IDO is used." From patchwork Mon Aug 14 15:44:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ding Tianhong X-Patchwork-Id: 110051 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp4504231qge; Mon, 14 Aug 2017 08:46:35 -0700 (PDT) X-Received: by 10.98.59.90 with SMTP id i87mr25896107pfa.300.1502725595446; Mon, 14 Aug 2017 08:46:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502725595; cv=none; d=google.com; s=arc-20160816; b=aYKtDOibu6KQa+zAj1+XUPkDmESqAa4nDFceD0RN2p7iKJxnOuYGAWtbu1XYMpAjQg KUNh7HFR+Pfo64TkPNPdRExKHI4PwW7cnkiYblZyEE8EOdGRV979Uhv3sV21Gtd2EkZh o1tqEIfc7JPRdntZ1glEjBiwVG85esY+9kdNrbLT+yOv4VgGN2FatSFapaMUnEU8cf2P BkLObOWPfcg/avERFg7K1ZarPxtePkOrv634PqqBqTDsZuBRAdSo1Ht0tHA8nWb0fsk/ ZtZjAs2IFWxG8jgdr6zMGOTZbAIXI72ij/qgqFzzY6apMZ3bUslE4jPpfa9F9/cDxnPx Aspg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=haLrfbOPuKqwKLbRy8tbXYaB/7neOUtBphv6Vw6sYA0=; b=YevCXQKzHSdyxPrAL+4s3kzwv23bmgJ6nODZF7yZelXqsgWMeGwn7y25Xy6c9m1e/R xTrdojg3uA4V0JIe861E9fxkAJ1JFrxqCKt8pd4aB/Df7I0TAW81WN17EOO9PLcwKQPS qMQDAGqS87NODf6lN4G+By48ARZJ/429sZMiKCToSK+efPIbdYarP/fsnNuAs+9lItci uklYW9A4j1s7WATjwFYun4do6e4JJqB4K5T+GpJwl3hgc6rkUp/OJj+HoqPhoYquJ/0A 8NZXBJiqyo7VuxLzMvMM7yercPXUZ1iARWU08jiyutj7MbU7o8JXlFrX4AnsRSF+01Sr b0RA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s6si4806499plj.368.2017.08.14.08.46.35; Mon, 14 Aug 2017 08:46:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753120AbdHNPqc (ORCPT + 25 others); Mon, 14 Aug 2017 11:46:32 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:3078 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752120AbdHNPph (ORCPT ); Mon, 14 Aug 2017 11:45:37 -0400 Received: from 172.30.72.58 (EHLO DGGEMS403-HUB.china.huawei.com) ([172.30.72.58]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DFF55869; Mon, 14 Aug 2017 23:45:15 +0800 (CST) Received: from localhost (10.177.23.32) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.301.0; Mon, 14 Aug 2017 23:45:08 +0800 From: Ding Tianhong To: , , , , , , , , , , , , , , , , , , , , , , , , , CC: Ding Tianhong Subject: [PATCH v10 4/5] net/cxgb4: Use new PCI_DEV_FLAGS_NO_RELAXED_ORDERING flag Date: Mon, 14 Aug 2017 23:44:58 +0800 Message-ID: <1502725499-11276-5-git-send-email-dingtianhong@huawei.com> X-Mailer: git-send-email 1.8.5.2.msysgit.0 In-Reply-To: <1502725499-11276-1-git-send-email-dingtianhong@huawei.com> References: <1502725499-11276-1-git-send-email-dingtianhong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.32] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A090206.5991C58C.001E, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 5b69f67882abe463515e2021b303acf7 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Casey Leedom cxgb4 Ethernet driver now queries PCIe configuration space to determine if it can send TLPs to it with the Relaxed Ordering Attribute set. Remove the enable_pcie_relaxed_ordering() to avoid enable PCIe Capability Device Control[Relaxed Ordering Enable] at probe routine, to make sure the driver will not send the Relaxed Ordering TLPs to the Root Complex which could not deal the Relaxed Ordering TLPs. Signed-off-by: Casey Leedom Signed-off-by: Ding Tianhong Reviewed-by: Casey Leedom --- drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 1 + drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 23 +++++++++++++++++------ drivers/net/ethernet/chelsio/cxgb4/sge.c | 5 +++-- 3 files changed, 21 insertions(+), 8 deletions(-) -- 1.8.3.1 diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h index ef4be78..09ea62e 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h @@ -529,6 +529,7 @@ enum { /* adapter flags */ USING_SOFT_PARAMS = (1 << 6), MASTER_PF = (1 << 7), FW_OFLD_CONN = (1 << 9), + ROOT_NO_RELAXED_ORDERING = (1 << 10), }; enum { diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c index e403fa1..33bb867 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c @@ -4654,11 +4654,6 @@ static void print_port_info(const struct net_device *dev) dev->name, adap->params.vpd.id, adap->name, buf); } -static void enable_pcie_relaxed_ordering(struct pci_dev *dev) -{ - pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_RELAX_EN); -} - /* * Free the following resources: * - memory used for tables @@ -4908,7 +4903,6 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent) } pci_enable_pcie_error_reporting(pdev); - enable_pcie_relaxed_ordering(pdev); pci_set_master(pdev); pci_save_state(pdev); @@ -4947,6 +4941,23 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent) adapter->msg_enable = DFLT_MSG_ENABLE; memset(adapter->chan_map, 0xff, sizeof(adapter->chan_map)); + /* If possible, we use PCIe Relaxed Ordering Attribute to deliver + * Ingress Packet Data to Free List Buffers in order to allow for + * chipset performance optimizations between the Root Complex and + * Memory Controllers. (Messages to the associated Ingress Queue + * notifying new Packet Placement in the Free Lists Buffers will be + * send without the Relaxed Ordering Attribute thus guaranteeing that + * all preceding PCIe Transaction Layer Packets will be processed + * first.) But some Root Complexes have various issues with Upstream + * Transaction Layer Packets with the Relaxed Ordering Attribute set. + * The PCIe devices which under the Root Complexes will be cleared the + * Relaxed Ordering bit in the configuration space, So we check our + * PCIe configuration space to see if it's flagged with advice against + * using Relaxed Ordering. + */ + if (!pcie_relaxed_ordering_enabled(pdev)) + adapter->flags |= ROOT_NO_RELAXED_ORDERING; + spin_lock_init(&adapter->stats_lock); spin_lock_init(&adapter->tid_release_lock); spin_lock_init(&adapter->win0_lock); diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c index ede1220..4ef68f6 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/sge.c +++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c @@ -2719,6 +2719,7 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, struct fw_iq_cmd c; struct sge *s = &adap->sge; struct port_info *pi = netdev_priv(dev); + int relaxed = !(adap->flags & ROOT_NO_RELAXED_ORDERING); /* Size needs to be multiple of 16, including status entry. */ iq->size = roundup(iq->size, 16); @@ -2772,8 +2773,8 @@ int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq, flsz = fl->size / 8 + s->stat_len / sizeof(struct tx_desc); c.iqns_to_fl0congen |= htonl(FW_IQ_CMD_FL0PACKEN_F | - FW_IQ_CMD_FL0FETCHRO_F | - FW_IQ_CMD_FL0DATARO_F | + FW_IQ_CMD_FL0FETCHRO_V(relaxed) | + FW_IQ_CMD_FL0DATARO_V(relaxed) | FW_IQ_CMD_FL0PADEN_F); if (cong >= 0) c.iqns_to_fl0congen |=