From patchwork Mon May 14 22:55:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 135798 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp218427lji; Mon, 14 May 2018 15:55:56 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqs4Rv1g05Chqfs5CT2CMk93B+KVYAohcp3PZwsHWfYg3rsGZPO+yGMYNc4zuDC7KjW2Uxd X-Received: by 2002:a17:902:6b47:: with SMTP id g7-v6mr11688575plt.251.1526338556647; Mon, 14 May 2018 15:55:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526338556; cv=none; d=google.com; s=arc-20160816; b=ifKZo/s8vfSVJHDA4h7uGtp+kPolAhvz2iwj1lTQ8aN08ncbjo2N5w4jgwyxUh1Ph/ O0JJQ3mC1PTiDnYTWRwUCGiqST9nO4lj2s4b0SFWUYH+wO+nIzs7AKYOGlc34xVj1vgu OypJ77ZzWeYwkl09axMhw3sXk5W/nbd7867vC/I0emFTkLgiIqZEnfqFY326M2Kt3Ces kDWjSiuUcasWv9vZwpur8g1w1EAFsmP4JUEoMGkFjQLIqaAf3uflHwrE1vQnMPa0mpIY FDZSDefFRpVxSm3DSwQahzDGkLjLd3baRcHCYpvdU+Hmm5Kpfhpkd5sl/dadVxe1FFUK M0cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:references:in-reply-to:date :subject:cc:to:from:arc-authentication-results; bh=yZ9uEpo9g5Wn34DaT5mYbGJpLcyLB44WNESTvZM+Pck=; b=Le/JYNydu+ZZmZK6WTRsl4Z4PmwdmPqIcxYc7Qr4888kl7uz6UqPtociQ/mTM9aDzz 3Mfe/7h6H/9O8D2KMEo6lZn12/2vWE7kXlBn3RmyLY5PEv4SaFmPBNyZQs4fOIg9VsfX vfLy7lh0aDDFrpgposiTqKCsJ8vr916iPaVDvOlpHGvpn4jksnv4eRmriO0GAGrmwi0t rPqHNJ/Wbf7pvrol5clOwoQfuXVPYs5CDzkSPP2S1pz4pcYgIwAdnTLtVFyd3xkc6LOI Xy3WDdGt9SKS6inrmdAijtgjGOCZoFnkhD5mQyzX8SnESeSU4ynSCITu4Aqyj5FQLDP7 t/2w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p11-v6si8164944pgn.267.2018.05.14.15.55.56; Mon, 14 May 2018 15:55:56 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752439AbeENWzy (ORCPT + 29 others); Mon, 14 May 2018 18:55:54 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:35804 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752249AbeENWyK (ORCPT ); Mon, 14 May 2018 18:54:10 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4EMs3ci111190 for ; Mon, 14 May 2018 18:54:10 -0400 Received: from e12.ny.us.ibm.com (e12.ny.us.ibm.com [129.33.205.202]) by mx0b-001b2d01.pphosted.com with ESMTP id 2hyfnsrh6y-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 14 May 2018 18:54:09 -0400 Received: from localhost by e12.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 14 May 2018 18:54:08 -0400 Received: from b01cxnp23032.gho.pok.ibm.com (9.57.198.27) by e12.ny.us.ibm.com (146.89.104.199) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 14 May 2018 18:54:04 -0400 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4EMs3j047972554; Mon, 14 May 2018 22:54:03 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 95490B204D; Mon, 14 May 2018 19:56:01 -0400 (EDT) Received: from paulmck-ThinkPad-W541 (unknown [9.70.82.108]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP id 5DB9BB2052; Mon, 14 May 2018 19:56:01 -0400 (EDT) Received: by paulmck-ThinkPad-W541 (Postfix, from userid 1000) id 1FDC916C379C; Mon, 14 May 2018 15:55:35 -0700 (PDT) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, mingo@kernel.org Cc: stern@rowland.harvard.edu, parri.andrea@gmail.com, will.deacon@arm.com, peterz@infradead.org, boqun.feng@gmail.com, npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr, akiyks@gmail.com, Benjamin Herrenschmidt , Arnd Bergmann , Jason Gunthorpe , "Paul E. McKenney" , Ingo Molnar , Jonathan Corbet Subject: [PATCH memory-model 1/8] docs/memory-barriers.txt: Fix broken DMA vs MMIO ordering example Date: Mon, 14 May 2018 15:55:26 -0700 X-Mailer: git-send-email 2.5.2 In-Reply-To: <20180514225457.GA5142@linux.vnet.ibm.com> References: <20180514225457.GA5142@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18051422-0048-0000-0000-0000026CF06E X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009026; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000260; SDB=6.01032370; UDB=6.00527776; IPR=6.00811502; MB=3.00021114; MTD=3.00000008; XFM=3.00000015; UTC=2018-05-14 22:54:08 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18051422-0049-0000-0000-0000451F32FE Message-Id: <1526338533-6044-1-git-send-email-paulmck@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-14_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805140226 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon The section of memory-barriers.txt that describes the dma_Xmb() barriers has an incorrect example claiming that a wmb() is required after writing to coherent memory in order for those writes to be visible to a device before a subsequent MMIO access using writel() can reach the device. In fact, this ordering guarantee is provided (at significant cost on some architectures such as arm and power) by writel, so the wmb() is not necessary. writel_relaxed exists for cases where this ordering is not required. Fix the example and update the text to make this clearer. Cc: Benjamin Herrenschmidt Cc: Arnd Bergmann Cc: Jason Gunthorpe Cc: "Paul E. McKenney" Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Jonathan Corbet Reported-by: Sinan Kaya Signed-off-by: Will Deacon Signed-off-by: Paul E. McKenney --- Documentation/memory-barriers.txt | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) -- 2.5.2 diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 6dafc8085acc..34c1970908a5 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -1920,9 +1920,6 @@ There are some more advanced barrier functions: /* assign ownership */ desc->status = DEVICE_OWN; - /* force memory to sync before notifying device via MMIO */ - wmb(); - /* notify device of new descriptors */ writel(DESC_NOTIFY, doorbell); } @@ -1930,11 +1927,15 @@ There are some more advanced barrier functions: The dma_rmb() allows us guarantee the device has released ownership before we read the data from the descriptor, and the dma_wmb() allows us to guarantee the data is written to the descriptor before the device - can see it now has ownership. The wmb() is needed to guarantee that the - cache coherent memory writes have completed before attempting a write to - the cache incoherent MMIO region. - - See Documentation/DMA-API.txt for more information on consistent memory. + can see it now has ownership. Note that, when using writel(), a prior + wmb() is not needed to guarantee that the cache coherent memory writes + have completed before writing to the MMIO region. The cheaper + writel_relaxed() does not provide this guarantee and must not be used + here. + + See the subsection "Kernel I/O barrier effects" for more information on + relaxed I/O accessors and the Documentation/DMA-API.txt file for more + information on consistent memory. MMIO WRITE BARRIER