From patchwork Mon Apr 16 16:12:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 133478 Delivered-To: patch@linaro.org Received: by 10.46.84.18 with SMTP id i18csp3743503ljb; Mon, 16 Apr 2018 09:16:43 -0700 (PDT) X-Google-Smtp-Source: AIpwx495isSEY+kDTLdnZSd1UQLe1M0mowGBV3ch8SxiIVR2UPw7zn/LPV1mijFmN0xYzxqg0P1S X-Received: by 10.99.1.10 with SMTP id 10mr8140983pgb.444.1523895403074; Mon, 16 Apr 2018 09:16:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523895403; cv=none; d=google.com; s=arc-20160816; b=TRVo/L/CCc5QJ7QUpaIS3QOB8YCG1OH1XTcGiNqMkqMZVT593X9wrx5yjGTLVJmdnJ 7yHR1HRrGQ2UXw2o8hVe4A4p69aNIb6JcpgLfkcrz/ZISSVLM885GtMw2SOLheqddKnb m+cbYyXBAYVYZtU1zJzIrNA0TzlISlHKK5JiqveZof+o4j6MMm08AQlhUUiGxQXNgzTh crw7kMlu2dCjhFWGk+6oAMcUNOoRM6PnXb4d/zRLRQ10ViX5qXnCrlCeMhawL3baYrQd CDX2V8DOJImogqc/Y8n7WlUz+49XEThifhqKwNdYGR26RLf2NJVG4jh9qfSrBgAzeZTx y92g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:references:in-reply-to:date :subject:cc:to:from:arc-authentication-results; bh=yZ9uEpo9g5Wn34DaT5mYbGJpLcyLB44WNESTvZM+Pck=; b=rCS4ITnAx/yi+nmznxgr+lQUVGv+NWD8vk61X+hGgMrUtpyuEyvq9TIOstYZ30twPE EjJC9cPVAmpWRICoeHZY1Lv0ZG6AOOiIUjgxBCdKBWSjVjd3c/t321H+CIWdmMRWFQgt L0WhU6yFU7dxzFRaEfuKU6dLEhtgYWZM0zMdoIM6z7rTLivX88jUM7oStA9AdI8CpNd9 G6qrL7MfcPRW1kiymjUu9zzsvUYoNfPNfIEpgvg13+5IjeLjMUcNNKbfSHzwm5u4dZ6E ZJnMjcCurlsGukhb50ayJS+qQ1WfwUBaIY4n6U+xxFLc7u9CLo8vXhTJweZ6iKUnFbci 0s1A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o12si9665978pgv.367.2018.04.16.09.16.42; Mon, 16 Apr 2018 09:16:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753493AbeDPQQj (ORCPT + 29 others); Mon, 16 Apr 2018 12:16:39 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:33548 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751135AbeDPQLq (ORCPT ); Mon, 16 Apr 2018 12:11:46 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3GFtZ7k095888 for ; Mon, 16 Apr 2018 12:11:46 -0400 Received: from e17.ny.us.ibm.com (e17.ny.us.ibm.com [129.33.205.207]) by mx0b-001b2d01.pphosted.com with ESMTP id 2hcvcw9cmn-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Mon, 16 Apr 2018 12:11:45 -0400 Received: from localhost by e17.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 16 Apr 2018 12:11:45 -0400 Received: from b01cxnp22035.gho.pok.ibm.com (9.57.198.25) by e17.ny.us.ibm.com (146.89.104.204) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 16 Apr 2018 12:11:39 -0400 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp22035.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3GGBd4M56754282; Mon, 16 Apr 2018 16:11:39 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CBA16B2046; Mon, 16 Apr 2018 13:13:43 -0400 (EDT) Received: from paulmck-ThinkPad-W541 (unknown [9.70.82.108]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP id 93724B205A; Mon, 16 Apr 2018 13:13:43 -0400 (EDT) Received: by paulmck-ThinkPad-W541 (Postfix, from userid 1000) id 527E416C0F61; Mon, 16 Apr 2018 09:12:47 -0700 (PDT) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org Cc: mingo@kernel.org, stern@rowland.harvard.edu, parri.andrea@gmail.com, will.deacon@arm.com, peterz@infradead.org, boqun.feng@gmail.com, npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr, akiyks@gmail.com, Benjamin Herrenschmidt , Arnd Bergmann , Jason Gunthorpe , "Paul E. McKenney" , Ingo Molnar , Jonathan Corbet Subject: [PATCH tools/memory-model 03/17] docs/memory-barriers.txt: Fix broken DMA vs MMIO ordering example Date: Mon, 16 Apr 2018 09:12:31 -0700 X-Mailer: git-send-email 2.5.2 In-Reply-To: <20180416161209.GA6895@linux.vnet.ibm.com> References: <20180416161209.GA6895@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041616-0040-0000-0000-0000041BB4AB X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008863; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000257; SDB=6.01018801; UDB=6.00519703; IPR=6.00798043; MB=3.00020599; MTD=3.00000008; XFM=3.00000015; UTC=2018-04-16 16:11:44 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041616-0041-0000-0000-00000821B83A Message-Id: <1523895165-17576-3-git-send-email-paulmck@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-16_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804160146 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon The section of memory-barriers.txt that describes the dma_Xmb() barriers has an incorrect example claiming that a wmb() is required after writing to coherent memory in order for those writes to be visible to a device before a subsequent MMIO access using writel() can reach the device. In fact, this ordering guarantee is provided (at significant cost on some architectures such as arm and power) by writel, so the wmb() is not necessary. writel_relaxed exists for cases where this ordering is not required. Fix the example and update the text to make this clearer. Cc: Benjamin Herrenschmidt Cc: Arnd Bergmann Cc: Jason Gunthorpe Cc: "Paul E. McKenney" Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Jonathan Corbet Reported-by: Sinan Kaya Signed-off-by: Will Deacon Signed-off-by: Paul E. McKenney --- Documentation/memory-barriers.txt | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) -- 2.5.2 diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 6dafc8085acc..34c1970908a5 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -1920,9 +1920,6 @@ There are some more advanced barrier functions: /* assign ownership */ desc->status = DEVICE_OWN; - /* force memory to sync before notifying device via MMIO */ - wmb(); - /* notify device of new descriptors */ writel(DESC_NOTIFY, doorbell); } @@ -1930,11 +1927,15 @@ There are some more advanced barrier functions: The dma_rmb() allows us guarantee the device has released ownership before we read the data from the descriptor, and the dma_wmb() allows us to guarantee the data is written to the descriptor before the device - can see it now has ownership. The wmb() is needed to guarantee that the - cache coherent memory writes have completed before attempting a write to - the cache incoherent MMIO region. - - See Documentation/DMA-API.txt for more information on consistent memory. + can see it now has ownership. Note that, when using writel(), a prior + wmb() is not needed to guarantee that the cache coherent memory writes + have completed before writing to the MMIO region. The cheaper + writel_relaxed() does not provide this guarantee and must not be used + here. + + See the subsection "Kernel I/O barrier effects" for more information on + relaxed I/O accessors and the Documentation/DMA-API.txt file for more + information on consistent memory. MMIO WRITE BARRIER