From patchwork Thu Sep 13 16:42:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 146632 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp812475ljw; Thu, 13 Sep 2018 09:42:43 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaqwKXvcRsSEAPpXbC0N+Iz5QeRXBTD7CO3WkWb+I6Gu8acleBRhF+fcPuhPFM/9CKq4zTO X-Received: by 2002:a17:902:4403:: with SMTP id k3-v6mr7958230pld.243.1536856963073; Thu, 13 Sep 2018 09:42:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536856963; cv=none; d=google.com; s=arc-20160816; b=JF2LaCiY0XhAcpQRxfQe6pXgmCmBxuvzer5gqUYs7S4r363vgrbX0eHlTIOSmdw1lk 7b4EO85zPTXFPtJWuK7e3b+vcxaSU1KCI4frm4n8wI9ILwVWxQnud0G+qj2FmhovZxhx EDbPVb2Bhxm/QtHbJHzgu4yvlcX8K6wMsyO16/mRJFH4OV3xELH62DjqRxETkHggDpXb usSbI4KTDLWAsv4AMihzFT5kAxTP3KoIl+afCJlgKMag1OnWBKhHDQoCiXTaGqcikRfo b2tdf0zxRyYIsnW7e4jbX3FAB1SOqaeq0ly/RIpGSm3C7nkXBpq5vqI+3R+N7QFZneNi kzfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=VO+SOhQYcjg8d4MISSqnbdk7eAwIW87RwAoIVXyZh3M=; b=qDOENlQp6msOO+jjiO+LjnEL4TIoWQkajNpaY8mbM2DLv6lPEsHxp8K696ZWX+e/co fUwymNrHxMQCKtTBOuFa0VqD3ppLjluA09XP++yv3Rw2IS+BHdONSf1/OSB7gOFTqA+4 diP4SXz1MwvGyvxLy26sOiPHZfHnfr2zXBHVdfCVt2hsMHMDR26OahguXFZQkNXlc0XQ NMCcun6jYh3fI2RGAq9ZAekkvavr1xRBtOa8o2xzkCo2lwULxmxVaACircN0T612CEwt Mfcb0LGyJaAcKo+ltBeVZXc8te/8ua+L4BqgiiOm2MBbB+wNqEzs0B+5ekFpcwDN01jy VZdg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 14-v6si4726138plb.230.2018.09.13.09.42.42; Thu, 13 Sep 2018 09:42:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728200AbeIMVw5 (ORCPT + 32 others); Thu, 13 Sep 2018 17:52:57 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:51114 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726452AbeIMVw4 (ORCPT ); Thu, 13 Sep 2018 17:52:56 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A504915BF; Thu, 13 Sep 2018 09:42:39 -0700 (PDT) Received: from e110467-lin.cambridge.arm.com (e110467-lin.emea.arm.com [10.4.12.131]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D93B83F703; Thu, 13 Sep 2018 09:42:37 -0700 (PDT) From: Robin Murphy To: joro@8bytes.org, will.deacon@arm.com, thunder.leizhen@huawei.com, iommu@lists.linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: linuxarm@huawei.com, guohanjun@huawei.com, huawei.libin@huawei.com, john.garry@huawei.com Subject: [PATCH v6 2/7] iommu/dma: Add support for non-strict mode Date: Thu, 13 Sep 2018 17:42:19 +0100 Message-Id: X-Mailer: git-send-email 2.19.0.dirty In-Reply-To: References: MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Zhen Lei 1. Save the related domain pointer in struct iommu_dma_cookie, make iovad capable call domain->ops->flush_iotlb_all to flush TLB. 2. During the iommu domain initialization phase, base on domain->non_strict field to check whether non-strict mode is supported or not. If so, call init_iova_flush_queue to register iovad->flush_cb callback. 3. All unmap(contains iova-free) APIs will finally invoke __iommu_dma_unmap -->iommu_dma_free_iova. If the domain is non-strict, call queue_iova to put off iova freeing, and omit iommu_tlb_sync operation. Signed-off-by: Zhen Lei [rm: convert raw boolean to domain attribute] Signed-off-by: Robin Murphy --- drivers/iommu/dma-iommu.c | 31 ++++++++++++++++++++++++++++++- include/linux/iommu.h | 1 + 2 files changed, 31 insertions(+), 1 deletion(-) -- 2.19.0.dirty diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 511ff9a1d6d9..d91849fe4ebe 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -55,8 +55,13 @@ struct iommu_dma_cookie { }; struct list_head msi_page_list; spinlock_t msi_lock; + + /* Only be assigned in non-strict mode, otherwise it's NULL */ + struct iommu_domain *domain; }; +static bool iommu_dma_non_strict __read_mostly; + static inline size_t cookie_msi_granule(struct iommu_dma_cookie *cookie) { if (cookie->type == IOMMU_DMA_IOVA_COOKIE) @@ -257,6 +262,17 @@ static int iova_reserve_iommu_regions(struct device *dev, return ret; } +static void iommu_dma_flush_iotlb_all(struct iova_domain *iovad) +{ + struct iommu_dma_cookie *cookie; + struct iommu_domain *domain; + + cookie = container_of(iovad, struct iommu_dma_cookie, iovad); + domain = cookie->domain; + + domain->ops->flush_iotlb_all(domain); +} + /** * iommu_dma_init_domain - Initialise a DMA mapping domain * @domain: IOMMU domain previously prepared by iommu_get_dma_cookie() @@ -275,6 +291,7 @@ int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, struct iommu_dma_cookie *cookie = domain->iova_cookie; struct iova_domain *iovad = &cookie->iovad; unsigned long order, base_pfn, end_pfn; + int attr = 1; if (!cookie || cookie->type != IOMMU_DMA_IOVA_COOKIE) return -EINVAL; @@ -308,6 +325,13 @@ int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, } init_iova_domain(iovad, 1UL << order, base_pfn); + + if (iommu_dma_non_strict && !iommu_domain_set_attr(domain, + DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE, &attr)) { + cookie->domain = domain; + init_iova_flush_queue(iovad, iommu_dma_flush_iotlb_all, NULL); + } + if (!dev) return 0; @@ -393,6 +417,9 @@ static void iommu_dma_free_iova(struct iommu_dma_cookie *cookie, /* The MSI case is only ever cleaning up its most recent allocation */ if (cookie->type == IOMMU_DMA_MSI_COOKIE) cookie->msi_iova -= size; + else if (cookie->domain) /* non-strict mode */ + queue_iova(iovad, iova_pfn(iovad, iova), + size >> iova_shift(iovad), 0); else free_iova_fast(iovad, iova_pfn(iovad, iova), size >> iova_shift(iovad)); @@ -408,7 +435,9 @@ static void __iommu_dma_unmap(struct iommu_domain *domain, dma_addr_t dma_addr, dma_addr -= iova_off; size = iova_align(iovad, size + iova_off); - WARN_ON(iommu_unmap(domain, dma_addr, size) != size); + WARN_ON(iommu_unmap_fast(domain, dma_addr, size) != size); + if (!cookie->domain) + iommu_tlb_sync(domain); iommu_dma_free_iova(cookie, dma_addr, size); } diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 87994c265bf5..decabe8e8dbe 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -124,6 +124,7 @@ enum iommu_attr { DOMAIN_ATTR_FSL_PAMU_ENABLE, DOMAIN_ATTR_FSL_PAMUV1, DOMAIN_ATTR_NESTING, /* two stages of translation */ + DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE, DOMAIN_ATTR_MAX, };