From patchwork Wed Aug 15 01:28:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 144256 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp249568ljj; Tue, 14 Aug 2018 18:32:43 -0700 (PDT) X-Google-Smtp-Source: AA+uWPzxSkHy+JjcSOVxjOJWqX57nk32wgPOCMoZbEIAIb6sjtaIE6SlGXDrdJxbrGm5UpdnAQTZ X-Received: by 2002:a63:7c5c:: with SMTP id l28-v6mr23613443pgn.352.1534296762855; Tue, 14 Aug 2018 18:32:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534296762; cv=none; d=google.com; s=arc-20160816; b=VztDIymB7WaUoBJ9GuqlgdFo1Ykp1Bv6TYv3zefnGei968sZgK3Tz8DKhMnOMCBrvc Syg+t14hjYy0XTnmSfKVvCvIILt+g8tuGh2yVAboNKLkgsv9NxjqcUv7UNVI1S34g0cJ ie1jkCXhyC0mGRcuTBqlW4RrHEgsY6AQT3K/+jF04Kj3zQG248dSHtayAv7foxbdOp4/ LYAHLqyDT/O8pmO2bc+2lIMJEhfYSkE7NBQSghV37TH64ElEImY67P0eZkOlOptqXs+/ HyJPoQFA7g0zY7wWZbEIiGHv4+qEkYzmwKeureCev4W2Cxy1uE+NIWMUcno137buEuqO lVsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=se9p82M/5zjhrYrEl2Ot+WxCbSN8kdTK8AnDR5u3G4c=; b=RfNWLphv1ySIoWM6GioO+8UGZj8QFGe379PpuOBt2XL9ShyLg9IFIKjgt/+Ka8hVSY CmRBhkIQQaMD9nKEXXDOWwevFVTSYdtT0Edx31c6C8pQNLRWYiAAZXN4rS0cEcBL6oTS tqDFwJ6SQYdZkGzFScpUoI8Sz7VdsBQ4m2WLxztssAfJsQ0Z4IHL/bQK4X2zLJXK904N YpNOPiuBwPI865THI40axnjZ4Ez8l6OyiTROsS6eq9Pef9gHZKFfTP8iM8iIP4drLmVm oPCkztFduAUWTUGjdAaFirecnV086tqcG3iMrmtcy3wZLy4Ng/gbX8vPKnW4CbXrNQqA Aumw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n6-v6si22708356pgt.268.2018.08.14.18.32.42; Tue, 14 Aug 2018 18:32:42 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727861AbeHOEWc (ORCPT + 32 others); Wed, 15 Aug 2018 00:22:32 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:44777 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726300AbeHOEWc (ORCPT ); Wed, 15 Aug 2018 00:22:32 -0400 Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 86C3FCE974D48; Wed, 15 Aug 2018 09:32:35 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEMS402-HUB.china.huawei.com (10.3.19.202) with Microsoft SMTP Server id 14.3.399.0; Wed, 15 Aug 2018 09:32:27 +0800 From: Zhen Lei To: Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel , iommu , linux-kernel CC: Zhen Lei , LinuxArm , Hanjun Guo , Libin , "John Garry" Subject: [PATCH v5 1/5] iommu/arm-smmu-v3: fix the implementation of flush_iotlb_all hook Date: Wed, 15 Aug 2018 09:28:26 +0800 Message-ID: <1534296510-12888-2-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> References: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org .flush_iotlb_all can not just wait for previous tlbi operations to be completed, but should also invalid all TLBs of the related domain. Signed-off-by: Zhen Lei Reviewed-by: Robin Murphy --- drivers/iommu/arm-smmu-v3.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) -- 1.8.3 diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 1d64710..4402187 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1770,6 +1770,14 @@ static int arm_smmu_map(struct iommu_domain *domain, unsigned long iova, return ops->unmap(ops, iova, size); } +static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + + if (smmu_domain->smmu) + arm_smmu_tlb_inv_context(smmu_domain); +} + static void arm_smmu_iotlb_sync(struct iommu_domain *domain) { struct arm_smmu_device *smmu = to_smmu_domain(domain)->smmu; @@ -1998,7 +2006,7 @@ static void arm_smmu_put_resv_regions(struct device *dev, .map = arm_smmu_map, .unmap = arm_smmu_unmap, .map_sg = default_iommu_map_sg, - .flush_iotlb_all = arm_smmu_iotlb_sync, + .flush_iotlb_all = arm_smmu_flush_iotlb_all, .iotlb_sync = arm_smmu_iotlb_sync, .iova_to_phys = arm_smmu_iova_to_phys, .add_device = arm_smmu_add_device, From patchwork Wed Aug 15 01:28:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 144255 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp249547ljj; Tue, 14 Aug 2018 18:32:40 -0700 (PDT) X-Google-Smtp-Source: AA+uWPwyYHAGIMgdql7cIzxAVjYla635JZgGH+6KS9OvdTrJVD5DlzqJsCmJYH0kl/Z05jOUvEmI X-Received: by 2002:a17:902:47c2:: with SMTP id d2-v6mr22197461plh.139.1534296760343; Tue, 14 Aug 2018 18:32:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534296760; cv=none; d=google.com; s=arc-20160816; b=A8uvFSTK0K7lMXRzBkcy4uNsU6bycqXpX251r1US/eMBIhGka7+YtvIeKe8iG3zuGc ylYuGsWKrtu/pMtFNmVPCKaykofauCZZziEMTHYODRLigutlRBckr5JHo5/4W/4c96H/ F9Ds1/MXvUSglgoOgrd8VUg0bswc3pVa5v422IMXqgurTCyn/XDAgUpgbSLwVHz1uFfo miKZ8AmfyzksJosap1OCmT7wWvPirO9veDt8mTtKVEKoQxI8sFSYgWbj9Zy0CU8IySBR akrvfapSapWZhBqcGGBxsc3jKhHUJNNXJDgXDNI+Odd8EfRMsgWqy6ICgeQsblmxR8FW cHsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=K13mHjTtmhhRT1dEN7hBy7iI4gd++I7lxeqryHWmrr8=; b=unvfy/7uYQYO32Hx955iZbwt7wVb2/qjBFqHHsJlUBM/sKiDBmnHDJpjUSxqA2SLJp g3igUx1hytWgleQ3+A/1fESAw+TUi3CphuAW02R+ubbAcG3qkRgT5A2ZmTWvDC/yXnHb q/Ior3YwMXXjPSe/d6lrQr1jvMkQpuC8NFbLu6+BQxvYFXcUjt9RwtHO5bxpkhpEwmJw xlZftpQFRuokHbjEjyA6gnzMGfXl1357tomdZbG6PO7cNK5UnZZ+Jr0uEANFD7H5/iyA egmkyO+93RFLQK8n9dBe6ZYImtOLG5evfMWE9G704bICTVFVLHP89GUyrs3i1Dk5HC7x u2KA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q27-v6si22423596pfj.149.2018.08.14.18.32.40; Tue, 14 Aug 2018 18:32:40 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727055AbeHOEWa (ORCPT + 32 others); Wed, 15 Aug 2018 00:22:30 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:10725 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725847AbeHOEWa (ORCPT ); Wed, 15 Aug 2018 00:22:30 -0400 Received: from DGGEMS412-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 4AAB8D7ED320D; Wed, 15 Aug 2018 09:32:34 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEMS412-HUB.china.huawei.com (10.3.19.212) with Microsoft SMTP Server id 14.3.399.0; Wed, 15 Aug 2018 09:32:28 +0800 From: Zhen Lei To: Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel , iommu , linux-kernel CC: Zhen Lei , LinuxArm , Hanjun Guo , Libin , "John Garry" Subject: [PATCH v5 2/5] iommu/dma: add support for non-strict mode Date: Wed, 15 Aug 2018 09:28:27 +0800 Message-ID: <1534296510-12888-3-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> References: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 1. Save the related domain pointer in struct iommu_dma_cookie, make iovad capable call domain->ops->flush_iotlb_all to flush TLB. 2. During the iommu domain initialization phase, base on domain->non_strict field to check whether non-strict mode is supported or not. If so, call init_iova_flush_queue to register iovad->flush_cb callback. 3. All unmap(contains iova-free) APIs will finally invoke __iommu_dma_unmap -->iommu_dma_free_iova. If the domain is non-strict, call queue_iova to put off iova freeing, and omit iommu_tlb_sync operation. Signed-off-by: Zhen Lei --- drivers/iommu/dma-iommu.c | 29 ++++++++++++++++++++++++++++- drivers/iommu/iommu.c | 1 + include/linux/iommu.h | 1 + 3 files changed, 30 insertions(+), 1 deletion(-) -- 1.8.3 diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index ddcbbdb..f0257e9 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -55,6 +55,9 @@ struct iommu_dma_cookie { }; struct list_head msi_page_list; spinlock_t msi_lock; + + /* Only be assigned in non-strict mode, otherwise it's NULL */ + struct iommu_domain *domain; }; static inline size_t cookie_msi_granule(struct iommu_dma_cookie *cookie) @@ -257,6 +260,17 @@ static int iova_reserve_iommu_regions(struct device *dev, return ret; } +static void iommu_dma_flush_iotlb_all(struct iova_domain *iovad) +{ + struct iommu_dma_cookie *cookie; + struct iommu_domain *domain; + + cookie = container_of(iovad, struct iommu_dma_cookie, iovad); + domain = cookie->domain; + + domain->ops->flush_iotlb_all(domain); +} + /** * iommu_dma_init_domain - Initialise a DMA mapping domain * @domain: IOMMU domain previously prepared by iommu_get_dma_cookie() @@ -308,6 +322,14 @@ int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, } init_iova_domain(iovad, 1UL << order, base_pfn); + + if (domain->non_strict) { + BUG_ON(!domain->ops->flush_iotlb_all); + + cookie->domain = domain; + init_iova_flush_queue(iovad, iommu_dma_flush_iotlb_all, NULL); + } + if (!dev) return 0; @@ -390,6 +412,9 @@ static void iommu_dma_free_iova(struct iommu_dma_cookie *cookie, /* The MSI case is only ever cleaning up its most recent allocation */ if (cookie->type == IOMMU_DMA_MSI_COOKIE) cookie->msi_iova -= size; + else if (cookie->domain) /* non-strict mode */ + queue_iova(iovad, iova_pfn(iovad, iova), + size >> iova_shift(iovad), 0); else free_iova_fast(iovad, iova_pfn(iovad, iova), size >> iova_shift(iovad)); @@ -405,7 +430,9 @@ static void __iommu_dma_unmap(struct iommu_domain *domain, dma_addr_t dma_addr, dma_addr -= iova_off; size = iova_align(iovad, size + iova_off); - WARN_ON(iommu_unmap(domain, dma_addr, size) != size); + WARN_ON(iommu_unmap_fast(domain, dma_addr, size) != size); + if (!domain->non_strict) + iommu_tlb_sync(domain); iommu_dma_free_iova(cookie, dma_addr, size); } diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 63b3756..6255a69 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1263,6 +1263,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus, domain->ops = bus->iommu_ops; domain->type = type; + domain->non_strict = false; /* Assume all sizes by default; the driver may override this later */ domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 19938ee..4bbcf39 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -88,6 +88,7 @@ struct iommu_domain_geometry { struct iommu_domain { unsigned type; + bool non_strict; const struct iommu_ops *ops; unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */ iommu_fault_handler_t handler; From patchwork Wed Aug 15 01:28:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 144257 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp249583ljj; Tue, 14 Aug 2018 18:32:44 -0700 (PDT) X-Google-Smtp-Source: AA+uWPxyQF2HjuHZ+UN+WTXzxAMXjFt/rYWhsQD5Y+Iz2Vgje2UhOCQUa0hT+7GykDKH8Ik4KyAe X-Received: by 2002:a62:a6db:: with SMTP id r88-v6mr25718226pfl.60.1534296764293; Tue, 14 Aug 2018 18:32:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534296764; cv=none; d=google.com; s=arc-20160816; b=vb5nqUKR55D8Py47fukOZYmy9+WqHcpsW3X7xe0hn2mN8jnQtoO1f664OTUsiON/2G R6N5DJ1KlwcWX95BMnzU7KMI3KUpRDFNAAOwDi36QhpqMMigqplLc2Npp2JZlegabYzo uyaF5TByStEg+fWfkvo9ZMWJwLF8wVg2YKGObzrt2+DPpuU4oKamzPOz70szQVBAgoC3 c8ypZgnER0uLEUiEOrR50Q6ofhLVZ4GBKinjS/jPE4So37cwVQkWy7fvniy78JeiZ7Jv 156WJ/s6hX8ezA7ahe723rkjBJ2DmH+PQGBwOoCjRwKNtRx1mfni5pb60I1aij7IiR2g q0uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=zDd6Ka1uRMtFbNXftwIto2qjXK8iAZElSqcNTX7p/wY=; b=KDyBqETQJ5kMrYhTSWamOeLqXkWE0s0WzdYDP/kMN8HuxE0h7Fi2XgAAbkuHbF1+5X K+JoHRfqZWTZapgav+DgKl8Bp8KkURGSrTAOrygGJ2msc1MrbhMSNyuoRxj5eEmAM6LF VvJaNFEgNtxMRJL92HAQjgQdSf2dLOlfBEDgolIWBp9lLEDe/Vk77ajaRT3H2muc/Ixv NzWSxOyqeZo3EH6R+0uKIkkFdmSUfSIS5KpCKdmOixNwVyvSBjAIYgBpCmGEEToXM9y6 oRgwawGNIpt6gwWHs8bl45uQfg2kh1RYFPdh235bm69/eN2SzYpzIYxUS/SW2MFkTU9b lblA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j1-v6si18196866pfh.63.2018.08.14.18.32.44; Tue, 14 Aug 2018 18:32:44 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728144AbeHOEWd (ORCPT + 32 others); Wed, 15 Aug 2018 00:22:33 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:43501 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725847AbeHOEWc (ORCPT ); Wed, 15 Aug 2018 00:22:32 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 19BC4471CA7EB; Wed, 15 Aug 2018 09:32:38 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.399.0; Wed, 15 Aug 2018 09:32:30 +0800 From: Zhen Lei To: Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel , iommu , linux-kernel CC: Zhen Lei , LinuxArm , Hanjun Guo , Libin , "John Garry" Subject: [PATCH v5 3/5] iommu/io-pgtable-arm: add support for non-strict mode Date: Wed, 15 Aug 2018 09:28:28 +0800 Message-ID: <1534296510-12888-4-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> References: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To support the non-strict mode, now we only tlbi and sync for the strict mode. But for the non-leaf case, always follow strict mode. Signed-off-by: Zhen Lei --- drivers/iommu/io-pgtable-arm.c | 20 ++++++++++++++------ drivers/iommu/io-pgtable.h | 3 +++ 2 files changed, 17 insertions(+), 6 deletions(-) -- 1.8.3 diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 010a254..20d3e98 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -538,6 +538,7 @@ static size_t arm_lpae_split_blk_unmap(struct arm_lpae_io_pgtable *data, phys_addr_t blk_paddr; size_t tablesz = ARM_LPAE_GRANULE(data); size_t split_sz = ARM_LPAE_BLOCK_SIZE(lvl, data); + size_t unmapped = size; int i, unmap_idx = -1; if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS)) @@ -575,11 +576,16 @@ static size_t arm_lpae_split_blk_unmap(struct arm_lpae_io_pgtable *data, tablep = iopte_deref(pte, data); } - if (unmap_idx < 0) - return __arm_lpae_unmap(data, iova, size, lvl, tablep); + if (unmap_idx < 0) { + unmapped = __arm_lpae_unmap(data, iova, size, lvl, tablep); + if (!(data->iop.cfg.quirks & IO_PGTABLE_QUIRK_NON_STRICT)) + return unmapped; + } io_pgtable_tlb_add_flush(&data->iop, iova, size, size, true); - return size; + io_pgtable_tlb_sync(&data->iop); + + return unmapped; } static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data, @@ -609,7 +615,7 @@ static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data, io_pgtable_tlb_sync(iop); ptep = iopte_deref(pte, data); __arm_lpae_free_pgtable(data, lvl + 1, ptep); - } else { + } else if (!(iop->cfg.quirks & IO_PGTABLE_QUIRK_NON_STRICT)) { io_pgtable_tlb_add_flush(iop, iova, size, size, true); } @@ -771,7 +777,8 @@ static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg) u64 reg; struct arm_lpae_io_pgtable *data; - if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS | IO_PGTABLE_QUIRK_NO_DMA)) + if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS | IO_PGTABLE_QUIRK_NO_DMA | + IO_PGTABLE_QUIRK_NON_STRICT)) return NULL; data = arm_lpae_alloc_pgtable(cfg); @@ -863,7 +870,8 @@ static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg) struct arm_lpae_io_pgtable *data; /* The NS quirk doesn't apply at stage 2 */ - if (cfg->quirks & ~IO_PGTABLE_QUIRK_NO_DMA) + if (cfg->quirks & ~(IO_PGTABLE_QUIRK_NO_DMA | + IO_PGTABLE_QUIRK_NON_STRICT)) return NULL; data = arm_lpae_alloc_pgtable(cfg); diff --git a/drivers/iommu/io-pgtable.h b/drivers/iommu/io-pgtable.h index 2df7909..beb14a3 100644 --- a/drivers/iommu/io-pgtable.h +++ b/drivers/iommu/io-pgtable.h @@ -71,12 +71,15 @@ struct io_pgtable_cfg { * be accessed by a fully cache-coherent IOMMU or CPU (e.g. for a * software-emulated IOMMU), such that pagetable updates need not * be treated as explicit DMA data. + * IO_PGTABLE_QUIRK_NON_STRICT: Put off TLBs invalidation and release + * memory first. */ #define IO_PGTABLE_QUIRK_ARM_NS BIT(0) #define IO_PGTABLE_QUIRK_NO_PERMS BIT(1) #define IO_PGTABLE_QUIRK_TLBI_ON_MAP BIT(2) #define IO_PGTABLE_QUIRK_ARM_MTK_4GB BIT(3) #define IO_PGTABLE_QUIRK_NO_DMA BIT(4) + #define IO_PGTABLE_QUIRK_NON_STRICT BIT(5) unsigned long quirks; unsigned long pgsize_bitmap; unsigned int ias; From patchwork Wed Aug 15 01:28:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 144258 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp249605ljj; Tue, 14 Aug 2018 18:32:46 -0700 (PDT) X-Google-Smtp-Source: AA+uWPwNzVhJOuGBflviscb62t9ZL7OIAE46849+95EYVgCWjPX5mulCTcH/vKiydDlc39JMn5SH X-Received: by 2002:a62:d085:: with SMTP id p127-v6mr25824026pfg.119.1534296766035; Tue, 14 Aug 2018 18:32:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534296766; cv=none; d=google.com; s=arc-20160816; b=HnuJBt4Vlf68sJKWZmMJ/3IBGDD+OivyQU4TOwch732KAeUBF6W5rHtL5JMCSloQU7 q6xecIqQwo0OfUBwkFAEt6hJaL6ps//KCX8FeU413ZJ/MbAFblrvBIlVod5jhIaIPDJ7 G4YsEr4wWiugXYw+PpMgXweprlw8Uinjl8ydIyTOuqOTpJc8SecPDp7UAQCCjLElsmOf /HUztxQd3U5bUL0bZUwgqZFDxHQvU5/t5Gbi9Y4yuRFDjauZZKUwmrXarJQbG8dcN9Gj 8E2HseO4AqQ6I/w2S8C0TWnWDrFlYEtiiH8nhahATJgtE0yuEIXnSxzidvkLindPc8QT o/Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=uwckzlGUE086/cPrL2vO0bQihYtJi0NA+b1kVGjDw+E=; b=uZ0K9dyd1q64HwWzMomhb4qhI1ebuVNx0KFeUwtt63Si1x4xbqqHLRxIfiPtbUenMX D6Wm2EnZlJXvIb/8pTfXuJ//JFjw+Swj3V4tWrMBFw4HNvsvbFjs8C5TvkAwkFP6S6xP VuimMkRba5Nx7AJat2Q41u2pJ5i0If4rtWbWQzyhIh+wpWTEhQiEx84fydPXk6PaBsQi eR0VwigkwKyaxmTGVZnHmwk/1jqL5UQUN6L//MgA8wkRQgPHwyh6/N2oSrs6IONcQzF4 y5tqBRrlvFlrLHyTX7k///LmZNp9Dph1eCXW1tcpvRHgGhGrD4d3wZAzpP6G5VsfNr52 U/9A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j3-v6si21864914pgs.329.2018.08.14.18.32.45; Tue, 14 Aug 2018 18:32:46 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728368AbeHOEWf (ORCPT + 32 others); Wed, 15 Aug 2018 00:22:35 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:44799 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725847AbeHOEWe (ORCPT ); Wed, 15 Aug 2018 00:22:34 -0400 Received: from DGGEMS412-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id BBF06FC51CAB9; Wed, 15 Aug 2018 09:32:38 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEMS412-HUB.china.huawei.com (10.3.19.212) with Microsoft SMTP Server id 14.3.399.0; Wed, 15 Aug 2018 09:32:31 +0800 From: Zhen Lei To: Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel , iommu , linux-kernel CC: Zhen Lei , LinuxArm , Hanjun Guo , Libin , "John Garry" Subject: [PATCH v5 4/5] iommu/arm-smmu-v3: add support for non-strict mode Date: Wed, 15 Aug 2018 09:28:29 +0800 Message-ID: <1534296510-12888-5-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> References: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Dynamically choose strict or non-strict mode for page table config based on the iommu domain type. Signed-off-by: Zhen Lei --- drivers/iommu/arm-smmu-v3.c | 5 +++++ 1 file changed, 5 insertions(+) -- 1.8.3 diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 4402187..61eb7ec 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1622,6 +1622,11 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain) if (smmu->features & ARM_SMMU_FEAT_COHERENCY) pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_NO_DMA; + if (domain->type == IOMMU_DOMAIN_DMA) { + domain->non_strict = true; + pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT; + } + pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain); if (!pgtbl_ops) return -ENOMEM; From patchwork Wed Aug 15 01:28:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 144259 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp249628ljj; Tue, 14 Aug 2018 18:32:48 -0700 (PDT) X-Google-Smtp-Source: AA+uWPwDW/QCwvbqqAIB87c7eyp4H5tTK34pAyuqKHSLr5jLHHmrkYI6QZY6JsrEKtYgK8aerf9P X-Received: by 2002:a63:24c:: with SMTP id 73-v6mr23730502pgc.252.1534296768449; Tue, 14 Aug 2018 18:32:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534296768; cv=none; d=google.com; s=arc-20160816; b=PKKVjj+4Sh3rADS74qlH98wG8u7qgxe0UXt55nIAX57AjtHqvEHBWToDh5fC/1jtIj MivqFUl1gi3orUFp49M1pgILMKEqRB+v4ctWTXsWypMXzZ2ZjtZ/BdRySvOZUWUqFp+h 1Wj66iNhGk/5nPkH6UhoKMCJy0uWHNgOFtKDCc2a5HTBDHXgzSNcb5Ys9zMCp8ZauV5T UvF5SgwM9BxZO2csEZG/wxFeWlDIy5YeZrOxGuMH4RAcL0OXaSB77l7Dx6aueXUcbhvQ 92aLptcXVTkqS4mjKTHWFWfBOwLFcDdvMkiFWCMaixKwBC1QfBDWSyivAicdolZcL7en oOqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=c2qYry4B0b2KU9VYW8Qj2Ur3iXZyAIlSlbpa2aPlB9k=; b=sqkXdmFfCppQ8aotlDKn2iPYSxdKCB0mBwxh9t/81kviPkbkYb6wZ9x/puL5xKTDxM paZEHyw1xebSMbx8gkxDUxVNFmWGFzFmCGCdEGHuekI6RpMjKbYKTPbh3eiBBUb+niG+ Lz3JWHisfZVDEcr077w+eBS7BF7xXY1DfQ6FU1kjAGJcQgtmSH2r8dZo2HbMyB2d3Tot lwxsx+lu0+kgXqJ2eLg5FRroGFu0VaeDWnvvB2y4Td/sERYNQW2XeB54BK8FvCPKRxdu JnbrEa52AvuKzyEhlUhGu1BaI3pTakNubcA7x8KZcwfuQG82ZDGNhqHJC6IYSqe7iCKW YeBg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3-v6si11524010plq.285.2018.08.14.18.32.48; Tue, 14 Aug 2018 18:32:48 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728472AbeHOEWi (ORCPT + 32 others); Wed, 15 Aug 2018 00:22:38 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:10726 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725847AbeHOEWh (ORCPT ); Wed, 15 Aug 2018 00:22:37 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 6F72B8C1BE635; Wed, 15 Aug 2018 09:32:42 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.399.0; Wed, 15 Aug 2018 09:32:33 +0800 From: Zhen Lei To: Robin Murphy , Will Deacon , Joerg Roedel , linux-arm-kernel , iommu , linux-kernel CC: Zhen Lei , LinuxArm , Hanjun Guo , Libin , "John Garry" Subject: [PATCH v5 5/5] iommu/arm-smmu-v3: add bootup option "iommu.non_strict" Date: Wed, 15 Aug 2018 09:28:30 +0800 Message-ID: <1534296510-12888-6-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> References: <1534296510-12888-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add a bootup option to make the system manager can choose which mode to be used. The default mode is strict. Signed-off-by: Zhen Lei --- Documentation/admin-guide/kernel-parameters.txt | 13 +++++++++++++ drivers/iommu/arm-smmu-v3.c | 22 +++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) -- 1.8.3 diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 5cde1ff..cb9d043e 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1720,6 +1720,19 @@ nobypass [PPC/POWERNV] Disable IOMMU bypass, using IOMMU for PCI devices. + iommu.non_strict= [ARM64] + Format: { "0" | "1" } + 0 - strict mode, default. + Release IOVAs after the related TLBs are invalid + completely. + 1 - non-strict mode. + Put off TLBs invalidation and release memory first. + It's good for scatter-gather performance but lacks + full isolation, an untrusted device can access the + reused memory because the TLBs may still valid. + Please take full consideration before choosing this + mode. Note that, VFIO will always use strict mode. + iommu.passthrough= [ARM64] Configure DMA to bypass the IOMMU by default. Format: { "0" | "1" } diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 61eb7ec..0eda90e 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -631,6 +631,26 @@ struct arm_smmu_option_prop { { 0, NULL}, }; +static bool smmu_non_strict __read_mostly; + +static int __init arm_smmu_setup(char *str) +{ + int ret; + + ret = kstrtobool(str, &smmu_non_strict); + if (ret) + return ret; + + if (smmu_non_strict) { + pr_warn("WARNING: iommu non-strict mode is chosen.\n" + "It's good for scatter-gather performance but lacks full isolation\n"); + add_taint(TAINT_WARN, LOCKDEP_STILL_OK); + } + + return 0; +} +early_param("iommu.non_strict", arm_smmu_setup); + static inline void __iomem *arm_smmu_page1_fixup(unsigned long offset, struct arm_smmu_device *smmu) { @@ -1622,7 +1642,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain) if (smmu->features & ARM_SMMU_FEAT_COHERENCY) pgtbl_cfg.quirks = IO_PGTABLE_QUIRK_NO_DMA; - if (domain->type == IOMMU_DOMAIN_DMA) { + if ((domain->type == IOMMU_DOMAIN_DMA) && smmu_non_strict) { domain->non_strict = true; pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT; }