From patchwork Wed Jul 8 14:18:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 278210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 527D5C433E0 for ; Wed, 8 Jul 2020 22:45:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 18D7D206F6 for ; Wed, 8 Jul 2020 22:45:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZefM/xwo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 18D7D206F6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54550 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtIpB-0008Lv-85 for qemu-devel@archiver.kernel.org; Wed, 08 Jul 2020 18:45:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48490) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtIRK-0001K5-6Q for qemu-devel@nongnu.org; Wed, 08 Jul 2020 18:21:19 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:56790 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtIRF-00011V-U5 for qemu-devel@nongnu.org; Wed, 08 Jul 2020 18:21:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594246873; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nDcp+wzmLs0Gb0DXNaH+rPXR0aLJmEU8tgRvZl5QKCg=; b=ZefM/xwoxIlZ/e8G7IN/wtVES3ogenHslSOvVHMd5TsVv2y8+dXMe8yyS6PKuqxDsKE2Ro jScLf4zoKhxEKrRTBUNotiJsWaOAEEGtBYCzzgeBFzYSJ25hFNjHiRDSIcuLOTdQ98tz5y VVh3jOtMxP8/NWtaJ8u40yecTIDNK8U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-500-Ct8XxbkLMeqlVd75Wic3jA-1; Wed, 08 Jul 2020 10:19:13 -0400 X-MC-Unique: Ct8XxbkLMeqlVd75Wic3jA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6D3301005504; Wed, 8 Jul 2020 14:19:11 +0000 (UTC) Received: from laptop.redhat.com (ovpn-113-241.ams2.redhat.com [10.36.113.241]) by smtp.corp.redhat.com (Postfix) with ESMTP id 423725BAC3; Wed, 8 Jul 2020 14:19:00 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, peterx@redhat.com Subject: [PATCH v3 00/11] SMMUv3.2 Range-based TLB Invalidation Support Date: Wed, 8 Jul 2020 16:18:45 +0200 Message-Id: <20200708141856.15776-1-eric.auger@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eric.auger@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=eric.auger@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/07 17:25:11 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, robh@kernel.org, robin.murphy@arm.com, mst@redhat.com, zhangfei.gao@foxmail.com, shameerali.kolothum.thodi@huawei.com, will@kernel.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" SMMU3.2 brings the support of range-based TLB invalidation and level hint. When this feature is supported, the SMMUv3 driver is allowed to send TLB invalidations for a range of IOVAs instead of using page based invalidation. Implementing this feature in the virtual SMMUv3 device is mandated for DPDK on guest use case: DPDK uses hugepage buffers and guest sends invalidations for blocks. Without this feature, a guest invalidation of a block of 1GB for instance translates into a storm of page invalidations. Each of them is trapped by the VMM and cascaded downto the physical IOMMU. This completely stalls the execution. This integration issue was initially reported in [1]. Now SMMUv3.2 specifies additional parameters to NH_VA and NH_VAA stage 1 invalidation commands so we can support those extensions. patches [1, 5] are cleanup patches. patches [6] changes the implementation of the VSMMUV3 IOTLB This IOTLB is a minimalist IOTLB implementation that avoids to do the page table walk in case we have an entry in the TLB. Previously entries were page mappings only. Now they can be blocks. patches [7, 9] bring support for range invalidation. Supporting block mappings in the IOTLB look sensible in terms of TLB entry consumption. However looking at virtio/vhost device usage, without block mapping and without range invalidation (< 5.7 kernels it may be less performant. However for recent guest kernels supporting range invalidations [2], the performance should be similar. Best Regards Eric This series can be found at: https://github.com/eauger/qemu.git branch: v5.0.0-smmuv3-ril-v3 References: [1] [RFC v2 4/4] iommu/arm-smmu-v3: add CMD_TLBI_NH_VA_AM command for iova range invalidation (https://lists.linuxfoundation.org/pipermail/iommu/2017-August/023679.html [2] 5.7+ kernels featuring 6a481a95d4c1 iommu/arm-smmu-v3: Add SMMUv3.2 range invalidation support History: v2 -> v3: - restore the Jenkins hash function and keep the key as a struct - simplify the logic in smmu_hash_remove_by_asid_iova - added HAD support - expose AIDR (advertise v3.2 support) and fix IIDR offset v1 -> v2: - added "hw/arm/smmu: Introduce smmu_get_iotlb_key()" - removed "[PATCH 5/9] hw/arm/smmuv3: Store the starting level in SMMUTransTableInfo" - Collected Peter's R-b - In this version the key still features TG/LVL. - More details in individual history logs Eric Auger (11): hw/arm/smmu-common: Factorize some code in smmu_ptw_64() hw/arm/smmu-common: Add IOTLB helpers hw/arm/smmu: Introduce smmu_get_iotlb_key() hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB value hw/arm/smmu-common: Manage IOTLB block entries hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helper hw/arm/smmuv3: Get prepared for range invalidation hw/arm/smmuv3: Fix IIDR offset hw/arm/smmuv3: Let AIDR advertise SMMUv3.0 support hw/arm/smmuv3: Support HAD and advertise SMMUv3.1 support hw/arm/smmuv3: Advertise SMMUv3.2 range invalidation hw/arm/smmu-internal.h | 8 ++ hw/arm/smmuv3-internal.h | 10 +- include/hw/arm/smmu-common.h | 19 +++- include/hw/arm/smmuv3.h | 1 + hw/arm/smmu-common.c | 211 +++++++++++++++++++++++------------ hw/arm/smmuv3.c | 142 +++++++++++------------ hw/arm/trace-events | 12 +- 7 files changed, 255 insertions(+), 148 deletions(-)