From patchwork Thu May 21 09:10:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 282206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6B25C433DF for ; Thu, 21 May 2020 09:18:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 86120206BE for ; Thu, 21 May 2020 09:18:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QpOe7hYt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 86120206BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56686 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jbhLP-0007RC-M7 for qemu-devel@archiver.kernel.org; Thu, 21 May 2020 05:18:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55574) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jbhEw-0004WQ-SO for qemu-devel@nongnu.org; Thu, 21 May 2020 05:11:46 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:55264 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jbhEv-0005Kw-Tg for qemu-devel@nongnu.org; Thu, 21 May 2020 05:11:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590052304; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xhpy5/KHWu7f7iBNn50umu1Z55N8FKEPXHzp1efxKzw=; b=QpOe7hYtdVQ8phZYlbIEKz4lEzFIZy/tfccJWzocNluPYyvC9dy61dFvbtj7P+7UP47kwJ 6uT4huVXlfSJHjnE4+c4J5tGhCDh1dO4XgH+UoESU3oAXBjcC+le/LkYCh11lNRUoObRax qS34cwvl6kIgMt8S3CWJfK+p9fHfJlI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-56-PCw5C86lNgSEPAipt1THbA-1; Thu, 21 May 2020 05:11:41 -0400 X-MC-Unique: PCw5C86lNgSEPAipt1THbA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A52C6EC1AF; Thu, 21 May 2020 09:11:39 +0000 (UTC) Received: from laptop.redhat.com (ovpn-113-56.ams2.redhat.com [10.36.113.56]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4E2A55D9C9; Thu, 21 May 2020 09:11:31 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, peterx@redhat.com Subject: [PATCH 3/9] hw/arm/smmu: Simplify the IOTLB key format Date: Thu, 21 May 2020 11:10:53 +0200 Message-Id: <20200521091059.9453-4-eric.auger@redhat.com> In-Reply-To: <20200521091059.9453-1-eric.auger@redhat.com> References: <20200521091059.9453-1-eric.auger@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/21 01:47:40 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, robh@kernel.org, robin.murphy@arm.com, mst@redhat.com, zhangfei.gao@foxmail.com, shameerali.kolothum.thodi@huawei.com, will@kernel.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Instead of using a Jenkins hash function to generate the key let's just use a 64 bit unsigned integer that contains the asid and the 40 upper bits of the iova. A maximum of 52-bit IOVA is supported. This change in the key format also prepares for the addition of new fields in subsequent patches (granule and level). Signed-off-by: Eric Auger --- hw/arm/smmu-internal.h | 4 +++ include/hw/arm/smmu-common.h | 6 +--- hw/arm/smmu-common.c | 53 ++++++++++++++---------------------- 3 files changed, 26 insertions(+), 37 deletions(-) diff --git a/hw/arm/smmu-internal.h b/hw/arm/smmu-internal.h index 7794d6d394..2ecb6f1dc6 100644 --- a/hw/arm/smmu-internal.h +++ b/hw/arm/smmu-internal.h @@ -96,4 +96,8 @@ uint64_t iova_level_offset(uint64_t iova, int inputsize, MAKE_64BIT_MASK(0, gsz - 3); } +#define SMMU_IOTLB_ASID_SHIFT 40 + +#define SMMU_IOTLB_ASID(key) (((key) >> SMMU_IOTLB_ASID_SHIFT) & 0xFFFF) +#define SMMU_IOTLB_IOVA(key) (((key) & MAKE_64BIT_MASK(0, 40)) << 12) #endif diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index 1dceec5cb1..7b9d2f0eb7 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -88,11 +88,6 @@ typedef struct SMMUPciBus { SMMUDevice *pbdev[]; /* Parent array is sparse, so dynamically alloc */ } SMMUPciBus; -typedef struct SMMUIOTLBKey { - uint64_t iova; - uint16_t asid; -} SMMUIOTLBKey; - typedef struct SMMUState { /* */ SysBusDevice dev; @@ -155,6 +150,7 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid); IOMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, hwaddr iova); void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, IOMMUTLBEntry *entry); +uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova); void smmu_iotlb_inv_all(SMMUState *s); void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid); void smmu_iotlb_inv_iova(SMMUState *s, uint16_t asid, dma_addr_t iova); diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 8409de052d..3101043540 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -32,10 +32,25 @@ /* IOTLB Management */ +static guint smmu_iotlb_key_hash(gconstpointer v) +{ + return (guint)*(const uint64_t *)v; +} + +static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2) +{ + return *((const uint64_t *)v1) == *((const uint64_t *)v2); +} + +uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova) +{ + return iova >> 12 | (uint64_t)(asid) << SMMU_IOTLB_ASID_SHIFT; +} + IOMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, hwaddr iova) { - SMMUIOTLBKey key = {.asid = cfg->asid, .iova = iova}; + uint64_t key = smmu_get_iotlb_key(cfg->asid, iova); IOMMUTLBEntry *entry = g_hash_table_lookup(bs->iotlb, &key); if (entry) { @@ -56,14 +71,13 @@ IOMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, IOMMUTLBEntry *entry) { - SMMUIOTLBKey *key = g_new0(SMMUIOTLBKey, 1); + uint64_t *key = g_new0(uint64_t, 1); if (g_hash_table_size(bs->iotlb) >= SMMU_IOTLB_MAX_SIZE) { smmu_iotlb_inv_all(bs); } - key->asid = cfg->asid; - key->iova = entry->iova; + *key = smmu_get_iotlb_key(cfg->asid, entry->iova); trace_smmu_iotlb_insert(cfg->asid, entry->iova); g_hash_table_insert(bs->iotlb, key, entry); } @@ -78,14 +92,14 @@ static gboolean smmu_hash_remove_by_asid(gpointer key, gpointer value, gpointer user_data) { uint16_t asid = *(uint16_t *)user_data; - SMMUIOTLBKey *iotlb_key = (SMMUIOTLBKey *)key; + uint64_t *iotlb_key = (uint64_t *)key; - return iotlb_key->asid == asid; + return SMMU_IOTLB_ASID(*iotlb_key) == asid; } inline void smmu_iotlb_inv_iova(SMMUState *s, uint16_t asid, dma_addr_t iova) { - SMMUIOTLBKey key = {.asid = asid, .iova = iova}; + uint64_t key = smmu_get_iotlb_key(asid, iova); trace_smmu_iotlb_inv_iova(asid, iova); g_hash_table_remove(s->iotlb, &key); @@ -382,31 +396,6 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid) return NULL; } -static guint smmu_iotlb_key_hash(gconstpointer v) -{ - SMMUIOTLBKey *key = (SMMUIOTLBKey *)v; - uint32_t a, b, c; - - /* Jenkins hash */ - a = b = c = JHASH_INITVAL + sizeof(*key); - a += key->asid; - b += extract64(key->iova, 0, 32); - c += extract64(key->iova, 32, 32); - - __jhash_mix(a, b, c); - __jhash_final(a, b, c); - - return c; -} - -static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2) -{ - const SMMUIOTLBKey *k1 = v1; - const SMMUIOTLBKey *k2 = v2; - - return (k1->asid == k2->asid) && (k1->iova == k2->iova); -} - /* Unmap the whole notifier's range */ static void smmu_unmap_notifier_range(IOMMUNotifier *n) { From patchwork Thu May 21 09:10:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 282210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68F1EC433DF for ; Thu, 21 May 2020 09:12:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 272BE2070A for ; Thu, 21 May 2020 09:12:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fRO9Hdn9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 272BE2070A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:37694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jbhFu-0006Av-Ak for qemu-devel@archiver.kernel.org; Thu, 21 May 2020 05:12:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55580) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jbhEy-0004Y3-1V for qemu-devel@nongnu.org; Thu, 21 May 2020 05:11:48 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:34177 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jbhEw-0005L6-N0 for qemu-devel@nongnu.org; Thu, 21 May 2020 05:11:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590052306; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1czAJ1x/IV6ed1bEr1v7sbxUSM/1M7JNSIvpPOu9AnM=; b=fRO9Hdn9wN8cAZm5yWghbmZXS1qijaRC0FR6r2yGjNV0rjTJQtTAWTWi/O9GO7nD4wTxQr OtaPH/1oKKUchZ9wq1fm1HP33tSrmbWmHA6DYbmzyBovlWbsxnzB307SJRbJnuJsb9HSmC 1vPGiH5ijU7KJabUzkLpjUsbBdiZP2E= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-145-vYGYtdXmMUCtSkWSzXc-hQ-1; Thu, 21 May 2020 05:11:44 -0400 X-MC-Unique: vYGYtdXmMUCtSkWSzXc-hQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D9D3218FF660; Thu, 21 May 2020 09:11:42 +0000 (UTC) Received: from laptop.redhat.com (ovpn-113-56.ams2.redhat.com [10.36.113.56]) by smtp.corp.redhat.com (Postfix) with ESMTP id 088C45D9CA; Thu, 21 May 2020 09:11:39 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, peterx@redhat.com Subject: [PATCH 4/9] hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB value Date: Thu, 21 May 2020 11:10:54 +0200 Message-Id: <20200521091059.9453-5-eric.auger@redhat.com> In-Reply-To: <20200521091059.9453-1-eric.auger@redhat.com> References: <20200521091059.9453-1-eric.auger@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/21 04:44:45 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, robh@kernel.org, robin.murphy@arm.com, mst@redhat.com, zhangfei.gao@foxmail.com, shameerali.kolothum.thodi@huawei.com, will@kernel.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Introduce a specialized SMMUTLBEntry to store the result of the PTW and cache in the IOTLB. This structure extends the generic IOMMUTLBEntry struct with the level of the entry and the granule size. Those latter will be useful when implementing range invalidation. Signed-off-by: Eric Auger --- include/hw/arm/smmu-common.h | 14 +++++++++++--- hw/arm/smmu-common.c | 30 ++++++++++++++++-------------- hw/arm/smmuv3.c | 10 +++++----- 3 files changed, 32 insertions(+), 22 deletions(-) diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index 7b9d2f0eb7..f6ee78e16c 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -52,6 +52,14 @@ typedef struct SMMUTransTableInfo { uint8_t granule_sz; /* granule page shift */ } SMMUTransTableInfo; +struct SMMUTLBEntry { + IOMMUTLBEntry entry; + uint8_t level; + uint8_t granule; +}; + +typedef struct SMMUTLBEntry SMMUTLBEntry; + /* * Generic structure populated by derived SMMU devices * after decoding the configuration information and used as @@ -135,7 +143,7 @@ static inline uint16_t smmu_get_sid(SMMUDevice *sdev) * pair, according to @cfg translation config */ int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, - IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info); + SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info); /** * select_tt - compute which translation table shall be used according to @@ -148,8 +156,8 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid); #define SMMU_IOTLB_MAX_SIZE 256 -IOMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, hwaddr iova); -void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, IOMMUTLBEntry *entry); +SMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, hwaddr iova); +void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, SMMUTLBEntry *entry); uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova); void smmu_iotlb_inv_all(SMMUState *s); void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid); diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 3101043540..aa88b62efb 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -47,11 +47,11 @@ uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova) return iova >> 12 | (uint64_t)(asid) << SMMU_IOTLB_ASID_SHIFT; } -IOMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, +SMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, hwaddr iova) { uint64_t key = smmu_get_iotlb_key(cfg->asid, iova); - IOMMUTLBEntry *entry = g_hash_table_lookup(bs->iotlb, &key); + SMMUTLBEntry *entry = g_hash_table_lookup(bs->iotlb, &key); if (entry) { cfg->iotlb_hits++; @@ -69,7 +69,7 @@ IOMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg, return entry; } -void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, IOMMUTLBEntry *entry) +void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, SMMUTLBEntry *new) { uint64_t *key = g_new0(uint64_t, 1); @@ -77,9 +77,9 @@ void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, IOMMUTLBEntry *entry) smmu_iotlb_inv_all(bs); } - *key = smmu_get_iotlb_key(cfg->asid, entry->iova); - trace_smmu_iotlb_insert(cfg->asid, entry->iova); - g_hash_table_insert(bs->iotlb, key, entry); + *key = smmu_get_iotlb_key(cfg->asid, new->entry.iova); + trace_smmu_iotlb_insert(cfg->asid, new->entry.iova); + g_hash_table_insert(bs->iotlb, key, new); } inline void smmu_iotlb_inv_all(SMMUState *s) @@ -199,7 +199,7 @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova) * @cfg: translation config * @iova: iova to translate * @perm: access type - * @tlbe: IOMMUTLBEntry (out) + * @tlbe: SMMUTLBEntry (out) * @info: handle to an error info * * Return 0 on success, < 0 on error. In case of error, @info is filled @@ -209,7 +209,7 @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova) */ static int smmu_ptw_64(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, - IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) + SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) { dma_addr_t baseaddr, indexmask; int stage = cfg->stage; @@ -229,8 +229,8 @@ static int smmu_ptw_64(SMMUTransCfg *cfg, baseaddr = extract64(tt->ttb, 0, 48); baseaddr &= ~indexmask; - tlbe->iova = iova; - tlbe->addr_mask = (1 << granule_sz) - 1; + tlbe->entry.iova = iova; + tlbe->entry.addr_mask = (1 << granule_sz) - 1; while (level <= 3) { uint64_t subpage_size = 1ULL << level_shift(level, granule_sz); @@ -281,13 +281,15 @@ static int smmu_ptw_64(SMMUTransCfg *cfg, goto error; } - tlbe->translated_addr = gpa + (iova & mask); - tlbe->perm = PTE_AP_TO_PERM(ap); + tlbe->entry.translated_addr = gpa + (iova & mask); + tlbe->entry.perm = PTE_AP_TO_PERM(ap); + tlbe->level = level; + tlbe->granule = granule_sz; return 0; } info->type = SMMU_PTW_ERR_TRANSLATION; error: - tlbe->perm = IOMMU_NONE; + tlbe->entry.perm = IOMMU_NONE; return -EINVAL; } @@ -303,7 +305,7 @@ error: * return 0 on success */ inline int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm, - IOMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) + SMMUTLBEntry *tlbe, SMMUPTWEventInfo *info) { if (!cfg->aa64) { /* diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index cd2a2e7e14..db74d27add 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -626,7 +626,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, SMMUTranslationStatus status; SMMUState *bs = ARM_SMMU(s); uint64_t page_mask, aligned_addr; - IOMMUTLBEntry *cached_entry = NULL; + SMMUTLBEntry *cached_entry = NULL; SMMUTransTableInfo *tt; SMMUTransCfg *cfg = NULL; IOMMUTLBEntry entry = { @@ -676,7 +676,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, cached_entry = smmu_iotlb_lookup(bs, cfg, aligned_addr); if (cached_entry) { - if ((flag & IOMMU_WO) && !(cached_entry->perm & IOMMU_WO)) { + if ((flag & IOMMU_WO) && !(cached_entry->entry.perm & IOMMU_WO)) { status = SMMU_TRANS_ERROR; if (event.record_trans_faults) { event.type = SMMU_EVT_F_PERMISSION; @@ -689,7 +689,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, goto epilogue; } - cached_entry = g_new0(IOMMUTLBEntry, 1); + cached_entry = g_new0(SMMUTLBEntry, 1); if (smmu_ptw(cfg, aligned_addr, flag, cached_entry, &ptw_info)) { g_free(cached_entry); @@ -743,9 +743,9 @@ epilogue: switch (status) { case SMMU_TRANS_SUCCESS: entry.perm = flag; - entry.translated_addr = cached_entry->translated_addr + + entry.translated_addr = cached_entry->entry.translated_addr + (addr & page_mask); - entry.addr_mask = cached_entry->addr_mask; + entry.addr_mask = cached_entry->entry.addr_mask; trace_smmuv3_translate_success(mr->parent_obj.name, sid, addr, entry.translated_addr, entry.perm); break; From patchwork Thu May 21 09:10:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 282208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67220C433E0 for ; Thu, 21 May 2020 09:14:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 27CF12070A for ; Thu, 21 May 2020 09:14:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="J31Tm4jB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27CF12070A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45162 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jbhHu-0001fm-Bx for qemu-devel@archiver.kernel.org; Thu, 21 May 2020 05:14:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55650) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jbhFH-00057X-01 for qemu-devel@nongnu.org; Thu, 21 May 2020 05:12:07 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:20437 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jbhFG-0005NK-4G for qemu-devel@nongnu.org; Thu, 21 May 2020 05:12:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590052325; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ARv8qSDziPHTJvaLmyQIW8b8LLLtOfE1f3+HLmQbobU=; b=J31Tm4jBftk0W4ezRL6PArkLk8bTG4Te1Q2EsUcsGoOH7dGYxq3DiG6kNAh3kMxuxtRVV3 lWHh0FQODiUCxG3J0sCbuWXO+1odEwb8Xg5p9jS9JD9VDKYfkeKNuY1cInUKuWI/JxTKd9 H8LQqosTBOBm50hAW/osdQcuUqqD3VE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-505-TQfVxmaFOqaQ8AUEYaPdmg-1; Thu, 21 May 2020 05:12:02 -0400 X-MC-Unique: TQfVxmaFOqaQ8AUEYaPdmg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 57ECF8005AA; Thu, 21 May 2020 09:12:00 +0000 (UTC) Received: from laptop.redhat.com (ovpn-113-56.ams2.redhat.com [10.36.113.56]) by smtp.corp.redhat.com (Postfix) with ESMTP id 47B4A5D9C9; Thu, 21 May 2020 09:11:52 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, peterx@redhat.com Subject: [PATCH 7/9] hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helper Date: Thu, 21 May 2020 11:10:57 +0200 Message-Id: <20200521091059.9453-8-eric.auger@redhat.com> In-Reply-To: <20200521091059.9453-1-eric.auger@redhat.com> References: <20200521091059.9453-1-eric.auger@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/21 04:44:45 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, robh@kernel.org, robin.murphy@arm.com, mst@redhat.com, zhangfei.gao@foxmail.com, shameerali.kolothum.thodi@huawei.com, will@kernel.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Let's introduce an helper for S1 IOVA range invalidation. This will be used for NH_VA and NH_VAA commands. It decodes the same fields, trace, calls the UNMAP notifiers and invalidate the corresponding IOTLB entries. At the moment, we do not support 3.2 range invalidation yet. So it reduces to a single IOVA invalidation. Signed-off-by: Eric Auger --- hw/arm/smmuv3.c | 36 +++++++++++++++++------------------- hw/arm/trace-events | 3 +-- 2 files changed, 18 insertions(+), 21 deletions(-) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index 931a2b6872..4eda16be7b 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -840,6 +840,22 @@ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) } } +static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) +{ + dma_addr_t addr = CMD_ADDR(cmd); + uint8_t type = CMD_TYPE(cmd); + uint16_t vmid = CMD_VMID(cmd); + bool leaf = CMD_LEAF(cmd); + int asid = -1; + + if (type == SMMU_CMD_TLBI_NH_VA) { + asid = CMD_ASID(cmd); + } + trace_smmuv3_s1_range_inval(vmid, asid, addr, leaf); + smmuv3_inv_notifiers_iova(s, asid, addr); + smmu_iotlb_inv_iova(s, asid, addr); +} + static int smmuv3_cmdq_consume(SMMUv3State *s) { SMMUState *bs = ARM_SMMU(s); @@ -970,27 +986,9 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) smmu_iotlb_inv_all(bs); break; case SMMU_CMD_TLBI_NH_VAA: - { - dma_addr_t addr = CMD_ADDR(&cmd); - uint16_t vmid = CMD_VMID(&cmd); - - trace_smmuv3_cmdq_tlbi_nh_vaa(vmid, addr); - smmuv3_inv_notifiers_iova(bs, -1, addr); - smmu_iotlb_inv_iova(bs, -1, addr); - break; - } case SMMU_CMD_TLBI_NH_VA: - { - uint16_t asid = CMD_ASID(&cmd); - uint16_t vmid = CMD_VMID(&cmd); - dma_addr_t addr = CMD_ADDR(&cmd); - bool leaf = CMD_LEAF(&cmd); - - trace_smmuv3_cmdq_tlbi_nh_va(vmid, asid, addr, leaf); - smmuv3_inv_notifiers_iova(bs, asid, addr); - smmu_iotlb_inv_iova(bs, asid, addr); + smmuv3_s1_range_inval(bs, &cmd); break; - } case SMMU_CMD_TLBI_EL3_ALL: case SMMU_CMD_TLBI_EL3_VA: case SMMU_CMD_TLBI_EL2_ALL: diff --git a/hw/arm/trace-events b/hw/arm/trace-events index 2d445c99b7..b366973739 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -45,8 +45,7 @@ smmuv3_cmdq_cfgi_ste_range(int start, int end) "start=0x%d - end=0x%d" smmuv3_cmdq_cfgi_cd(uint32_t sid) "streamid = %d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache HIT for sid %d (hits=%d, misses=%d, hit rate=%d)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache MISS for sid %d (hits=%d, misses=%d, hit rate=%d)" -smmuv3_cmdq_tlbi_nh_va(int vmid, int asid, uint64_t addr, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" leaf=%d" -smmuv3_cmdq_tlbi_nh_vaa(int vmid, uint64_t addr) "vmid =%d addr=0x%"PRIx64 +smmuv3_s1_range_inval(int vmid, int asid, uint64_t addr, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" leaf=%d" smmuv3_cmdq_tlbi_nh(void) "" smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=%d" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d" From patchwork Thu May 21 09:10:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 282207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4C33C433DF for ; Thu, 21 May 2020 09:17:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 81ED520759 for ; Thu, 21 May 2020 09:17:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N6InNOKV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81ED520759 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jbhK8-0005Dy-Lx for qemu-devel@archiver.kernel.org; Thu, 21 May 2020 05:17:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55662) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jbhFK-0005Gv-TZ for qemu-devel@nongnu.org; Thu, 21 May 2020 05:12:10 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:59864 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jbhFJ-0005Nn-NF for qemu-devel@nongnu.org; Thu, 21 May 2020 05:12:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590052329; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=28JeEQFaQKpR65Fvr4h2OzB54px8vEV5aJWYpQDbyYY=; b=N6InNOKV0TJAeeFba1VKdivgnEodze9s1A4WO/c1f7umqTzsMRecN6MTZ0cTnGqwsqSNlZ wm/ttyEayZxWIi6ANmZwy02pRg5B02Mmx1sQ7fJdLxlan0PH6yp6f6lJzhs++g3ybb2PRJ WHCIdaBj0n7j/5CL0BTrlLyQPsKq0D0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-107-SfmX2BWhNwChuhBrSQN2oA-1; Thu, 21 May 2020 05:12:05 -0400 X-MC-Unique: SfmX2BWhNwChuhBrSQN2oA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8B6B480183C; Thu, 21 May 2020 09:12:03 +0000 (UTC) Received: from laptop.redhat.com (ovpn-113-56.ams2.redhat.com [10.36.113.56]) by smtp.corp.redhat.com (Postfix) with ESMTP id AFC345D9C9; Thu, 21 May 2020 09:12:00 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org, peterx@redhat.com Subject: [PATCH 8/9] hw/arm/smmuv3: Get prepared for range invalidation Date: Thu, 21 May 2020 11:10:58 +0200 Message-Id: <20200521091059.9453-9-eric.auger@redhat.com> In-Reply-To: <20200521091059.9453-1-eric.auger@redhat.com> References: <20200521091059.9453-1-eric.auger@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=eric.auger@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/21 01:44:25 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, robh@kernel.org, robin.murphy@arm.com, mst@redhat.com, zhangfei.gao@foxmail.com, shameerali.kolothum.thodi@huawei.com, will@kernel.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Enhance the smmu_iotlb_inv_iova() helper with range invalidation. This uses the new fields passed in the NH_VA and NH_VAA commands: the size of the range, the level and the granule. As NH_VA and NH_VAA both use those fields, their decoding and handling is factorized in a new smmuv3_s1_range_inval() helper. Signed-off-by: Eric Auger --- hw/arm/smmuv3-internal.h | 4 +++ include/hw/arm/smmu-common.h | 3 +- hw/arm/smmu-common.c | 28 ++++++++++++---- hw/arm/smmuv3.c | 64 +++++++++++++++++++++++------------- hw/arm/trace-events | 4 +-- 5 files changed, 71 insertions(+), 32 deletions(-) diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h index 4112394129..5babf72f7d 100644 --- a/hw/arm/smmuv3-internal.h +++ b/hw/arm/smmuv3-internal.h @@ -298,6 +298,8 @@ enum { /* Command completion notification */ }; #define CMD_TYPE(x) extract32((x)->word[0], 0 , 8) +#define CMD_NUM(x) extract32((x)->word[0], 12 , 5) +#define CMD_SCALE(x) extract32((x)->word[0], 20 , 5) #define CMD_SSEC(x) extract32((x)->word[0], 10, 1) #define CMD_SSV(x) extract32((x)->word[0], 11, 1) #define CMD_RESUME_AC(x) extract32((x)->word[0], 12, 1) @@ -310,6 +312,8 @@ enum { /* Command completion notification */ #define CMD_RESUME_STAG(x) extract32((x)->word[2], 0 , 16) #define CMD_RESP(x) extract32((x)->word[2], 11, 2) #define CMD_LEAF(x) extract32((x)->word[2], 0 , 1) +#define CMD_TTL(x) extract32((x)->word[2], 8 , 2) +#define CMD_TG(x) extract32((x)->word[2], 10, 2) #define CMD_STE_RANGE(x) extract32((x)->word[2], 0 , 5) #define CMD_ADDR(x) ({ \ uint64_t high = (uint64_t)(x)->word[3]; \ diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index 34a13c7cd6..6dd638a485 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -164,7 +164,8 @@ uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova, uint8_t tg, uint8_t level); void smmu_iotlb_inv_all(SMMUState *s); void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid); -void smmu_iotlb_inv_iova(SMMUState *s, int asid, dma_addr_t iova); +void smmu_iotlb_inv_iova(SMMUState *s, int asid, dma_addr_t iova, + uint8_t tg, uint64_t num_pages, uint8_t ttl); /* Unmap the range of all the notifiers registered to any IOMMU mr */ void smmu_inv_notifiers_all(SMMUState *s); diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 27804fbb2d..0a86afe69d 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -125,18 +125,34 @@ static gboolean smmu_hash_remove_by_asid_iova(gpointer key, gpointer value, if (info->asid >= 0) { return (info->asid == SMMU_IOTLB_ASID(*iotlb_key)) && - ((info->iova & ~entry->addr_mask) == entry->iova); + (((entry->iova & ~info->mask) == info->iova) || + ((info->iova & ~entry->addr_mask) == entry->iova)); } else { - return (info->iova & ~entry->addr_mask) == entry->iova; + return (((entry->iova & ~info->mask) == info->iova) || + ((info->iova & ~entry->addr_mask) == entry->iova)); } } -inline void smmu_iotlb_inv_iova(SMMUState *s, int asid, dma_addr_t iova) +inline void +smmu_iotlb_inv_iova(SMMUState *s, int asid, dma_addr_t iova, + uint8_t tg, uint64_t num_pages, uint8_t ttl) { - SMMUIOTLBPageInvInfo info = {.asid = asid, .iova = iova}; + if (ttl && (num_pages == 1)) { + uint64_t key = smmu_get_iotlb_key(asid, iova, tg, ttl); - trace_smmu_iotlb_inv_iova(asid, iova); - g_hash_table_foreach_remove(s->iotlb, smmu_hash_remove_by_asid_iova, &info); + g_hash_table_remove(s->iotlb, &key); + } else { + /* if tg is not set we use 4KB range invalidation */ + uint8_t granule = tg ? tg * 2 + 10 : 12; + + SMMUIOTLBPageInvInfo info = { + .asid = asid, .iova = iova, + .mask = (num_pages * 1 << granule) - 1}; + + g_hash_table_foreach_remove(s->iotlb, + smmu_hash_remove_by_asid_iova, + &info); + } } inline void smmu_iotlb_inv_asid(SMMUState *s, uint16_t asid) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index 4eda16be7b..4dff438809 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -789,42 +789,49 @@ epilogue: * @n: notifier to be called * @asid: address space ID or negative value if we don't care * @iova: iova + * @tg: translation granule (if communicated through range invalidation) + * @num_pages: number of @granule sized pages (if tg != 0), otherwise 1 */ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, IOMMUNotifier *n, - int asid, - dma_addr_t iova) + int asid, dma_addr_t iova, + uint8_t tg, uint64_t num_pages) { SMMUDevice *sdev = container_of(mr, SMMUDevice, iommu); - SMMUEventInfo event = {.inval_ste_allowed = true}; - SMMUTransTableInfo *tt; - SMMUTransCfg *cfg; IOMMUTLBEntry entry; + uint8_t granule = tg; - cfg = smmuv3_get_config(sdev, &event); - if (!cfg) { - return; - } + if (!tg) { + SMMUEventInfo event = {.inval_ste_allowed = true}; + SMMUTransCfg *cfg = smmuv3_get_config(sdev, &event); + SMMUTransTableInfo *tt; - if (asid >= 0 && cfg->asid != asid) { - return; - } + if (!cfg) { + return; + } - tt = select_tt(cfg, iova); - if (!tt) { - return; + if (asid >= 0 && cfg->asid != asid) { + return; + } + + tt = select_tt(cfg, iova); + if (!tt) { + return; + } + granule = tt->granule_sz; } entry.target_as = &address_space_memory; entry.iova = iova; - entry.addr_mask = (1 << tt->granule_sz) - 1; + entry.addr_mask = num_pages * (1 << granule) - 1; entry.perm = IOMMU_NONE; memory_region_notify_one(n, &entry); } -/* invalidate an asid/iova tuple in all mr's */ -static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) +/* invalidate an asid/iova range tuple in all mr's */ +static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova, + uint8_t tg, uint64_t num_pages) { SMMUDevice *sdev; @@ -832,28 +839,39 @@ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) IOMMUMemoryRegion *mr = &sdev->iommu; IOMMUNotifier *n; - trace_smmuv3_inv_notifiers_iova(mr->parent_obj.name, asid, iova); + trace_smmuv3_inv_notifiers_iova(mr->parent_obj.name, asid, iova, + tg, num_pages); IOMMU_NOTIFIER_FOREACH(n, mr) { - smmuv3_notify_iova(mr, n, asid, iova); + smmuv3_notify_iova(mr, n, asid, iova, tg, num_pages); } } } static void smmuv3_s1_range_inval(SMMUState *s, Cmd *cmd) { + uint8_t scale = 0, num = 0, ttl = 0; dma_addr_t addr = CMD_ADDR(cmd); uint8_t type = CMD_TYPE(cmd); uint16_t vmid = CMD_VMID(cmd); bool leaf = CMD_LEAF(cmd); + uint8_t tg = CMD_TG(cmd); + hwaddr num_pages = 1; int asid = -1; + if (tg) { + scale = CMD_SCALE(cmd); + num = CMD_NUM(cmd); + ttl = CMD_TTL(cmd); + num_pages = (num + 1) * (1 << (scale)); + } + if (type == SMMU_CMD_TLBI_NH_VA) { asid = CMD_ASID(cmd); } - trace_smmuv3_s1_range_inval(vmid, asid, addr, leaf); - smmuv3_inv_notifiers_iova(s, asid, addr); - smmu_iotlb_inv_iova(s, asid, addr); + trace_smmuv3_s1_range_inval(vmid, asid, addr, tg, num_pages, ttl, leaf); + smmuv3_inv_notifiers_iova(s, asid, addr, tg, num_pages); + smmu_iotlb_inv_iova(s, asid, addr, tg, num_pages, ttl); } static int smmuv3_cmdq_consume(SMMUv3State *s) diff --git a/hw/arm/trace-events b/hw/arm/trace-events index b366973739..c47943f1c7 100644 --- a/hw/arm/trace-events +++ b/hw/arm/trace-events @@ -45,11 +45,11 @@ smmuv3_cmdq_cfgi_ste_range(int start, int end) "start=0x%d - end=0x%d" smmuv3_cmdq_cfgi_cd(uint32_t sid) "streamid = %d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache HIT for sid %d (hits=%d, misses=%d, hit rate=%d)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache MISS for sid %d (hits=%d, misses=%d, hit rate=%d)" -smmuv3_s1_range_inval(int vmid, int asid, uint64_t addr, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" leaf=%d" +smmuv3_s1_range_inval(int vmid, int asid, uint64_t addr, uint8_t tg, uint64_t num_pages, uint8_t ttl, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64" ttl=%d leaf=%d" smmuv3_cmdq_tlbi_nh(void) "" smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=%d" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d" smmuv3_notify_flag_add(const char *iommu) "ADD SMMUNotifier node for iommu mr=%s" smmuv3_notify_flag_del(const char *iommu) "DEL SMMUNotifier node for iommu mr=%s" -smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint64_t iova) "iommu mr=%s asid=%d iova=0x%"PRIx64 +smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint64_t iova, uint8_t tg, uint64_t num_pages) "iommu mr=%s asid=%d iova=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64