From patchwork Fri Jan 13 10:50:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hector Martin X-Patchwork-Id: 642229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37A97C54EBD for ; Fri, 13 Jan 2023 10:53:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241320AbjAMKxu (ORCPT ); Fri, 13 Jan 2023 05:53:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241221AbjAMKxM (ORCPT ); Fri, 13 Jan 2023 05:53:12 -0500 Received: from mail.marcansoft.com (marcansoft.com [212.63.210.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 581E777AC5; Fri, 13 Jan 2023 02:51:38 -0800 (PST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sendonly@marcansoft.com) by mail.marcansoft.com (Postfix) with ESMTPSA id 2D25841A42; Fri, 13 Jan 2023 10:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=marcan.st; s=default; t=1673607096; bh=kZ5l2St5RyPUjKSIYXwu/uDFjCyPWx8nT0BPyLp8JKg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Ae6JuKjU00sx2HlcWmPSHOOZ6+AsJUs2stuywa5lTiHEy/YHVyhuP8cLlNhyfNuDJ dT3X5k/glqXl8aDaTc7iEgvdq3mBUWuTTxrJjvJ0m6ev4ZWmxmuz8KFkN0Yx0nsw+9 8XtaZYbyBtGNF0p4jHFJ6tRhMVlRv36PMWCZ1mDTXZGbdnarK9OQRR6c61JH9fc1Pm WFhv3oJAorH1hQBYSw1Pz6jqdI0Z+T4/gNFmsYCLxPjYyT9gpLvnvIbNfRiJ2+Olq+ HJcQVI4ORIKsrIzTMm2Ksl6Av2ZiazoV589IOT3Y0nMlMuJbZlHhypOuOJK8LXjOGq 4bK0m04MRxKfw== From: Hector Martin To: Joerg Roedel , Will Deacon , Robin Murphy Cc: Sven Peter , Alyssa Rosenzweig , Janne Grunau , Rob Herring , Krzysztof Kozlowski , devicetree@vger.kernel.org, iommu@lists.linux.dev, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Hector Martin Subject: [PATCH 2/7] iommu: dart: Add suspend/resume support Date: Fri, 13 Jan 2023 19:50:25 +0900 Message-Id: <20230113105029.26654-3-marcan@marcan.st> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230113105029.26654-1-marcan@marcan.st> References: <20230113105029.26654-1-marcan@marcan.st> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org We need to save/restore the TCR/TTBR registers, since they are lost on power gate. Reviewed-by: Sven Peter Signed-off-by: Hector Martin --- drivers/iommu/apple-dart.c | 43 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c index 4f4a323be0d0..2458416122f8 100644 --- a/drivers/iommu/apple-dart.c +++ b/drivers/iommu/apple-dart.c @@ -121,6 +121,9 @@ struct apple_dart { struct iommu_group *sid2group[DART_MAX_STREAMS]; struct iommu_device iommu; + + u32 save_tcr[DART_MAX_STREAMS]; + u32 save_ttbr[DART_MAX_STREAMS][DART_MAX_TTBR]; }; /* @@ -932,6 +935,45 @@ static const struct apple_dart_hw apple_dart_hw_t6000 = { .fmt = APPLE_DART2, }; +static __maybe_unused int apple_dart_suspend(struct device *dev) +{ + struct apple_dart *dart = dev_get_drvdata(dev); + unsigned int sid, idx; + + for (sid = 0; sid < DART_MAX_STREAMS; sid++) { + dart->save_tcr[sid] = readl_relaxed(dart->regs + DART_TCR(sid)); + for (idx = 0; idx < DART_MAX_TTBR; idx++) + dart->save_ttbr[sid][idx] = + readl(dart->regs + DART_TTBR(sid, idx)); + } + + return 0; +} + +static __maybe_unused int apple_dart_resume(struct device *dev) +{ + struct apple_dart *dart = dev_get_drvdata(dev); + unsigned int sid, idx; + int ret; + + ret = apple_dart_hw_reset(dart); + if (ret) { + dev_err(dev, "Failed to reset DART on resume\n"); + return ret; + } + + for (sid = 0; sid < DART_MAX_STREAMS; sid++) { + for (idx = 0; idx < DART_MAX_TTBR; idx++) + writel(dart->save_ttbr[sid][idx], + dart->regs + DART_TTBR(sid, idx)); + writel(dart->save_tcr[sid], dart->regs + DART_TCR(sid)); + } + + return 0; +} + +DEFINE_SIMPLE_DEV_PM_OPS(apple_dart_pm_ops, apple_dart_suspend, apple_dart_resume); + static const struct of_device_id apple_dart_of_match[] = { { .compatible = "apple,t8103-dart", .data = &apple_dart_hw_t8103 }, { .compatible = "apple,t6000-dart", .data = &apple_dart_hw_t6000 }, @@ -944,6 +986,7 @@ static struct platform_driver apple_dart_driver = { .name = "apple-dart", .of_match_table = apple_dart_of_match, .suppress_bind_attrs = true, + .pm = pm_sleep_ptr(&apple_dart_pm_ops), }, .probe = apple_dart_probe, .remove = apple_dart_remove, From patchwork Fri Jan 13 10:50:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hector Martin X-Patchwork-Id: 642228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7382C678D6 for ; Fri, 13 Jan 2023 10:53:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241407AbjAMKxv (ORCPT ); Fri, 13 Jan 2023 05:53:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49586 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232730AbjAMKxN (ORCPT ); Fri, 13 Jan 2023 05:53:13 -0500 Received: from mail.marcansoft.com (marcansoft.com [212.63.210.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49143777F1; Fri, 13 Jan 2023 02:51:42 -0800 (PST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sendonly@marcansoft.com) by mail.marcansoft.com (Postfix) with ESMTPSA id 796154206F; Fri, 13 Jan 2023 10:51:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=marcan.st; s=default; t=1673607101; bh=IZPr+adOIY8SXVwOiM5uDY4tUsn/+VInYGYdG3JduHk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=HLqyVzIBqrHnMQ3lNmvKBXGWekk/cyFa0F5x07sbubxn/YG1GCmvsSEnEWi5gQmoZ AJA/fij3tR1W3JRfqDVYj6MoQrsg7l4bMvNKja/pq0p9PWVIcrpAYaxamHmIYrMEsH +22qB6RxBHDDGBbiv1qkH+AunhBF5p7TeSiHGIar5kVa+VvatfR0inNIYwOAp/rhET 9SzFoZDbCMtwkD8G6LePfiopo2bzNYa3b5BTpJ53GmBZQW/QT/UgQ0wRxpxFl1OP7q ALr2vkAxGf4jKewIWA2dZuBAk3Ajxybozh6xkzGO8cXb5xl0IJz5pdQG+FEFdKH1lQ M7/0XZFrKwtqA== From: Hector Martin To: Joerg Roedel , Will Deacon , Robin Murphy Cc: Sven Peter , Alyssa Rosenzweig , Janne Grunau , Rob Herring , Krzysztof Kozlowski , devicetree@vger.kernel.org, iommu@lists.linux.dev, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Hector Martin Subject: [PATCH 3/7] iommu: dart: Support >64 stream IDs Date: Fri, 13 Jan 2023 19:50:26 +0900 Message-Id: <20230113105029.26654-4-marcan@marcan.st> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230113105029.26654-1-marcan@marcan.st> References: <20230113105029.26654-1-marcan@marcan.st> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org T8110 DARTs have up to 256 SIDs, so we need to switch to a bitmap to handle them properly. Reviewed-by: Sven Peter Signed-off-by: Hector Martin --- drivers/iommu/apple-dart.c | 114 +++++++++++++++++++++++-------------- 1 file changed, 71 insertions(+), 43 deletions(-) diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c index 2458416122f8..80e4436ee4de 100644 --- a/drivers/iommu/apple-dart.c +++ b/drivers/iommu/apple-dart.c @@ -34,11 +34,10 @@ #include "dma-iommu.h" -#define DART_MAX_STREAMS 16 +#define DART_MAX_STREAMS 256 #define DART_MAX_TTBR 4 #define MAX_DARTS_PER_DEVICE 2 -#define DART_STREAM_ALL 0xffff #define DART_PARAMS1 0x00 #define DART_PARAMS_PAGE_SHIFT GENMASK(27, 24) @@ -85,6 +84,8 @@ struct apple_dart_hw { u32 oas; enum io_pgtable_fmt fmt; + + int max_sid_count; }; /* @@ -116,6 +117,7 @@ struct apple_dart { spinlock_t lock; u32 pgsize; + u32 num_streams; u32 supports_bypass : 1; u32 force_bypass : 1; @@ -143,11 +145,11 @@ struct apple_dart { */ struct apple_dart_stream_map { struct apple_dart *dart; - unsigned long sidmap; + DECLARE_BITMAP(sidmap, DART_MAX_STREAMS); }; struct apple_dart_atomic_stream_map { struct apple_dart *dart; - atomic64_t sidmap; + atomic_long_t sidmap[BITS_TO_LONGS(DART_MAX_STREAMS)]; }; /* @@ -205,50 +207,55 @@ static struct apple_dart_domain *to_dart_domain(struct iommu_domain *dom) static void apple_dart_hw_enable_translation(struct apple_dart_stream_map *stream_map) { + struct apple_dart *dart = stream_map->dart; int sid; - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) + for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) writel(DART_TCR_TRANSLATE_ENABLE, - stream_map->dart->regs + DART_TCR(sid)); + dart->regs + DART_TCR(sid)); } static void apple_dart_hw_disable_dma(struct apple_dart_stream_map *stream_map) { + struct apple_dart *dart = stream_map->dart; int sid; - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) - writel(0, stream_map->dart->regs + DART_TCR(sid)); + for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) + writel(0, dart->regs + DART_TCR(sid)); } static void apple_dart_hw_enable_bypass(struct apple_dart_stream_map *stream_map) { + struct apple_dart *dart = stream_map->dart; int sid; WARN_ON(!stream_map->dart->supports_bypass); - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) + for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) writel(DART_TCR_BYPASS0_ENABLE | DART_TCR_BYPASS1_ENABLE, - stream_map->dart->regs + DART_TCR(sid)); + dart->regs + DART_TCR(sid)); } static void apple_dart_hw_set_ttbr(struct apple_dart_stream_map *stream_map, u8 idx, phys_addr_t paddr) { + struct apple_dart *dart = stream_map->dart; int sid; WARN_ON(paddr & ((1 << DART_TTBR_SHIFT) - 1)); - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) + for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) writel(DART_TTBR_VALID | (paddr >> DART_TTBR_SHIFT), - stream_map->dart->regs + DART_TTBR(sid, idx)); + dart->regs + DART_TTBR(sid, idx)); } static void apple_dart_hw_clear_ttbr(struct apple_dart_stream_map *stream_map, u8 idx) { + struct apple_dart *dart = stream_map->dart; int sid; - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) - writel(0, stream_map->dart->regs + DART_TTBR(sid, idx)); + for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) + writel(0, dart->regs + DART_TTBR(sid, idx)); } static void @@ -270,7 +277,7 @@ apple_dart_hw_stream_command(struct apple_dart_stream_map *stream_map, spin_lock_irqsave(&stream_map->dart->lock, flags); - writel(stream_map->sidmap, stream_map->dart->regs + DART_STREAM_SELECT); + writel(stream_map->sidmap[0], stream_map->dart->regs + DART_STREAM_SELECT); writel(command, stream_map->dart->regs + DART_STREAM_COMMAND); ret = readl_poll_timeout_atomic( @@ -283,7 +290,7 @@ apple_dart_hw_stream_command(struct apple_dart_stream_map *stream_map, if (ret) { dev_err(stream_map->dart->dev, "busy bit did not clear after command %x for streams %lx\n", - command, stream_map->sidmap); + command, stream_map->sidmap[0]); return ret; } @@ -301,6 +308,7 @@ static int apple_dart_hw_reset(struct apple_dart *dart) { u32 config; struct apple_dart_stream_map stream_map; + int i; config = readl(dart->regs + DART_CONFIG); if (config & DART_CONFIG_LOCK) { @@ -310,12 +318,14 @@ static int apple_dart_hw_reset(struct apple_dart *dart) } stream_map.dart = dart; - stream_map.sidmap = DART_STREAM_ALL; + bitmap_zero(stream_map.sidmap, DART_MAX_STREAMS); + bitmap_set(stream_map.sidmap, 0, dart->num_streams); apple_dart_hw_disable_dma(&stream_map); apple_dart_hw_clear_all_ttbrs(&stream_map); /* enable all streams globally since TCR is used to control isolation */ - writel(DART_STREAM_ALL, dart->regs + DART_STREAMS_ENABLE); + for (i = 0; i < BITS_TO_U32(dart->num_streams); i++) + writel(U32_MAX, dart->regs + DART_STREAMS_ENABLE + 4 * i); /* clear any pending errors before the interrupt is unmasked */ writel(readl(dart->regs + DART_ERROR), dart->regs + DART_ERROR); @@ -325,13 +335,16 @@ static int apple_dart_hw_reset(struct apple_dart *dart) static void apple_dart_domain_flush_tlb(struct apple_dart_domain *domain) { - int i; + int i, j; struct apple_dart_atomic_stream_map *domain_stream_map; struct apple_dart_stream_map stream_map; for_each_stream_map(i, domain, domain_stream_map) { stream_map.dart = domain_stream_map->dart; - stream_map.sidmap = atomic64_read(&domain_stream_map->sidmap); + + for (j = 0; j < BITS_TO_LONGS(stream_map.dart->num_streams); j++) + stream_map.sidmap[j] = atomic_long_read(&domain_stream_map->sidmap[j]); + apple_dart_hw_invalidate_tlb(&stream_map); } } @@ -416,7 +429,7 @@ static int apple_dart_finalize_domain(struct iommu_domain *domain, struct apple_dart *dart = cfg->stream_maps[0].dart; struct io_pgtable_cfg pgtbl_cfg; int ret = 0; - int i; + int i, j; mutex_lock(&dart_domain->init_lock); @@ -425,8 +438,9 @@ static int apple_dart_finalize_domain(struct iommu_domain *domain, for (i = 0; i < MAX_DARTS_PER_DEVICE; ++i) { dart_domain->stream_maps[i].dart = cfg->stream_maps[i].dart; - atomic64_set(&dart_domain->stream_maps[i].sidmap, - cfg->stream_maps[i].sidmap); + for (j = 0; j < BITS_TO_LONGS(dart->num_streams); j++) + atomic_long_set(&dart_domain->stream_maps[i].sidmap[j], + cfg->stream_maps[i].sidmap[j]); } pgtbl_cfg = (struct io_pgtable_cfg){ @@ -461,7 +475,7 @@ apple_dart_mod_streams(struct apple_dart_atomic_stream_map *domain_maps, struct apple_dart_stream_map *master_maps, bool add_streams) { - int i; + int i, j; for (i = 0; i < MAX_DARTS_PER_DEVICE; ++i) { if (domain_maps[i].dart != master_maps[i].dart) @@ -471,12 +485,14 @@ apple_dart_mod_streams(struct apple_dart_atomic_stream_map *domain_maps, for (i = 0; i < MAX_DARTS_PER_DEVICE; ++i) { if (!domain_maps[i].dart) break; - if (add_streams) - atomic64_or(master_maps[i].sidmap, - &domain_maps[i].sidmap); - else - atomic64_and(~master_maps[i].sidmap, - &domain_maps[i].sidmap); + for (j = 0; j < BITS_TO_LONGS(domain_maps[i].dart->num_streams); j++) { + if (add_streams) + atomic_long_or(master_maps[i].sidmap[j], + &domain_maps[i].sidmap[j]); + else + atomic_long_and(~master_maps[i].sidmap[j], + &domain_maps[i].sidmap[j]); + } } return 0; @@ -640,14 +656,14 @@ static int apple_dart_of_xlate(struct device *dev, struct of_phandle_args *args) for (i = 0; i < MAX_DARTS_PER_DEVICE; ++i) { if (cfg->stream_maps[i].dart == dart) { - cfg->stream_maps[i].sidmap |= 1 << sid; + set_bit(sid, cfg->stream_maps[i].sidmap); return 0; } } for (i = 0; i < MAX_DARTS_PER_DEVICE; ++i) { if (!cfg->stream_maps[i].dart) { cfg->stream_maps[i].dart = dart; - cfg->stream_maps[i].sidmap = 1 << sid; + set_bit(sid, cfg->stream_maps[i].sidmap); return 0; } } @@ -666,7 +682,7 @@ static void apple_dart_release_group(void *iommu_data) mutex_lock(&apple_dart_groups_lock); for_each_stream_map(i, group_master_cfg, stream_map) - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) + for_each_set_bit(sid, stream_map->sidmap, stream_map->dart->num_streams) stream_map->dart->sid2group[sid] = NULL; kfree(iommu_data); @@ -685,7 +701,7 @@ static struct iommu_group *apple_dart_device_group(struct device *dev) mutex_lock(&apple_dart_groups_lock); for_each_stream_map(i, cfg, stream_map) { - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) { + for_each_set_bit(sid, stream_map->sidmap, stream_map->dart->num_streams) { struct iommu_group *stream_group = stream_map->dart->sid2group[sid]; @@ -724,7 +740,7 @@ static struct iommu_group *apple_dart_device_group(struct device *dev) apple_dart_release_group); for_each_stream_map(i, cfg, stream_map) - for_each_set_bit(sid, &stream_map->sidmap, DART_MAX_STREAMS) + for_each_set_bit(sid, stream_map->sidmap, stream_map->dart->num_streams) stream_map->dart->sid2group[sid] = group; res = group; @@ -869,16 +885,26 @@ static int apple_dart_probe(struct platform_device *pdev) if (ret) return ret; - ret = apple_dart_hw_reset(dart); - if (ret) - goto err_clk_disable; - dart_params[0] = readl(dart->regs + DART_PARAMS1); dart_params[1] = readl(dart->regs + DART_PARAMS2); dart->pgsize = 1 << FIELD_GET(DART_PARAMS_PAGE_SHIFT, dart_params[0]); dart->supports_bypass = dart_params[1] & DART_PARAMS_BYPASS_SUPPORT; + + dart->num_streams = dart->hw->max_sid_count; + + if (dart->num_streams > DART_MAX_STREAMS) { + dev_err(&pdev->dev, "Too many streams (%d > %d)\n", + dart->num_streams, DART_MAX_STREAMS); + ret = -EINVAL; + goto err_clk_disable; + } + dart->force_bypass = dart->pgsize > PAGE_SIZE; + ret = apple_dart_hw_reset(dart); + if (ret) + goto err_clk_disable; + ret = request_irq(dart->irq, apple_dart_irq, IRQF_SHARED, "apple-dart fault handler", dart); if (ret) @@ -897,8 +923,8 @@ static int apple_dart_probe(struct platform_device *pdev) dev_info( &pdev->dev, - "DART [pagesize %x, bypass support: %d, bypass forced: %d] initialized\n", - dart->pgsize, dart->supports_bypass, dart->force_bypass); + "DART [pagesize %x, %d streams, bypass support: %d, bypass forced: %d] initialized\n", + dart->pgsize, dart->num_streams, dart->supports_bypass, dart->force_bypass); return 0; err_sysfs_remove: @@ -929,10 +955,12 @@ static int apple_dart_remove(struct platform_device *pdev) static const struct apple_dart_hw apple_dart_hw_t8103 = { .oas = 36, .fmt = APPLE_DART, + .max_sid_count = 16, }; static const struct apple_dart_hw apple_dart_hw_t6000 = { .oas = 42, .fmt = APPLE_DART2, + .max_sid_count = 16, }; static __maybe_unused int apple_dart_suspend(struct device *dev) @@ -940,7 +968,7 @@ static __maybe_unused int apple_dart_suspend(struct device *dev) struct apple_dart *dart = dev_get_drvdata(dev); unsigned int sid, idx; - for (sid = 0; sid < DART_MAX_STREAMS; sid++) { + for (sid = 0; sid < dart->num_streams; sid++) { dart->save_tcr[sid] = readl_relaxed(dart->regs + DART_TCR(sid)); for (idx = 0; idx < DART_MAX_TTBR; idx++) dart->save_ttbr[sid][idx] = @@ -962,7 +990,7 @@ static __maybe_unused int apple_dart_resume(struct device *dev) return ret; } - for (sid = 0; sid < DART_MAX_STREAMS; sid++) { + for (sid = 0; sid < dart->num_streams; sid++) { for (idx = 0; idx < DART_MAX_TTBR; idx++) writel(dart->save_ttbr[sid][idx], dart->regs + DART_TTBR(sid, idx)); From patchwork Fri Jan 13 10:50:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hector Martin X-Patchwork-Id: 642227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DA94C54EBE for ; Fri, 13 Jan 2023 10:53:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241425AbjAMKxx (ORCPT ); Fri, 13 Jan 2023 05:53:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241041AbjAMKxP (ORCPT ); Fri, 13 Jan 2023 05:53:15 -0500 Received: from mail.marcansoft.com (marcansoft.com [212.63.210.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31DC46D51E; Fri, 13 Jan 2023 02:51:47 -0800 (PST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sendonly@marcansoft.com) by mail.marcansoft.com (Postfix) with ESMTPSA id 0AA4642462; Fri, 13 Jan 2023 10:51:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=marcan.st; s=default; t=1673607105; bh=/0eh/wjENX+WrhMO2/l0+bCl+2z/YSpUY/j6eHDCLAI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=HVq8saZ5Ct7yAHJduh9FouRpjc1i84U08YqHnLnFAAGlcTiYEkL01f6bpKKUD/rPf gSU5xqU941Vf7XIqFlgU5jHb9fNDWC8Ikbux8BS5Or/RKRPyYatGH79ZLDM04Diwp+ uWv6GbrHthVfOlLuT+lilbBXvH1LilHZNEtumdYVTewJ6tOw7ZcsjXcd+gA/PlG43a aboaFRFX0HxqijVLnGq9TqdSWAlkyS71sAQsSqXkk+kmigYc4RZfNKMby6xEyrThm2 Bm6+s2aFh6Tnik8y1N+eIeqX5LBJNOrPKvIhx5sufib7JI3dqjU8GFLnObWc79Sye8 XFKerl341dgeQ== From: Hector Martin To: Joerg Roedel , Will Deacon , Robin Murphy Cc: Sven Peter , Alyssa Rosenzweig , Janne Grunau , Rob Herring , Krzysztof Kozlowski , devicetree@vger.kernel.org, iommu@lists.linux.dev, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Hector Martin Subject: [PATCH 4/7] iommu: dart: Support a variable number of TTBRs per stream Date: Fri, 13 Jan 2023 19:50:27 +0900 Message-Id: <20230113105029.26654-5-marcan@marcan.st> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230113105029.26654-1-marcan@marcan.st> References: <20230113105029.26654-1-marcan@marcan.st> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org T8110 only has one TTBR per stream, so un-hardcode that. Reviewed-by: Sven Peter Signed-off-by: Hector Martin --- drivers/iommu/apple-dart.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c index 80e4436ee4de..82b2581d909c 100644 --- a/drivers/iommu/apple-dart.c +++ b/drivers/iommu/apple-dart.c @@ -77,15 +77,21 @@ #define DART_TCR_BYPASS0_ENABLE BIT(8) #define DART_TCR_BYPASS1_ENABLE BIT(12) -#define DART_TTBR(sid, idx) (0x200 + 16 * (sid) + 4 * (idx)) #define DART_TTBR_VALID BIT(31) #define DART_TTBR_SHIFT 12 +#define DART_TTBR(dart, sid, idx) (0x200 + \ + (((dart)->hw->ttbr_count * (sid)) << 2) + \ + ((idx) << 2)) + + struct apple_dart_hw { u32 oas; enum io_pgtable_fmt fmt; int max_sid_count; + + int ttbr_count; }; /* @@ -245,7 +251,7 @@ static void apple_dart_hw_set_ttbr(struct apple_dart_stream_map *stream_map, WARN_ON(paddr & ((1 << DART_TTBR_SHIFT) - 1)); for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) writel(DART_TTBR_VALID | (paddr >> DART_TTBR_SHIFT), - dart->regs + DART_TTBR(sid, idx)); + dart->regs + DART_TTBR(dart, sid, idx)); } static void apple_dart_hw_clear_ttbr(struct apple_dart_stream_map *stream_map, @@ -255,7 +261,7 @@ static void apple_dart_hw_clear_ttbr(struct apple_dart_stream_map *stream_map, int sid; for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) - writel(0, dart->regs + DART_TTBR(sid, idx)); + writel(0, dart->regs + DART_TTBR(dart, sid, idx)); } static void @@ -263,7 +269,7 @@ apple_dart_hw_clear_all_ttbrs(struct apple_dart_stream_map *stream_map) { int i; - for (i = 0; i < DART_MAX_TTBR; ++i) + for (i = 0; i < stream_map->dart->hw->ttbr_count; ++i) apple_dart_hw_clear_ttbr(stream_map, i); } @@ -415,7 +421,7 @@ apple_dart_setup_translation(struct apple_dart_domain *domain, for (i = 0; i < pgtbl_cfg->apple_dart_cfg.n_ttbrs; ++i) apple_dart_hw_set_ttbr(stream_map, i, pgtbl_cfg->apple_dart_cfg.ttbr[i]); - for (; i < DART_MAX_TTBR; ++i) + for (; i < stream_map->dart->hw->ttbr_count; ++i) apple_dart_hw_clear_ttbr(stream_map, i); apple_dart_hw_enable_translation(stream_map); @@ -956,11 +962,15 @@ static const struct apple_dart_hw apple_dart_hw_t8103 = { .oas = 36, .fmt = APPLE_DART, .max_sid_count = 16, + + .ttbr_count = 4, }; static const struct apple_dart_hw apple_dart_hw_t6000 = { .oas = 42, .fmt = APPLE_DART2, .max_sid_count = 16, + + .ttbr_count = 4, }; static __maybe_unused int apple_dart_suspend(struct device *dev) @@ -970,9 +980,9 @@ static __maybe_unused int apple_dart_suspend(struct device *dev) for (sid = 0; sid < dart->num_streams; sid++) { dart->save_tcr[sid] = readl_relaxed(dart->regs + DART_TCR(sid)); - for (idx = 0; idx < DART_MAX_TTBR; idx++) + for (idx = 0; idx < dart->hw->ttbr_count; idx++) dart->save_ttbr[sid][idx] = - readl(dart->regs + DART_TTBR(sid, idx)); + readl(dart->regs + DART_TTBR(dart, sid, idx)); } return 0; @@ -991,9 +1001,9 @@ static __maybe_unused int apple_dart_resume(struct device *dev) } for (sid = 0; sid < dart->num_streams; sid++) { - for (idx = 0; idx < DART_MAX_TTBR; idx++) + for (idx = 0; idx < dart->hw->ttbr_count; idx++) writel(dart->save_ttbr[sid][idx], - dart->regs + DART_TTBR(sid, idx)); + dart->regs + DART_TTBR(dart, sid, idx)); writel(dart->save_tcr[sid], dart->regs + DART_TCR(sid)); } From patchwork Fri Jan 13 10:50:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hector Martin X-Patchwork-Id: 642226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B59D9C61DB3 for ; Fri, 13 Jan 2023 10:54:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241167AbjAMKym (ORCPT ); Fri, 13 Jan 2023 05:54:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231218AbjAMKx1 (ORCPT ); Fri, 13 Jan 2023 05:53:27 -0500 Received: from mail.marcansoft.com (marcansoft.com [212.63.210.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B4A977AF4; Fri, 13 Jan 2023 02:51:51 -0800 (PST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sendonly@marcansoft.com) by mail.marcansoft.com (Postfix) with ESMTPSA id 578D44206F; Fri, 13 Jan 2023 10:51:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=marcan.st; s=default; t=1673607110; bh=XN7W4wHaV4CYylxXNUod7fpaKbnCRKcr/LnEs1SZri8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=lm1OI8ZmxoHebhJKLPlRDpInHCgUpIecT+3XN2nnu+fLs+th+5YQYk9IqXLrQdImt K0fOEhs0+OpDPLNaB0AjR/iMUL2GYphZ6BY5NQt8gWnza+U7iOaCA/QMufkefDwgkV CCjpMh8GUSJhKBWwpkD56lcslFmfDjyyeasxUEBpqfKnAin/8boa4Y5kPGdN5OTZJy 7bAmCAz4hO3eFWLZmAcvlp37pw65DlnDQcUcAh13Quc+Me2ojJhVKwW/acErloBMlS DrEuMagqhKOCtaOl1oc8+EWkUrjeSbyZhxcBPlh1zC6icxZSj+pAYlXtpffMV/lg0D H/9ywWdQHcTiA== From: Hector Martin To: Joerg Roedel , Will Deacon , Robin Murphy Cc: Sven Peter , Alyssa Rosenzweig , Janne Grunau , Rob Herring , Krzysztof Kozlowski , devicetree@vger.kernel.org, iommu@lists.linux.dev, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Hector Martin Subject: [PATCH 5/7] iommu: dart: Fix DART_PARAMS1/2 bit define names Date: Fri, 13 Jan 2023 19:50:28 +0900 Message-Id: <20230113105029.26654-6-marcan@marcan.st> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230113105029.26654-1-marcan@marcan.st> References: <20230113105029.26654-1-marcan@marcan.st> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org They didn't have the PARAMS reg index in them, but they should. Reviewed-by: Sven Peter Signed-off-by: Hector Martin --- drivers/iommu/apple-dart.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c index 82b2581d909c..89be055a44be 100644 --- a/drivers/iommu/apple-dart.c +++ b/drivers/iommu/apple-dart.c @@ -40,10 +40,10 @@ #define DART_PARAMS1 0x00 -#define DART_PARAMS_PAGE_SHIFT GENMASK(27, 24) +#define DART_PARAMS1_PAGE_SHIFT GENMASK(27, 24) #define DART_PARAMS2 0x04 -#define DART_PARAMS_BYPASS_SUPPORT BIT(0) +#define DART_PARAMS2_BYPASS_SUPPORT BIT(0) #define DART_STREAM_COMMAND 0x20 #define DART_STREAM_COMMAND_BUSY BIT(2) @@ -893,8 +893,8 @@ static int apple_dart_probe(struct platform_device *pdev) dart_params[0] = readl(dart->regs + DART_PARAMS1); dart_params[1] = readl(dart->regs + DART_PARAMS2); - dart->pgsize = 1 << FIELD_GET(DART_PARAMS_PAGE_SHIFT, dart_params[0]); - dart->supports_bypass = dart_params[1] & DART_PARAMS_BYPASS_SUPPORT; + dart->pgsize = 1 << FIELD_GET(DART_PARAMS1_PAGE_SHIFT, dart_params[0]); + dart->supports_bypass = dart_params[1] & DART_PARAMS2_BYPASS_SUPPORT; dart->num_streams = dart->hw->max_sid_count; From patchwork Fri Jan 13 10:50:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hector Martin X-Patchwork-Id: 642225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD514C54EBE for ; Fri, 13 Jan 2023 10:55:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241516AbjAMKzg (ORCPT ); Fri, 13 Jan 2023 05:55:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241200AbjAMKxs (ORCPT ); Fri, 13 Jan 2023 05:53:48 -0500 Received: from mail.marcansoft.com (marcansoft.com [212.63.210.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 218277816D; Fri, 13 Jan 2023 02:52:00 -0800 (PST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sendonly@marcansoft.com) by mail.marcansoft.com (Postfix) with ESMTPSA id 3FB7F42462; Fri, 13 Jan 2023 10:51:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=marcan.st; s=default; t=1673607119; bh=mG3NRAZAV+UBx4Xvm5DWiRmT3F55WtTracCeUatlMoA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=c/hHoA3GDnC7lbvESA1dL5QcOWTHSAbkJtFnHcYLeDiTRcFuvUSf7avULLfByXHnC B/eXxT9gDNc81XNtopHFv0tYyZRkWf1BxVExcBdgvpgMrf4WOOsiO2AX5ndrGAX2qb cz0BN6IAbkP0FTTJXo79fz0jNTMrkMvD+74U1yOBLbX+q6UmDnxYF/f1FkY+yjq9rp e9y/pAmkzt36jvtaS3H2T631wPS8gpZQVUTADrxI8GCQrYcAT/tvVWytbldCHEm+E8 KcKE5zQawf+M3IW7nOxSWRaT5gz6ALnJWcF5hidXqAXuXc+Q6pc9BfuF2RAj42uySp WO0MLEZblTHBA== From: Hector Martin To: Joerg Roedel , Will Deacon , Robin Murphy Cc: Sven Peter , Alyssa Rosenzweig , Janne Grunau , Rob Herring , Krzysztof Kozlowski , devicetree@vger.kernel.org, iommu@lists.linux.dev, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Hector Martin Subject: [PATCH 7/7] iommu: dart: Add t8110 DART support Date: Fri, 13 Jan 2023 19:50:30 +0900 Message-Id: <20230113105029.26654-8-marcan@marcan.st> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230113105029.26654-1-marcan@marcan.st> References: <20230113105029.26654-1-marcan@marcan.st> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Now that we have the driver properly parameterized, we can add support for T8110 DARTs. These DARTs drop the multiple TTBRs (which only make sense with legacy 4K page platforms) and instead add support for new features and more stream IDs. The register layout is different, but the pagetable format is the same as T6000. Reviewed-by: Sven Peter Signed-off-by: Hector Martin --- drivers/iommu/apple-dart.c | 205 ++++++++++++++++++++++++++++++++++++- 1 file changed, 200 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c index 75647433df9d..42666617803d 100644 --- a/drivers/iommu/apple-dart.c +++ b/drivers/iommu/apple-dart.c @@ -85,6 +85,62 @@ #define DART_T8020_TTBR_ADDR_FIELD_SHIFT 0 #define DART_T8020_TTBR_SHIFT 12 +/* T8110 registers */ + +#define DART_T8110_PARAMS3 0x08 +#define DART_T8110_PARAMS3_PA_WIDTH GENMASK(29, 24) +#define DART_T8110_PARAMS3_VA_WIDTH GENMASK(21, 16) +#define DART_T8110_PARAMS3_VER_MAJ GENMASK(15, 8) +#define DART_T8110_PARAMS3_VER_MIN GENMASK(7, 0) + +#define DART_T8110_PARAMS4 0x0c +#define DART_T8110_PARAMS4_NUM_CLIENTS GENMASK(24, 16) +#define DART_T8110_PARAMS4_NUM_SIDS GENMASK(8, 0) + +#define DART_T8110_TLB_CMD 0x80 +#define DART_T8110_TLB_CMD_BUSY BIT(31) +#define DART_T8110_TLB_CMD_OP GENMASK(10, 8) +#define DART_T8110_TLB_CMD_OP_FLUSH_ALL 0 +#define DART_T8110_TLB_CMD_OP_FLUSH_SID 1 +#define DART_T8110_TLB_CMD_STREAM GENMASK(7, 0) + +#define DART_T8110_ERROR 0x100 +#define DART_T8110_ERROR_STREAM GENMASK(27, 20) +#define DART_T8110_ERROR_CODE GENMASK(14, 0) +#define DART_T8110_ERROR_FLAG BIT(31) + +#define DART_T8110_ERROR_MASK 0x104 + +#define DART_T8110_ERROR_READ_FAULT BIT(4) +#define DART_T8110_ERROR_WRITE_FAULT BIT(3) +#define DART_T8110_ERROR_NO_PTE BIT(3) +#define DART_T8110_ERROR_NO_PMD BIT(2) +#define DART_T8110_ERROR_NO_PGD BIT(1) +#define DART_T8110_ERROR_NO_TTBR BIT(0) + +#define DART_T8110_ERROR_ADDR_LO 0x170 +#define DART_T8110_ERROR_ADDR_HI 0x174 + +#define DART_T8110_PROTECT 0x200 +#define DART_T8110_UNPROTECT 0x204 +#define DART_T8110_PROTECT_LOCK 0x208 +#define DART_T8110_PROTECT_TTBR_TCR BIT(0) + +#define DART_T8110_ENABLE_STREAMS 0xc00 +#define DART_T8110_DISABLE_STREAMS 0xc20 + +#define DART_T8110_TCR 0x1000 +#define DART_T8110_TCR_REMAP GENMASK(11, 8) +#define DART_T8110_TCR_REMAP_EN BIT(7) +#define DART_T8110_TCR_BYPASS_DAPF BIT(2) +#define DART_T8110_TCR_BYPASS_DART BIT(1) +#define DART_T8110_TCR_TRANSLATE_ENABLE BIT(0) + +#define DART_T8110_TTBR 0x1400 +#define DART_T8110_TTBR_VALID BIT(0) +#define DART_T8110_TTBR_ADDR_FIELD_SHIFT 2 +#define DART_T8110_TTBR_SHIFT 14 + #define DART_TCR(dart, sid) ((dart)->hw->tcr + ((sid) << 2)) #define DART_TTBR(dart, sid, idx) ((dart)->hw->ttbr + \ @@ -93,7 +149,14 @@ struct apple_dart_stream_map; +enum dart_type { + DART_T8020, + DART_T6000, + DART_T8110, +}; + struct apple_dart_hw { + enum dart_type type; irqreturn_t (*irq_handler)(int irq, void *dev); int (*invalidate_tlb)(struct apple_dart_stream_map *stream_map); @@ -149,6 +212,8 @@ struct apple_dart { spinlock_t lock; + u32 ias; + u32 oas; u32 pgsize; u32 num_streams; u32 supports_bypass : 1; @@ -330,6 +395,44 @@ apple_dart_t8020_hw_stream_command(struct apple_dart_stream_map *stream_map, return 0; } +static int +apple_dart_t8110_hw_tlb_command(struct apple_dart_stream_map *stream_map, + u32 command) +{ + struct apple_dart *dart = stream_map->dart; + unsigned long flags; + int ret = 0; + int sid; + + spin_lock_irqsave(&dart->lock, flags); + + for_each_set_bit(sid, stream_map->sidmap, dart->num_streams) { + u32 val = FIELD_PREP(DART_T8110_TLB_CMD_OP, command) | + FIELD_PREP(DART_T8110_TLB_CMD_STREAM, sid); + writel(val, dart->regs + DART_T8110_TLB_CMD); + + ret = readl_poll_timeout_atomic( + dart->regs + DART_T8110_TLB_CMD, val, + !(val & DART_T8110_TLB_CMD_BUSY), 1, + DART_STREAM_COMMAND_BUSY_TIMEOUT); + + if (ret) + break; + + } + + spin_unlock_irqrestore(&dart->lock, flags); + + if (ret) { + dev_err(stream_map->dart->dev, + "busy bit did not clear after command %x for stream %d\n", + command, sid); + return ret; + } + + return 0; +} + static int apple_dart_t8020_hw_invalidate_tlb(struct apple_dart_stream_map *stream_map) { @@ -337,6 +440,13 @@ apple_dart_t8020_hw_invalidate_tlb(struct apple_dart_stream_map *stream_map) stream_map, DART_T8020_STREAM_COMMAND_INVALIDATE); } +static int +apple_dart_t8110_hw_invalidate_tlb(struct apple_dart_stream_map *stream_map) +{ + return apple_dart_t8110_hw_tlb_command( + stream_map, DART_T8110_TLB_CMD_OP_FLUSH_SID); +} + static int apple_dart_hw_reset(struct apple_dart *dart) { u32 config; @@ -363,6 +473,9 @@ static int apple_dart_hw_reset(struct apple_dart *dart) /* clear any pending errors before the interrupt is unmasked */ writel(readl(dart->regs + dart->hw->error), dart->regs + dart->hw->error); + if (dart->hw->type == DART_T8110) + writel(0, dart->regs + DART_T8110_ERROR_MASK); + return dart->hw->invalidate_tlb(&stream_map); } @@ -478,8 +591,8 @@ static int apple_dart_finalize_domain(struct iommu_domain *domain, pgtbl_cfg = (struct io_pgtable_cfg){ .pgsize_bitmap = dart->pgsize, - .ias = 32, - .oas = dart->hw->oas, + .ias = dart->ias, + .oas = dart->oas, .coherent_walk = 1, .iommu_dev = dart->dev, }; @@ -493,7 +606,7 @@ static int apple_dart_finalize_domain(struct iommu_domain *domain, domain->pgsize_bitmap = pgtbl_cfg.pgsize_bitmap; domain->geometry.aperture_start = 0; - domain->geometry.aperture_end = DMA_BIT_MASK(32); + domain->geometry.aperture_end = (dma_addr_t)DMA_BIT_MASK(dart->ias); domain->geometry.force_aperture = true; dart_domain->finalized = true; @@ -880,10 +993,49 @@ static irqreturn_t apple_dart_t8020_irq(int irq, void *dev) return IRQ_HANDLED; } +static irqreturn_t apple_dart_t8110_irq(int irq, void *dev) +{ + struct apple_dart *dart = dev; + const char *fault_name = NULL; + u32 error = readl(dart->regs + DART_T8110_ERROR); + u32 error_code = FIELD_GET(DART_T8110_ERROR_CODE, error); + u32 addr_lo = readl(dart->regs + DART_T8110_ERROR_ADDR_LO); + u32 addr_hi = readl(dart->regs + DART_T8110_ERROR_ADDR_HI); + u64 addr = addr_lo | (((u64)addr_hi) << 32); + u8 stream_idx = FIELD_GET(DART_T8110_ERROR_STREAM, error); + + if (!(error & DART_T8110_ERROR_FLAG)) + return IRQ_NONE; + + /* there should only be a single bit set but let's use == to be sure */ + if (error_code == DART_T8110_ERROR_READ_FAULT) + fault_name = "READ FAULT"; + else if (error_code == DART_T8110_ERROR_WRITE_FAULT) + fault_name = "WRITE FAULT"; + else if (error_code == DART_T8110_ERROR_NO_PTE) + fault_name = "NO PTE FOR IOVA"; + else if (error_code == DART_T8110_ERROR_NO_PMD) + fault_name = "NO PMD FOR IOVA"; + else if (error_code == DART_T8110_ERROR_NO_PGD) + fault_name = "NO PGD FOR IOVA"; + else if (error_code == DART_T8110_ERROR_NO_TTBR) + fault_name = "NO TTBR FOR IOVA"; + else + fault_name = "unknown"; + + dev_err_ratelimited( + dart->dev, + "translation fault: status:0x%x stream:%d code:0x%x (%s) at 0x%llx", + error, stream_idx, error_code, fault_name, addr); + + writel(error, dart->regs + DART_T8110_ERROR); + return IRQ_HANDLED; +} + static int apple_dart_probe(struct platform_device *pdev) { int ret; - u32 dart_params[2]; + u32 dart_params[4]; struct resource *res; struct apple_dart *dart; struct device *dev = &pdev->dev; @@ -923,7 +1075,22 @@ static int apple_dart_probe(struct platform_device *pdev) dart->pgsize = 1 << FIELD_GET(DART_PARAMS1_PAGE_SHIFT, dart_params[0]); dart->supports_bypass = dart_params[1] & DART_PARAMS2_BYPASS_SUPPORT; - dart->num_streams = dart->hw->max_sid_count; + switch (dart->hw->type) { + case DART_T8020: + case DART_T6000: + dart->ias = 32; + dart->oas = dart->hw->oas; + dart->num_streams = dart->hw->max_sid_count; + break; + + case DART_T8110: + dart_params[2] = readl(dart->regs + DART_T8110_PARAMS3); + dart_params[3] = readl(dart->regs + DART_T8110_PARAMS4); + dart->ias = FIELD_GET(DART_T8110_PARAMS3_VA_WIDTH, dart_params[2]); + dart->oas = FIELD_GET(DART_T8110_PARAMS3_PA_WIDTH, dart_params[2]); + dart->num_streams = FIELD_GET(DART_T8110_PARAMS4_NUM_SIDS, dart_params[3]); + break; + } if (dart->num_streams > DART_MAX_STREAMS) { dev_err(&pdev->dev, "Too many streams (%d > %d)\n", @@ -986,6 +1153,7 @@ static int apple_dart_remove(struct platform_device *pdev) } static const struct apple_dart_hw apple_dart_hw_t8103 = { + .type = DART_T8020, .irq_handler = apple_dart_t8020_irq, .invalidate_tlb = apple_dart_t8020_hw_invalidate_tlb, .oas = 36, @@ -1010,6 +1178,7 @@ static const struct apple_dart_hw apple_dart_hw_t8103 = { .ttbr_count = 4, }; static const struct apple_dart_hw apple_dart_hw_t6000 = { + .type = DART_T6000, .irq_handler = apple_dart_t8020_irq, .invalidate_tlb = apple_dart_t8020_hw_invalidate_tlb, .oas = 42, @@ -1034,6 +1203,31 @@ static const struct apple_dart_hw apple_dart_hw_t6000 = { .ttbr_count = 4, }; +static const struct apple_dart_hw apple_dart_hw_t8110 = { + .type = DART_T8110, + .irq_handler = apple_dart_t8110_irq, + .invalidate_tlb = apple_dart_t8110_hw_invalidate_tlb, + .fmt = APPLE_DART2, + .max_sid_count = 256, + + .enable_streams = DART_T8110_ENABLE_STREAMS, + .lock = DART_T8110_PROTECT, + .lock_bit = DART_T8110_PROTECT_TTBR_TCR, + + .error = DART_T8110_ERROR, + + .tcr = DART_T8110_TCR, + .tcr_enabled = DART_T8110_TCR_TRANSLATE_ENABLE, + .tcr_disabled = 0, + .tcr_bypass = DART_T8110_TCR_BYPASS_DAPF | DART_T8110_TCR_BYPASS_DART, + + .ttbr = DART_T8110_TTBR, + .ttbr_valid = DART_T8110_TTBR_VALID, + .ttbr_addr_field_shift = DART_T8110_TTBR_ADDR_FIELD_SHIFT, + .ttbr_shift = DART_T8110_TTBR_SHIFT, + .ttbr_count = 1, +}; + static __maybe_unused int apple_dart_suspend(struct device *dev) { struct apple_dart *dart = dev_get_drvdata(dev); @@ -1075,6 +1269,7 @@ DEFINE_SIMPLE_DEV_PM_OPS(apple_dart_pm_ops, apple_dart_suspend, apple_dart_resum static const struct of_device_id apple_dart_of_match[] = { { .compatible = "apple,t8103-dart", .data = &apple_dart_hw_t8103 }, + { .compatible = "apple,t8110-dart", .data = &apple_dart_hw_t8110 }, { .compatible = "apple,t6000-dart", .data = &apple_dart_hw_t6000 }, {}, };