From patchwork Wed Feb 25 18:52:59 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 45099 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f198.google.com (mail-lb0-f198.google.com [209.85.217.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 041C120502 for ; Wed, 25 Feb 2015 18:55:09 +0000 (UTC) Received: by lbdu10 with SMTP id u10sf4402441lbd.3 for ; Wed, 25 Feb 2015 10:55:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:cc:subject:precedence:list-id:list-unsubscribe:list-post :list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:sender:errors-to:x-original-sender :x-original-authentication-results:mailing-list:list-archive; bh=+fTPpnnMPLR633F7++AGmTtViWUxR8hHb+VXhoYI2D0=; b=BM7JDIcjTcgZ1OtASLflZ4AyKhLZtphzpKbN6X1NyxQLFloZpIldm8MkUTR9BVsjuN qJPCw/6KVqpyLu8INGjAKpWnu2dVNFRt2QrnVaPcE7nJ88DPfb3g0gKR0+DtK7iW+BuZ Nweb22Am1CFGu6U99gvGacnkg0CW5Tq13h3svkS2+yBSABFxvh5AfzraSpnL08xeJ3DN VI3xpJty92ZdhFBawQ/kR454xu9PFg/SqwtvXPfzlYgz4nXCAUxS34FYNQKJJDzTAYZd H9+E/nMHeTikwWWijMx+RPvqL/74VnBwhwgUEcTe2z++oGNgpWOfKSjnqcjgr7ksnlHP d//Q== X-Gm-Message-State: ALoCoQl0q/IVOvc/3v6ucK5inn/qj7ZXnxefER04SKEyYu/hVz8Ht8AYnfBt9RF7PJ+s+D0c7N7/ X-Received: by 10.112.159.227 with SMTP id xf3mr734461lbb.11.1424890507909; Wed, 25 Feb 2015 10:55:07 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.115.180 with SMTP id jp20ls735214lab.64.gmail; Wed, 25 Feb 2015 10:55:07 -0800 (PST) X-Received: by 10.112.146.163 with SMTP id td3mr4051241lbb.119.1424890507756; Wed, 25 Feb 2015 10:55:07 -0800 (PST) Received: from mail-lb0-f180.google.com (mail-lb0-f180.google.com. [209.85.217.180]) by mx.google.com with ESMTPS id kw10si83368lac.50.2015.02.25.10.55.07 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Feb 2015 10:55:07 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.180 as permitted sender) client-ip=209.85.217.180; Received: by lbjb6 with SMTP id b6so5967293lbj.2 for ; Wed, 25 Feb 2015 10:55:07 -0800 (PST) X-Received: by 10.152.28.5 with SMTP id x5mr4111660lag.112.1424890507163; Wed, 25 Feb 2015 10:55:07 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp2819640lbj; Wed, 25 Feb 2015 10:55:05 -0800 (PST) X-Received: by 10.52.11.73 with SMTP id o9mr5282504vdb.70.1424890504654; Wed, 25 Feb 2015 10:55:04 -0800 (PST) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id w4si2747901vdr.28.2015.02.25.10.55.03 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 25 Feb 2015 10:55:04 -0800 (PST) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YQh5e-0005uP-PM; Wed, 25 Feb 2015 18:53:46 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YQh5c-0005sx-Ew for xen-devel@lists.xenproject.org; Wed, 25 Feb 2015 18:53:44 +0000 Received: from [85.158.137.68] by server-9.bemta-3.messagelabs.com id 89/9B-02995-73A1EE45; Wed, 25 Feb 2015 18:53:43 +0000 X-Env-Sender: julien.grall@linaro.org X-Msg-Ref: server-10.tower-31.messagelabs.com!1424890421!10964635!1 X-Originating-IP: [74.125.82.169] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 6.13.4; banners=-,-,- X-VirusChecked: Checked Received: (qmail 23906 invoked from network); 25 Feb 2015 18:53:42 -0000 Received: from mail-we0-f169.google.com (HELO mail-we0-f169.google.com) (74.125.82.169) by server-10.tower-31.messagelabs.com with RC4-SHA encrypted SMTP; 25 Feb 2015 18:53:42 -0000 Received: by wevk48 with SMTP id k48so5561234wev.3 for ; Wed, 25 Feb 2015 10:53:41 -0800 (PST) X-Received: by 10.194.60.203 with SMTP id j11mr9208150wjr.5.1424890421738; Wed, 25 Feb 2015 10:53:41 -0800 (PST) Received: from chilopoda.uk.xensource.com. ([185.25.64.249]) by mx.google.com with ESMTPSA id kr5sm61071839wjc.1.2015.02.25.10.53.39 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 25 Feb 2015 10:53:40 -0800 (PST) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Wed, 25 Feb 2015 18:52:59 +0000 Message-Id: <1424890381-4225-7-git-send-email-julien.grall@linaro.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1424890381-4225-1-git-send-email-julien.grall@linaro.org> References: <1424890381-4225-1-git-send-email-julien.grall@linaro.org> Cc: stefano.stabellini@citrix.com, manish.jaggi@caviumnetworks.com, Julien Grall , tim@xen.org, ian.campbell@citrix.com Subject: [Xen-devel] [PATCH v4 6/8] xen/iommu: smmu: Add Xen specific code to be able to use the driver X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: julien.grall@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.180 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: The main goal is to modify as little the Linux code to be able to port easily new feature added in Linux repo for the driver. To achieve that we: - Add helpers to Linux function not implemented on Xen - Add callbacks used by Xen to do our own stuff and call Linux ones - Only modify when required the code which comes from Linux. If so a comment has been added with /* Xen: ... */ explaining why it's necessary. The support for PCI has been commented because it's not yet supported by Xen ARM and therefore won't compile. Signed-off-by: Julien Grall --- Changes in v4: - Re-shuffle the code to rationalize the number of lines changed in the Linux SMMU code - Use #if 0 /* Xen: ... */ rather than using 2 lines - Add a bunch of definitions to avoid some #if 0 - Remove the "hack" for the Midway SMMU Changes in v2: - Add the ACCESS_ONCE definition in the drivers. The patch to introduce the one in common code has been dropped. - The include xen/device.h has been dropped in favor of asm/device.h --- xen/drivers/passthrough/arm/Makefile | 1 + xen/drivers/passthrough/arm/smmu.c | 661 ++++++++++++++++++++++++++++++++--- 2 files changed, 617 insertions(+), 45 deletions(-) diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile index 0484b79..f4cd26e 100644 --- a/xen/drivers/passthrough/arm/Makefile +++ b/xen/drivers/passthrough/arm/Makefile @@ -1 +1,2 @@ obj-y += iommu.o +obj-y += smmu.o diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index 6cd47b7..d01a26a 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -18,6 +18,13 @@ * * Author: Will Deacon * + * Based on Linux drivers/iommu/arm-smmu.c + * => commit e6b5be2be4e30037eb551e0ed09dd97bd00d85d3 + * + * Xen modification: + * Julien Grall + * Copyright (C) 2014 Linaro Limited. + * * This driver currently supports: * - SMMUv1 and v2 implementations * - Stream-matching and stream-indexing @@ -28,25 +35,287 @@ * - Context fault reporting */ -#define pr_fmt(fmt) "arm-smmu: " fmt -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Xen: The below defines are redefined within the file. Undef it */ +#undef SCTLR_AFE +#undef SCTLR_TRE +#undef SCTLR_M +#undef TTBCR_EAE + +/* Alias to Xen device tree helpers */ +#define device_node dt_device_node +#define of_phandle_args dt_phandle_args +#define of_device_id dt_device_match +#define of_match_node dt_match_node +#define of_property_read_u32(np, pname, out) (!dt_property_read_u32(np, pname, out)) +#define of_property_read_bool dt_property_read_bool +#define of_parse_phandle_with_args dt_parse_phandle_with_args + +/* Xen: Helpers to get device MMIO and IRQs */ +struct resource +{ + u64 addr; + u64 size; + unsigned int type; +}; + +#define resource_size(res) (res)->size; + +#define platform_device dt_device_node + +#define IORESOURCE_MEM 0 +#define IORESOURCE_IRQ 1 + +static struct resource *platform_get_resource(struct platform_device *pdev, + unsigned int type, + unsigned int num) +{ + /* + * The resource is only used between 2 calls of platform_get_resource. + * It's quite ugly but it's avoid to add too much code in the part + * imported from Linux + */ + static struct resource res; + int ret = 0; + + res.type = type; + + switch (type) { + case IORESOURCE_MEM: + ret = dt_device_get_address(pdev, num, &res.addr, &res.size); + + return ((ret) ? NULL : &res); + + case IORESOURCE_IRQ: + ret = platform_get_irq(pdev, num); + if (ret < 0) + return NULL; + + res.addr = ret; + res.size = 1; + + return &res; + + default: + return NULL; + } +} + +/* Xen: Helpers for IRQ functions */ +#define request_irq(irq, func, flags, name, dev) request_irq(irq, flags, func, name, dev) +#define free_irq release_irq + +enum irqreturn { + IRQ_NONE = (0 << 0), + IRQ_HANDLED = (1 << 0), +}; + +typedef enum irqreturn irqreturn_t; + +/* Device logger functions + * TODO: Handle PCI + */ +#define dev_print(dev, lvl, fmt, ...) \ + printk(lvl "smmu: %s: " fmt, dt_node_full_name(dev_to_dt(dev)), ## __VA_ARGS__) + +#define dev_dbg(dev, fmt, ...) dev_print(dev, XENLOG_DEBUG, fmt, ## __VA_ARGS__) +#define dev_notice(dev, fmt, ...) dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) +#define dev_warn(dev, fmt, ...) dev_print(dev, XENLOG_WARNING, fmt, ## __VA_ARGS__) +#define dev_err(dev, fmt, ...) dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__) + +#define dev_err_ratelimited(dev, fmt, ...) \ + dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__) + +#define dev_name(dev) dt_node_full_name(dev_to_dt(dev)) + +/* Alias to Xen allocation helpers */ +#define kfree xfree +#define kmalloc(size, flags) _xmalloc(size, sizeof(void *)) +#define kzalloc(size, flags) _xzalloc(size, sizeof(void *)) +#define devm_kzalloc(dev, size, flags) _xzalloc(size, sizeof(void *)) +#define kmalloc_array(size, n, flags) _xmalloc_array(size, sizeof(void *), n) + +static void __iomem *devm_ioremap_resource(struct device *dev, + struct resource *res) +{ + void __iomem *ptr; + + if (!res || res->type != IORESOURCE_MEM) { + dev_err(dev, "Invalid resource\n"); + return ERR_PTR(-EINVAL); + } + + ptr = ioremap_nocache(res->addr, res->size); + if (!ptr) { + dev_err(dev, + "ioremap failed (addr 0x%"PRIx64" size 0x%"PRIx64")\n", + res->addr, res->size); + return ERR_PTR(-ENOMEM); + } + + return ptr; +} + +/* Xen doesn't handle IOMMU fault */ +#define report_iommu_fault(...) 1 + +#define IOMMU_FAULT_READ 0 +#define IOMMU_FAULT_WRITE 1 + +/* + * Xen: PCI functions + * TODO: It should be implemented when PCI will be supported + */ +#define to_pci_dev(dev) (NULL) +static inline int pci_for_each_dma_alias(struct pci_dev *pdev, + int (*fn) (struct pci_dev *pdev, + u16 alias, void *data), + void *data) +{ + BUG(); + return 0; +} + +/* Xen: misc */ +#define PHYS_MASK_SHIFT PADDR_BITS +typedef paddr_t phys_addr_t; + +#ifdef CONFIG_ARM_64 +# define CONFIG_64BIT +#endif + +#define VA_BITS 0 /* Only used for configuring stage-1 input size */ + +/* The macro ACCESS_ONCE start to be replaced in Linux in favor of + * {READ, WRITE}_ONCE. Rather than introducing in the common code, keep a + * version here. We will have to drop it when the SMMU code in Linux will + * switch to {READ, WRITE}_ONCE. + */ +#define __ACCESS_ONCE(x) ({ \ + __maybe_unused typeof(x) __var = 0; \ + (volatile typeof(x) *)&(x); }) +#define ACCESS_ONCE(x) (*__ACCESS_ONCE(x)) + +#define MODULE_DEVICE_TABLE(type, name) +#define module_param_named(name, value, type, perm) +#define MODULE_PARM_DESC(_parm, desc) + +/* Xen: Dummy iommu_domain */ +struct iommu_domain +{ + /* Runtime SMMU configuration for this iommu_domain */ + struct arm_smmu_domain *priv; + + /* Used to link iommu_domain contexts for a same domain. + * There is at least one per-SMMU to used by the domain. + * */ + struct list_head list; +}; + +/* Xen: Describes informations required for a Xen domain */ +struct arm_smmu_xen_domain { + spinlock_t lock; + /* List of context (i.e iommu_domain) associated to this domain */ + struct list_head contexts; +}; + +/* + * Xen: Information about each device stored in dev->archdata.iommu + * + * Initially dev->archdata.iommu only stores the iommu_domain (runtime + * configuration of the SMMU) but, on Xen, we also have to store the + * iommu_group (list of streamIDs associated to the device). + * + * This is because Linux has a field iommu_group in the struct device. On Xen, + * that would require to move so hackery (dummy iommu_group) in a more generic + * place. + * */ +struct arm_smmu_xen_device { + struct iommu_domain *domain; + struct iommu_group *group; +}; + +#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->archdata.iommu) +#define dev_iommu_domain(dev) (dev_archdata(dev)->domain) +#define dev_iommu_group(dev) (dev_archdata(dev)->group) + +/* Xen: Dummy iommu_group */ +struct iommu_group +{ + /* Streamids of the device */ + struct arm_smmu_master_cfg *cfg; + + atomic_t ref; +}; + +static struct iommu_group *iommu_group_alloc(void) +{ + struct iommu_group *group = xzalloc(struct iommu_group); + + if (!group) + return ERR_PTR(-ENOMEM); -#include + atomic_set(&group->ref, 1); -#include + return group; +} + +static void iommu_group_put(struct iommu_group *group) +{ + if (atomic_dec_and_test(&group->ref)) + xfree(group); +} + +static void iommu_group_set_iommudata(struct iommu_group *group, + struct arm_smmu_master_cfg *cfg, + void (*releasefn)(void *)) +{ + /* TODO: Store the releasefn for the PCI */ + ASSERT(releasefn == NULL); + + group->cfg = cfg; +} + +static int iommu_group_add_device(struct iommu_group *group, + struct device *dev) +{ + dev_iommu_group(dev) = group; + + atomic_inc(&group->ref); + + return 0; +} + +static struct iommu_group *iommu_group_get(struct device *dev) +{ + struct iommu_group *group = dev_iommu_group(dev); + + if (group) + atomic_inc(&group->ref); + + return group; +} + +#define iommu_group_get_iommudata(group) (group)->cfg + +/***** Start of Linux SMMU code *****/ /* Maximum number of stream IDs assigned to a single device */ #define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS @@ -397,7 +666,9 @@ struct arm_smmu_cfg { u8 cbndx; u8 irptndx; u32 cbar; - pgd_t *pgd; + + /* Xen: Domain associated to this configuration */ + struct domain *domain; }; #define INVALID_IRPTNDX 0xff @@ -446,6 +717,7 @@ static void parse_driver_options(struct arm_smmu_device *smmu) static struct device_node *dev_get_dev_node(struct device *dev) { +#if 0 /* Xen: TODO: Add support for PCI */ if (dev_is_pci(dev)) { struct pci_bus *bus = to_pci_dev(dev)->bus; @@ -453,6 +725,7 @@ static struct device_node *dev_get_dev_node(struct device *dev) bus = bus->parent; return bus->bridge->parent->of_node; } +#endif return dev->of_node; } @@ -546,6 +819,9 @@ static int register_smmu_master(struct arm_smmu_device *smmu, master->of_node = masterspec->np; master->cfg.num_streamids = masterspec->args_count; + /* Xen: Let Xen knows that the device is protected by an SMMU */ + dt_device_set_protected(masterspec->np); + for (i = 0; i < master->cfg.num_streamids; ++i) { u16 streamid = masterspec->args[i]; @@ -712,6 +988,24 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev) return IRQ_HANDLED; } +/* Xen: Interrupt handlers wrapper */ +static void arm_smmu_context_fault_xen(int irq, void *dev, + struct cpu_user_regs *regs) +{ + arm_smmu_context_fault(irq, dev); +} + +#define arm_smmu_context_fault arm_smmu_context_fault_xen + +static void arm_smmu_global_fault_xen(int irq, void *dev, + struct cpu_user_regs *regs) +{ + arm_smmu_global_fault(irq, dev); +} + +#define arm_smmu_global_fault arm_smmu_global_fault_xen + +#if 0 /* Xen: Page tables are shared with the processor */ static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr, size_t size) { @@ -733,6 +1027,7 @@ static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr, DMA_TO_DEVICE); } } +#endif static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain) { @@ -741,6 +1036,7 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain) struct arm_smmu_cfg *cfg = &smmu_domain->cfg; struct arm_smmu_device *smmu = smmu_domain->smmu; void __iomem *cb_base, *gr0_base, *gr1_base; + paddr_t p2maddr; gr0_base = ARM_SMMU_GR0(smmu); gr1_base = ARM_SMMU_GR1(smmu); @@ -824,11 +1120,16 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain) } /* TTBR0 */ - arm_smmu_flush_pgtable(smmu, cfg->pgd, - PTRS_PER_PGD * sizeof(pgd_t)); - reg = __pa(cfg->pgd); + /* Xen: The page table is shared with the P2M code */ + ASSERT(smmu_domain->cfg.domain != NULL); + p2maddr = page_to_maddr(smmu_domain->cfg.domain->arch.p2m.root); + + dev_notice(smmu->dev, "d%u: p2maddr 0x%"PRIpaddr"\n", + smmu_domain->cfg.domain->domain_id, p2maddr); + + reg = (p2maddr & ((1ULL << 32) - 1)); writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO); - reg = (phys_addr_t)__pa(cfg->pgd) >> 32; + reg = (p2maddr >> 32); if (stage1) reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT; writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI); @@ -873,6 +1174,9 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain) reg = 0; } + /* Xen: The attributes to walk the page table should be the same as + * VTCR_EL2. Currently doesn't differ from Linux ones. + */ reg |= TTBCR_EAE | (TTBCR_SH_IS << TTBCR_SH0_SHIFT) | (TTBCR_RGN_WBWA << TTBCR_ORGN0_SHIFT) | @@ -1015,7 +1319,6 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain) static int arm_smmu_domain_init(struct iommu_domain *domain) { struct arm_smmu_domain *smmu_domain; - pgd_t *pgd; /* * Allocate the domain and initialise some of its data structures. @@ -1026,20 +1329,12 @@ static int arm_smmu_domain_init(struct iommu_domain *domain) if (!smmu_domain) return -ENOMEM; - pgd = kcalloc(PTRS_PER_PGD, sizeof(pgd_t), GFP_KERNEL); - if (!pgd) - goto out_free_domain; - smmu_domain->cfg.pgd = pgd; - spin_lock_init(&smmu_domain->lock); domain->priv = smmu_domain; return 0; - -out_free_domain: - kfree(smmu_domain); - return -ENOMEM; } +#if 0 /* Xen: Page tables are shared with the processor */ static void arm_smmu_free_ptes(pmd_t *pmd) { pgtable_t table = pmd_pgtable(*pmd); @@ -1102,6 +1397,7 @@ static void arm_smmu_free_pgtables(struct arm_smmu_domain *smmu_domain) kfree(pgd_base); } +#endif static void arm_smmu_domain_destroy(struct iommu_domain *domain) { @@ -1112,7 +1408,6 @@ static void arm_smmu_domain_destroy(struct iommu_domain *domain) * already been detached. */ arm_smmu_destroy_domain_context(domain); - arm_smmu_free_pgtables(smmu_domain); kfree(smmu_domain); } @@ -1229,11 +1524,12 @@ static void arm_smmu_domain_remove_master(struct arm_smmu_domain *smmu_domain, /* * We *must* clear the S2CR first, because freeing the SMR means * that it can be re-allocated immediately. + * Xen: Unlike Linux, any access to non-configured stream will fault. */ for (i = 0; i < cfg->num_streamids; ++i) { u32 idx = cfg->smrs ? cfg->smrs[i].idx : cfg->streamids[i]; - writel_relaxed(S2CR_TYPE_BYPASS, + writel_relaxed(S2CR_TYPE_FAULT, gr0_base + ARM_SMMU_GR0_S2CR(idx)); } @@ -1253,7 +1549,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -ENXIO; } - if (dev->archdata.iommu) { + if (dev_iommu_domain(dev)) { dev_err(dev, "already attached to IOMMU domain\n"); return -EEXIST; } @@ -1285,8 +1581,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -ENODEV; ret = arm_smmu_domain_add_master(smmu_domain, cfg); + if (!ret) - dev->archdata.iommu = domain; + dev_iommu_domain(dev) = domain; return ret; } @@ -1299,10 +1596,14 @@ static void arm_smmu_detach_dev(struct iommu_domain *domain, struct device *dev) if (!cfg) return; - dev->archdata.iommu = NULL; + dev_iommu_domain(dev) = NULL; arm_smmu_domain_remove_master(smmu_domain, cfg); } +#if 0 /* + * Xen: The page table is shared with the processor, therefore + * helpers to implement separate is not necessary. + */ static bool arm_smmu_pte_is_contiguous_range(unsigned long addr, unsigned long end) { @@ -1591,7 +1892,9 @@ static phys_addr_t arm_smmu_iova_to_phys(struct iommu_domain *domain, return __pfn_to_phys(pte_pfn(pte)) | (iova & ~PAGE_MASK); } +#endif +#if 0 /* Xen: arm_smmu_capable is not used at the moment */ static bool arm_smmu_capable(enum iommu_cap cap) { switch (cap) { @@ -1609,6 +1912,7 @@ static bool arm_smmu_capable(enum iommu_cap cap) return false; } } +#endif static int __arm_smmu_get_pci_sid(struct pci_dev *pdev, u16 alias, void *data) { @@ -1676,6 +1980,7 @@ out_put_group: return ret; } +#if 0 /* Xen: We don't support remove device for now. Will be useful for PCI */ static void arm_smmu_remove_device(struct device *dev) { iommu_group_remove_device(dev); @@ -1733,6 +2038,7 @@ static const struct iommu_ops arm_smmu_ops = { ARM_SMMU_PTE_CONT_SIZE | PAGE_SIZE), }; +#endif static void arm_smmu_device_reset(struct arm_smmu_device *smmu) { @@ -1748,7 +2054,11 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu) /* Mark all SMRn as invalid and all S2CRn as bypass */ for (i = 0; i < smmu->num_mapping_groups; ++i) { writel_relaxed(0, gr0_base + ARM_SMMU_GR0_SMR(i)); - writel_relaxed(S2CR_TYPE_BYPASS, + /* + * Xen: Unlike Linux, any access to a non-configure stream + * will fault by default. + */ + writel_relaxed(S2CR_TYPE_FAULT, gr0_base + ARM_SMMU_GR0_S2CR(i)); } @@ -1774,6 +2084,8 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu) /* Enable client access, but bypass when no mapping is found */ reg &= ~(sCR0_CLIENTPD | sCR0_USFCFG); + /* Xen: Unlike Linux, generate a fault when no mapping is found */ + reg |= sCR0_USFCFG; /* Disable forced broadcasting */ reg &= ~sCR0_FB; @@ -1882,7 +2194,7 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) } dev_notice(smmu->dev, - "\tstream matching with %u register groups, mask 0x%x", + "\tstream matching with %u register groups, mask 0x%x\n", smmu->num_mapping_groups, mask); } else { smmu->num_mapping_groups = (id >> ID0_NUMSIDB_SHIFT) & @@ -1917,12 +2229,16 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) size = arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) & ID2_IAS_MASK); smmu->s1_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, size); + /* Xen: Stage-2 input size is not restricted */ + smmu->s2_input_size = size; +#if 0 /* Stage-2 input size limited due to pgd allocation (PTRS_PER_PGD) */ #ifdef CONFIG_64BIT smmu->s2_input_size = min_t(unsigned long, VA_BITS, size); #else smmu->s2_input_size = min(32UL, size); #endif +#endif /* The stage-2 output mask is also applied for bypass */ size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) & ID2_OAS_MASK); @@ -1969,6 +2285,10 @@ static const struct of_device_id arm_smmu_of_match[] = { }; MODULE_DEVICE_TABLE(of, arm_smmu_of_match); +/* + * Xen: We don't have refcount for allocated memory so manually free memory + * when an error occured. + */ static int arm_smmu_device_dt_probe(struct platform_device *pdev) { const struct of_device_id *of_id; @@ -1991,14 +2311,17 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev) res = platform_get_resource(pdev, IORESOURCE_MEM, 0); smmu->base = devm_ioremap_resource(dev, res); - if (IS_ERR(smmu->base)) - return PTR_ERR(smmu->base); + if (IS_ERR(smmu->base)) { + err = PTR_ERR(smmu->base); + goto out_free; + } smmu->size = resource_size(res); if (of_property_read_u32(dev->of_node, "#global-interrupts", &smmu->num_global_irqs)) { dev_err(dev, "missing #global-interrupts property\n"); - return -ENODEV; + err = -ENODEV; + goto out_free; } num_irqs = 0; @@ -2011,14 +2334,16 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev) if (!smmu->num_context_irqs) { dev_err(dev, "found %d interrupts but expected at least %d\n", num_irqs, smmu->num_global_irqs + 1); - return -ENODEV; + err = -ENODEV; + goto out_free; } smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs, GFP_KERNEL); if (!smmu->irqs) { dev_err(dev, "failed to allocate %d irqs\n", num_irqs); - return -ENOMEM; + err = -ENOMEM; + goto out_free; } for (i = 0; i < num_irqs; ++i) { @@ -2026,7 +2351,8 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev) if (irq < 0) { dev_err(dev, "failed to get irq index %d\n", i); - return -ENODEV; + err = -ENODEV; + goto out_free; } smmu->irqs[i] = irq; } @@ -2091,12 +2417,19 @@ out_put_masters: for (node = rb_first(&smmu->masters); node; node = rb_next(node)) { struct arm_smmu_master *master = container_of(node, struct arm_smmu_master, node); - of_node_put(master->of_node); + kfree(master); } +out_free: + kfree(smmu->irqs); + if (!IS_ERR(smmu->base)) + iounmap(smmu->base); + kfree(smmu); + return err; } +#if 0 /* Xen: We never remove SMMU */ static int arm_smmu_device_remove(struct platform_device *pdev) { int i; @@ -2191,3 +2524,241 @@ module_exit(arm_smmu_exit); MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations"); MODULE_AUTHOR("Will Deacon "); MODULE_LICENSE("GPL v2"); +#endif + +/***** Start of Xen specific code *****/ + +/* Xen only supports stage-2 translation, so force the value to 2. */ +static int force_stage = 2; + +static void arm_smmu_iotlb_flush_all(struct domain *d) +{ + struct arm_smmu_xen_domain *smmu_domain = domain_hvm_iommu(d)->arch.priv; + struct iommu_domain *cfg; + + spin_lock(&smmu_domain->lock); + list_for_each_entry(cfg, &smmu_domain->contexts, list) { + /* + * Only invalidate the context when SMMU is present. + * This is because the context initialization is delayed + * until a master has been added. + */ + if (unlikely(!ACCESS_ONCE(cfg->priv->smmu))) + continue; + arm_smmu_tlb_inv_context(cfg->priv); + } + spin_unlock(&smmu_domain->lock); +} + +static void arm_smmu_iotlb_flush(struct domain *d, unsigned long gfn, + unsigned int page_count) +{ + /* ARM SMMU v1 doesn't have flush by VMA and VMID */ + arm_smmu_iotlb_flush_all(d); +} + +static int arm_smmu_assign_dev(struct domain *d, u8 devfn, + struct device *dev) +{ + struct iommu_domain *domain; + struct arm_smmu_xen_domain *xen_domain; + int ret; + + xen_domain = domain_hvm_iommu(d)->arch.priv; + + if (!dev->archdata.iommu) { + dev->archdata.iommu = xzalloc(struct arm_smmu_xen_device); + if (!dev->archdata.iommu) + return -ENOMEM; + } + + if (!dev_iommu_group(dev)) { + ret = arm_smmu_add_device(dev); + if (ret) + return ret; + } + + /* + * TODO: Share the context bank (i.e iommu_domain) when the device is + * under the same SMMU as another device assigned to this domain. + * Would it useful for PCI + */ + domain = xzalloc(struct iommu_domain); + if (!domain) + return -ENOMEM; + + ret = arm_smmu_domain_init(domain); + if (ret) + goto err_dom_init; + + domain->priv->cfg.domain = d; + + ret = arm_smmu_attach_dev(domain, dev); + if (ret) + goto err_attach_dev; + + spin_lock(&xen_domain->lock); + /* Chain the new context to the domain */ + list_add(&domain->list, &xen_domain->contexts); + spin_unlock(&xen_domain->lock); + + return 0; + +err_attach_dev: + arm_smmu_domain_destroy(domain); +err_dom_init: + xfree(domain); + + return ret; +} + +static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) +{ + struct iommu_domain *domain = dev_iommu_domain(dev); + struct arm_smmu_xen_domain *xen_domain; + + xen_domain = domain_hvm_iommu(d)->arch.priv; + + if (!domain || domain->priv->cfg.domain != d) { + dev_err(dev, " not attached to domain %d\n", d->domain_id); + return -ESRCH; + } + + arm_smmu_detach_dev(domain, dev); + + spin_lock(&xen_domain->lock); + list_del(&domain->list); + spin_unlock(&xen_domain->lock); + + arm_smmu_domain_destroy(domain); + xfree(domain); + + return 0; +} + +static int arm_smmu_reassign_dev(struct domain *s, struct domain *t, + u8 devfn, struct device *dev) +{ + int ret = 0; + + /* Don't allow remapping on other domain than hwdom */ + if (t != hardware_domain) + return -EPERM; + + if (t == s) + return 0; + + ret = arm_smmu_deassign_dev(s, dev); + if (ret) + return ret; + + return 0; +} + +static int arm_smmu_iommu_domain_init(struct domain *d) +{ + struct arm_smmu_xen_domain *xen_domain; + + xen_domain = xzalloc(struct arm_smmu_xen_domain); + if ( !xen_domain ) + return -ENOMEM; + + spin_lock_init(&xen_domain->lock); + INIT_LIST_HEAD(&xen_domain->contexts); + + domain_hvm_iommu(d)->arch.priv = xen_domain; + + return 0; +} + +static void __hwdom_init arm_smmu_iommu_hwdom_init(struct domain *d) +{ +} + +static void arm_smmu_iommu_domain_teardown(struct domain *d) +{ + struct arm_smmu_xen_domain *xen_domain = domain_hvm_iommu(d)->arch.priv; + + ASSERT(list_empty(&xen_domain->contexts)); + xfree(xen_domain); +} + +static int arm_smmu_map_page(struct domain *d, unsigned long gfn, + unsigned long mfn, unsigned int flags) +{ + p2m_type_t t; + + /* + * Grant mappings can be used for DMA requests. The dev_bus_addr + * returned by the hypercall is the MFN (not the IPA). For device + * protected by an IOMMU, Xen needs to add a 1:1 mapping in the domain + * p2m to allow DMA request to work. + * This is only valid when the domain is directed mapped. Hence this + * function should only be used by gnttab code with gfn == mfn. + */ + BUG_ON(!is_domain_direct_mapped(d)); + BUG_ON(mfn != gfn); + + /* We only support readable and writable flags */ + if (!(flags & (IOMMUF_readable | IOMMUF_writable))) + return -EINVAL; + + t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw : p2m_iommu_map_ro; + + /* + * The function guest_physmap_add_entry replaces the current mapping + * if there is already one... + */ + return guest_physmap_add_entry(d, gfn, mfn, 0, t); +} + +static int arm_smmu_unmap_page(struct domain *d, unsigned long gfn) +{ + /* + * This function should only be used by gnttab code when the domain + * is direct mapped + */ + if ( !is_domain_direct_mapped(d) ) + return -EINVAL; + + guest_physmap_remove_page(d, gfn, gfn, 0); + + return 0; +} + +static const struct iommu_ops arm_smmu_iommu_ops = { + .init = arm_smmu_iommu_domain_init, + .hwdom_init = arm_smmu_iommu_hwdom_init, + .teardown = arm_smmu_iommu_domain_teardown, + .iotlb_flush = arm_smmu_iotlb_flush, + .iotlb_flush_all = arm_smmu_iotlb_flush_all, + .assign_device = arm_smmu_assign_dev, + .reassign_device = arm_smmu_reassign_dev, + .map_page = arm_smmu_map_page, + .unmap_page = arm_smmu_unmap_page, +}; + +static __init int arm_smmu_dt_init(struct dt_device_node *dev, + const void *data) +{ + int rc; + + /* + * Even if the device can't be initialized, we don't want to + * give the SMMU device to dom0. + */ + dt_device_set_used_by(dev, DOMID_XEN); + + rc = arm_smmu_device_dt_probe(dev); + if (rc) + return rc; + + iommu_set_ops(&arm_smmu_iommu_ops); + + return 0; +} + +DT_DEVICE_START(smmu, "ARM SMMU", DEVICE_IOMMU) + .dt_match = arm_smmu_of_match, + .init = arm_smmu_dt_init, +DT_DEVICE_END