From patchwork Mon Jan 25 11:06:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370185 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp3888975jam; Mon, 25 Jan 2021 03:14:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJyncmZpEs2+t3gNGv7H44GJSMnskNFTwij7HMyTTHlSNIFYyu6cNzy+rwt3tAlY6ExsBGtl X-Received: by 2002:a17:906:2755:: with SMTP id a21mr45461ejd.374.1611573288745; Mon, 25 Jan 2021 03:14:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611573288; cv=none; d=google.com; s=arc-20160816; b=rfQFjabpfiOMS/AJWalah2zsOBBDWEjwHLFmSAM5Vx05tNHigFWYDo0SKmQ07B1gfS oY7bsM3O5ntAQosLd6P+t11T4QfjXGCMHloTjwIuQRe1oDMWpvMEU5su8GjA3+rD0OEK DXtjJF70dj9ZpZyRB8LZ7x5S4CmictvNpLeRnAT3DpDQ3P3Aj+GQPoBLdEd1nw2xrKbW 0DNoEBXSP02HX0xsGFlVD6D7uazDDhkIrnV+lOd08lZMhPM5SE3dEDiwzmFCHfts+Xf4 mMJRxFzIuQ7x/QEE4O3AYUV2KU3qye33y+33RKFqEZY9B+eHD9VklQeBCB8c7V2R/ixg hNJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0PBUJkI6WkOQ4c7ebH11XhAeKbkF/R51HzjghE20E74=; b=bPBggFpdTSZLoKG1PM3B5tppNowbWfnYZg3OwFseuB3RRzyCd7mTbrjhXSvR8+PlPq 6fZOmQl8lKjtoJa3Qj6n500bTGspr//Rj6ogB9l6BvhGiBYo7jXJ+CKMtJzaBwgWdq9s 2sq/Uk+7jFP25BJTp1hh36KUcmV8KR+0pWRFsbTXw57+5DiWtEsfgJq7CrKXXYcp8Dph QH7vMy74hU9fe4lpFfq7m/MG0caPyK/Cilm/VDfmJpaCL56GJyAkD5dQlNm8ACs047sK 8ESS8STzksNoDl5Dxq5kd2DtNiDbrFEIu3xSQDQCPVLVzC6mOpzlLHMQLj7F/h1AsqVX OX6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=B9KaUOkk; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si7641532edj.297.2021.01.25.03.14.48; Mon, 25 Jan 2021 03:14:48 -0800 (PST) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=B9KaUOkk; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727487AbhAYLLd (ORCPT + 6 others); Mon, 25 Jan 2021 06:11:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727749AbhAYLJZ (ORCPT ); Mon, 25 Jan 2021 06:09:25 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7AEE0C061788 for ; Mon, 25 Jan 2021 03:08:44 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id a9so11867821wrt.5 for ; Mon, 25 Jan 2021 03:08:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0PBUJkI6WkOQ4c7ebH11XhAeKbkF/R51HzjghE20E74=; b=B9KaUOkkiIKWv8s2IR7smiU5sFPjAiN4DvikF1sFMVhKC/uIOvl6ALbKbwpshTHYhq uYlkIkMlTVMct834n78aPdvWscG0Eyoc3EnZzONmkn1iXAjMvIy14TGKqXDA4TRJF+ty C1CX4PhEEl2T7Ze64pq+ykjrVSqBRYatUZ/6X2KCNmaNWfVgcV20lYHtHjonRCOnLcMH USX0CG8hmctAWUpJOgFG0Dkm38Mlx+PprBnJWCsMtGrpZiyguw/Shyso3CvfZCLsaMwh aKw5HbfQon2fs6p5JnEGwxdfmg2YXdVUGZpsLiN5qVpWj3UQkrgaxM8HgHAN529NHgQF Kpyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0PBUJkI6WkOQ4c7ebH11XhAeKbkF/R51HzjghE20E74=; b=bDU3bxezoXjG/0x/A7mOgvLRokK35tFlQIoW3S4ZELCZ1AICrstaN7uhG4ujyXq04Y yeRBBs8IEhaDloqTePYONghWwx9QZ/0TXnrp4zZRAv7KRBW7+tzwta5UQsnJ9prKc4uq 2Z5uUUrEaztx2/psMeDsBdw1C47Rg19+FC6c38V655VYMBP3E63/XgA9PHlQ57FVOhCl abPK3Xk8Vl2Chy6PKY9tHDlf2LnUjl4cJS03XZhl0u0LUvCsOJz1EinRvNd52V3qyM30 AOOw3wQ5iYoZ1u+whL0VcTVEZ1zGSi71vO/iZr1bGKUVRP02m7rgbkyc6oZAlFkcC5R0 Ts1g== X-Gm-Message-State: AOAM530hXNQ8mf+OF+yu/oRhicj2KH14fj3R+0Zq+scdeX2ht/sul2AL 5yy0TZxJnfZny8Sz+TDEUQqa1Q== X-Received: by 2002:adf:e8c5:: with SMTP id k5mr338835wrn.242.1611572923092; Mon, 25 Jan 2021 03:08:43 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:42 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v11 01/10] iommu: Fix comment for struct iommu_fwspec Date: Mon, 25 Jan 2021 12:06:42 +0100 Message-Id: <20210125110650.3232195-2-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Commit 986d5ecc5699 ("iommu: Move fwspec->iommu_priv to struct dev_iommu") removed iommu_priv from fwspec and commit 5702ee24182f ("ACPI/IORT: Check ATS capability in root complex nodes") added @flags. Update the struct doc. Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- include/linux/iommu.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.30.0 diff --git a/include/linux/iommu.h b/include/linux/iommu.h index b3f0e2018c62..bdf3f34a4457 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -570,7 +570,7 @@ struct iommu_group *fsl_mc_device_group(struct device *dev); * struct iommu_fwspec - per-device IOMMU instance data * @ops: ops for this device's IOMMU * @iommu_fwnode: firmware handle for this device's IOMMU - * @iommu_priv: IOMMU driver private data for this device + * @flags: IOMMU_FWSPEC_* flags * @num_pasid_bits: number of PASID bits supported by this device * @num_ids: number of associated device IDs * @ids: IDs which this device may present to the IOMMU From patchwork Mon Jan 25 11:06:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8105C433DB for ; Mon, 25 Jan 2021 11:28:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7429A224D1 for ; Mon, 25 Jan 2021 11:28:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727647AbhAYLLY (ORCPT ); Mon, 25 Jan 2021 06:11:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727753AbhAYLJ0 (ORCPT ); Mon, 25 Jan 2021 06:09:26 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C3A6C06178A for ; Mon, 25 Jan 2021 03:08:45 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id m187so10307997wme.2 for ; Mon, 25 Jan 2021 03:08:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D32J+akdetMmA5Myd4Zx5B3eWdLb4pZJ7wlpCkerL8M=; b=sIgFUy1B4NVZTMdif41pOLYx/+zRTjX1CmBp28JOvA9dBq+ncegVdQKTKx6zFKpMA0 RLrBDORpb4Wopov1wmsqCNwtxU/OTA+4T/HidEMzrcX/l10yuWmh2vn1tHVSCMf1WGwn k/I3+x++pEPyRk48uxdzw2k8HGR8Ad4NRGsrzffLFLL0/IVqTdb8ajx4Lw+x49oM0mb+ iPBIL+SI8d3oTuaFhQanBe3jnURITn3n3J5wb8bR46LkJ/HFRCz+dKaO+UEi+7Murd4i KSPMi9rgFEX2zKXtfEEoY7s4DzLcaxbp9ooMwRJIcrA/HFyS6BmTyiujmteB8sgRLdFG CK3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D32J+akdetMmA5Myd4Zx5B3eWdLb4pZJ7wlpCkerL8M=; b=KhkvJB4ntYt8NXNnO8oabvtHDPzFXpypp6bBKEn8NVWgkchK0FeTxUuRl3+F+WHDqu ksqvWzYM+YQyt+ZPNB3o+X0XJxpW//SLEzw1npd9hpMMqQYsW+j+IGnwQJdczR7DCYK5 cw5u1G0FLru7vDqo23OUpr2Y9OokCsPQKaFgQHPvHHHHjA6brhdgqGo6MhP3U/bO0bmQ ZETtZTZwjyYpaNW3pjB+JY15fJ59SKSdHlCuUzGx50FxxkIRlL9sFR6iw12XbSBzXrej VmJKKYTYnA6qfKjai9Hmt2z9ekdPaNdCIcseCrgRi7s6BGAvJz3OS/E0Rfzf2DfwyNnd Upmg== X-Gm-Message-State: AOAM531eMcBrgYFlNL8KFszsoK6PH/4TqTgyEl6xAWQ5ImYIF8o/1cmT NyFrrkbDQ9U3h0zNN0b8Dh1bxQ== X-Google-Smtp-Source: ABdhPJwWUAbaa+u9PhjGecAC0HvYf8RfDHP6EKMUj3KGOJ93rmXeB4FDwvfcCPo+QdcrgOx8578F9Q== X-Received: by 2002:a05:600c:4ed3:: with SMTP id g19mr579133wmq.95.1611572924335; Mon, 25 Jan 2021 03:08:44 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:43 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v11 02/10] iommu/arm-smmu-v3: Use device properties for pasid-num-bits Date: Mon, 25 Jan 2021 12:06:43 +0100 Message-Id: <20210125110650.3232195-3-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org The pasid-num-bits property shouldn't need a dedicated fwspec field, it's a job for device properties. Add properties for IORT, and access the number of PASID bits using device_property_read_u32(). Suggested-by: Robin Murphy Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- include/linux/iommu.h | 2 -- drivers/acpi/arm64/iort.c | 13 +++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 ++- drivers/iommu/of_iommu.c | 5 ----- 4 files changed, 9 insertions(+), 14 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index bdf3f34a4457..b7ea11fc1a93 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -571,7 +571,6 @@ struct iommu_group *fsl_mc_device_group(struct device *dev); * @ops: ops for this device's IOMMU * @iommu_fwnode: firmware handle for this device's IOMMU * @flags: IOMMU_FWSPEC_* flags - * @num_pasid_bits: number of PASID bits supported by this device * @num_ids: number of associated device IDs * @ids: IDs which this device may present to the IOMMU */ @@ -579,7 +578,6 @@ struct iommu_fwspec { const struct iommu_ops *ops; struct fwnode_handle *iommu_fwnode; u32 flags; - u32 num_pasid_bits; unsigned int num_ids; u32 ids[]; }; diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index d4eac6d7e9fb..c9a8bbb74b09 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -968,15 +968,16 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data) static void iort_named_component_init(struct device *dev, struct acpi_iort_node *node) { + struct property_entry props[2] = {}; struct acpi_iort_named_component *nc; - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - - if (!fwspec) - return; nc = (struct acpi_iort_named_component *)node->node_data; - fwspec->num_pasid_bits = FIELD_GET(ACPI_IORT_NC_PASID_BITS, - nc->node_flags); + props[0] = PROPERTY_ENTRY_U32("pasid-num-bits", + FIELD_GET(ACPI_IORT_NC_PASID_BITS, + nc->node_flags)); + + if (device_add_properties(dev, props)) + dev_warn(dev, "Could not add device properties\n"); } static int iort_nc_iommu_map(struct device *dev, struct acpi_iort_node *node) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index baebaac34a83..88dd9feb32f4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2392,7 +2392,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) } } - master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits); + device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits); + master->ssid_bits = min(smmu->ssid_bits, master->ssid_bits); /* * Note that PASID must be enabled before, and disabled after ATS: diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c index e505b9130a1c..a9d2df001149 100644 --- a/drivers/iommu/of_iommu.c +++ b/drivers/iommu/of_iommu.c @@ -210,11 +210,6 @@ const struct iommu_ops *of_iommu_configure(struct device *dev, of_pci_iommu_init, &info); } else { err = of_iommu_configure_device(master_np, dev, id); - - fwspec = dev_iommu_fwspec_get(dev); - if (!err && fwspec) - of_property_read_u32(master_np, "pasid-num-bits", - &fwspec->num_pasid_bits); } /* From patchwork Mon Jan 25 11:06:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09C1CC433E9 for ; Mon, 25 Jan 2021 11:27:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B5900229C5 for ; Mon, 25 Jan 2021 11:27:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727738AbhAYLZt (ORCPT ); Mon, 25 Jan 2021 06:25:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727697AbhAYLMg (ORCPT ); Mon, 25 Jan 2021 06:12:36 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED358C06178C for ; Mon, 25 Jan 2021 03:08:46 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id b5so11864399wrr.10 for ; Mon, 25 Jan 2021 03:08:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uiSdj3ahcV4jfiq23D9UAVEBduTBnIEfUfM+Rzx87y0=; b=o1trokFtSTCfvkocLi895coCTyzkTkfVye5g5jBBpjAgejVTdH5D1p5iBs5mtxk5kx mn6KNV0w7qjzSiXB1pheQswkz5HrDXxVo3wDWV5rM/ULRbJ1r3psOC3kaJ/pE4BmTo36 HPLBwStDbURpW834zKOhjGc51y2kENbOdl5bZvNk+Hz9cjGYWNACLOU7IA/iXFH3SHIP yiu6DtXmE6rzyuRYBY2OIUHos1ZOqe0LMFDLcDopc70vRuGNN8t4ia4kUd7an+ncEQwD dvpm1J7Ho4IXV8x4cXbBE8WbeXlwGKe2KLODyr6dUadfFvfEr8JoTpffCM0YnSwz9oK3 Rmxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uiSdj3ahcV4jfiq23D9UAVEBduTBnIEfUfM+Rzx87y0=; b=OOr/qvI0hL8DBDAsHjuv1IFnUSTQ/bi2n+6xgdLQBQYkfmSwO+PmiX6JWwH8jc3bHU KrpsszMiqTU4TX9sTUOf68lA+VS9BAl19W81K6QmhtyS+XtyDpRXjwVui5tUPfrHmBbe FNQa1tHcnzBPXaPWW7fe1Gc0LJBWNav9Cg4dxrScA8pKWYHp9Pmr+pmkWl4Rd4MK1lNn Kp75EMYEmsqI4icxw+iM1o91VACCvTFzGdHm6qsFyU4/DgrwqH0SwJj1t/O8IUG6hOTs AeCHa784qjqlPXdglLEQyRPySbljsxwZaMtps6WzCgcYapeYZxOOHGeaXFxYB8WfuumO OmbA== X-Gm-Message-State: AOAM531pmZCpXHbC0f3Tf5o2wfVJngCCEwEp2v+DCsJYaXR6GoUyNEvK qdAcKopeQyUn3ucEvOOFLBgw/w== X-Google-Smtp-Source: ABdhPJwwlQQTzJJVFrK3jv6EdbzkxepXZyhe6MIPrsPL57ZER4N2Q2U1BVzXSW+FFTj3InzYFSgEFQ== X-Received: by 2002:adf:e807:: with SMTP id o7mr334734wrm.308.1611572925581; Mon, 25 Jan 2021 03:08:45 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:45 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , Arnd Bergmann , David Woodhouse , Greg Kroah-Hartman , Zhou Wang Subject: [PATCH v11 03/10] iommu: Separate IOMMU_DEV_FEAT_IOPF from IOMMU_DEV_FEAT_SVA Date: Mon, 25 Jan 2021 12:06:44 +0100 Message-Id: <20210125110650.3232195-4-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Some devices manage I/O Page Faults (IOPF) themselves instead of relying on PCIe PRI or Arm SMMU stall. Allow their drivers to enable SVA without mandating IOMMU-managed IOPF. The other device drivers now need to first enable IOMMU_DEV_FEAT_IOPF before enabling IOMMU_DEV_FEAT_SVA. Enabling IOMMU_DEV_FEAT_IOPF on its own doesn't have any effect visible to the device driver, it is used in combination with other features. Signed-off-by: Jean-Philippe Brucker --- Cc: Arnd Bergmann Cc: David Woodhouse Cc: Greg Kroah-Hartman Cc: Joerg Roedel Cc: Lu Baolu Cc: Will Deacon Cc: Zhangfei Gao Cc: Zhou Wang --- include/linux/iommu.h | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index b7ea11fc1a93..00348e4c3c26 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -156,10 +156,24 @@ struct iommu_resv_region { enum iommu_resv_type type; }; -/* Per device IOMMU features */ +/** + * enum iommu_dev_features - Per device IOMMU features + * @IOMMU_DEV_FEAT_AUX: Auxiliary domain feature + * @IOMMU_DEV_FEAT_SVA: Shared Virtual Addresses + * @IOMMU_DEV_FEAT_IOPF: I/O Page Faults such as PRI or Stall. Generally + * enabling %IOMMU_DEV_FEAT_SVA requires + * %IOMMU_DEV_FEAT_IOPF, but some devices manage I/O Page + * Faults themselves instead of relying on the IOMMU. When + * supported, this feature must be enabled before and + * disabled after %IOMMU_DEV_FEAT_SVA. + * + * Device drivers query whether a feature is supported using + * iommu_dev_has_feature(), and enable it using iommu_dev_enable_feature(). + */ enum iommu_dev_features { - IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */ - IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */ + IOMMU_DEV_FEAT_AUX, + IOMMU_DEV_FEAT_SVA, + IOMMU_DEV_FEAT_IOPF, }; #define IOMMU_PASID_INVALID (-1U) From patchwork Mon Jan 25 11:06:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370188 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp3898079jam; Mon, 25 Jan 2021 03:25:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJx9Wi3GYDOk7wH03dy7yNLdExAbjTd/EK0eiETCCzSWOOYYVIW94WrhdnLpCXyOmz/3I34Z X-Received: by 2002:aa7:c3d9:: with SMTP id l25mr58514edr.188.1611573951647; Mon, 25 Jan 2021 03:25:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611573951; cv=none; d=google.com; s=arc-20160816; b=evlrloFei/JxjRmedCuJFK7u8lqdkkPLvIXAqABm8gokVdM4ijUlYS2BQJTcDdjomC 5O5hJ4fiPBpr91B9g202KF75fdXcl7De3hpFruGu20IexsRqPVBTwPkYNsZ1c28ZaKcM a9ErIWT6HLJABbwtoPn43NjF9h0rONWo9snQ5Fw1yrIhmt/MFa3KPGqrSVLSU21nfzbi DYoZQsacNsrE0CQtsrGDOYni8hGuxn/KfWK9kw9VJLUkAvstYU2iS6SjAtEFQvd35Is+ nOvKhfkyUj8K5rxxciGesTH+EqxKGZ1spPD7GSFV8pyPHO3rRMnRijczTLTnfIAfjSRb vqzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0HAbfAEz8TnDqiVsICUxE3hnnLknlvV/GMLCZp7lxlM=; b=PJcEGVGYRjLuQn0By30H75T+hiz3QaugiklZlvy5/wsBVP5yUBfz5bbQNI1owBCe+0 gehqV9M/s4NfjXWDr+0EYyPUVjEqIoL2wH1Fm62Pus3HyU4lgbFkWrX6YkbgWhAj04+E Ock6w8s/M39KTzSypGUywtpMhCQE1vacYfuNg8xNCaFs5ShkJgxZXX9UBgb8opgS6WRf e4yYp5BVcJzTXJYs7weERodAOZAntEKC7n4XzUvw8j7mLcejYdcn+xHDW3CmsljA9qjk lVXvSKp08b4aQK6fENGe9QxD7+PZIgZP5Jbs8ikUH2ugwsOhp4VWS560fucifbIggpF/ P4QQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rSVIjdC8; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si7641532edj.297.2021.01.25.03.25.51; Mon, 25 Jan 2021 03:25:51 -0800 (PST) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rSVIjdC8; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727552AbhAYLZp (ORCPT + 6 others); Mon, 25 Jan 2021 06:25:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727690AbhAYLMl (ORCPT ); Mon, 25 Jan 2021 06:12:41 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CB99C061797 for ; Mon, 25 Jan 2021 03:08:48 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id s24so10253236wmj.0 for ; Mon, 25 Jan 2021 03:08:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0HAbfAEz8TnDqiVsICUxE3hnnLknlvV/GMLCZp7lxlM=; b=rSVIjdC8auMNf3CzZHMj2V5f//KBePOMSE1xJsfhrdqnk+R7tLKov8OtTs4OoGyGCz tAhzdrunXOWEUWgI/eb8yjnFbAsLVxgel2IAvs86EletOyYtNUovgXfYIV/Mx1Fi3mdR 7+g6OkoWx1rQ61ah5fpmX3QQHoF8QWYU4/iT+iSly7H2QW+JG+Gv4HbeLoheGKRPUKjf SOCKdkQlMv+MModmy88QGqNISODCenboQjcS5OLpyPGnhdTdQnAug522uBBWgWOHnUoc SiOrsSKcq9FBUMUmBZvZT7pj6bYmnWHY2BJjKW8gIDFP/J+hu+5xrm+Atpu14Y4SUVc2 cHCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0HAbfAEz8TnDqiVsICUxE3hnnLknlvV/GMLCZp7lxlM=; b=dN873E03o5/PZM/cb1S3nqX0oNIRk+xQLDbIBIbFBe4S6p/SIVQegnqHFIBPtTR1be zVFk3gE2G90IyqOjMiFAJm9iiGtjk8bvX+npVxFWb0y2YN9+wyaxBPUt68+Wc+en9nNX gKk42n+4dLHnrtE3YBbd2k3w437Sz52wET87S/nbTb91azH+k+YlJ/OyReKzS/fmsDwt 4P8Q0/NWS3evqN2Lw67exuRfNLMpG6zuKbXm5Svk0CieEyGEZjwt80IebeTkc04lzkDD B1QEVzqJk1tEu8GvSRMjVXP4lHX1ePlV2611MZHS4s2tP2Y7LaVm/97wxJuhhj5tQ2i2 LltQ== X-Gm-Message-State: AOAM532VwpOwtVA+OxWkIXDIv50Fd90bGVFZMXvWxbDrPXe1/UU70r/s UpZArpCBoNJ28yA46F+LhrePyQ== X-Received: by 2002:a1c:7312:: with SMTP id d18mr2104533wmb.155.1611572926856; Mon, 25 Jan 2021 03:08:46 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:46 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , David Woodhouse Subject: [PATCH v11 04/10] iommu/vt-d: Support IOMMU_DEV_FEAT_IOPF Date: Mon, 25 Jan 2021 12:06:45 +0100 Message-Id: <20210125110650.3232195-5-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Allow drivers to query and enable IOMMU_DEV_FEAT_IOPF, which amounts to checking whether PRI is enabled. Reviewed-by: Lu Baolu Signed-off-by: Jean-Philippe Brucker --- Cc: David Woodhouse Cc: Lu Baolu --- drivers/iommu/intel/iommu.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) -- 2.30.0 diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index f665322a0991..c777bd94df5d 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -5330,6 +5330,8 @@ static int siov_find_pci_dvsec(struct pci_dev *pdev) static bool intel_iommu_dev_has_feat(struct device *dev, enum iommu_dev_features feat) { + struct device_domain_info *info = get_domain_info(dev); + if (feat == IOMMU_DEV_FEAT_AUX) { int ret; @@ -5344,13 +5346,13 @@ intel_iommu_dev_has_feat(struct device *dev, enum iommu_dev_features feat) return !!siov_find_pci_dvsec(to_pci_dev(dev)); } - if (feat == IOMMU_DEV_FEAT_SVA) { - struct device_domain_info *info = get_domain_info(dev); + if (feat == IOMMU_DEV_FEAT_IOPF) + return info && info->pri_supported; + if (feat == IOMMU_DEV_FEAT_SVA) return info && (info->iommu->flags & VTD_FLAG_SVM_CAPABLE) && info->pasid_supported && info->pri_supported && info->ats_supported; - } return false; } @@ -5361,6 +5363,9 @@ intel_iommu_dev_enable_feat(struct device *dev, enum iommu_dev_features feat) if (feat == IOMMU_DEV_FEAT_AUX) return intel_iommu_enable_auxd(dev); + if (feat == IOMMU_DEV_FEAT_IOPF) + return intel_iommu_dev_has_feat(dev, feat) ? 0 : -ENODEV; + if (feat == IOMMU_DEV_FEAT_SVA) { struct device_domain_info *info = get_domain_info(dev); From patchwork Mon Jan 25 11:06:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370186 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp3897666jam; Mon, 25 Jan 2021 03:25:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJwVYipBeqNTP7Ue0OobCvin63ndpky3KmnkRbBL20Rff6jvc4R5hQexVI3qDlr+ESJzcxLC X-Received: by 2002:a17:906:154d:: with SMTP id c13mr40574ejd.471.1611573927962; Mon, 25 Jan 2021 03:25:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611573927; cv=none; d=google.com; s=arc-20160816; b=gF0hvLmHMbpQxakWMNrivB28YiD0URHU3NqgDnHB26nTzhly91js2FXMdnmZXN+Na4 mp3MhhcB8tJz1Py1wBe2lyonBqkocfbJmUER8bG+9KSXGe0SgXuIutQSs3gObFI+nEqV GFgdH9ZmGgRwfyfGCtGuBgxNzgQoVv1HfHHV3cCsORDFu/ukiO0vVBA9NBf3R9M0BuSv 3wd6StepdPzR+FpqVT0iMmkwojlKAqrIXl5r/MWUW+9qJPSyUjbVZIpn2RDEKsum62T8 lDvpy/aOVRxGk5MXwqzL8XiAWWLWV7dHtuwrGeydMouDNh8hKp1sVgoi4ZJAqQ8ICATs 8tLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2eRhKsVau0r7rYJc5kg9wODttGekaxkRDREYSkG2V3g=; b=DpWu/ULkfMkNAzUeGxspg39L0VKyNTauHazkpCKjMaaXXsNyrcd2V5pyrN/MrhcL1Q OQxHMhnnD7UbmNdZTXoRVYNH8QpOXflR6bEoCGrCf9Ejhci+4qb5vgtw5riuXX8z+lvT aOlrvrwSoC0RAsKzALM2kfoQLk7Zk9/iSruwM4GM/7WzTzQWLEDfn46lUBdqtxYYKp2f Xfgaf2eEyCiCdf9WffYfMyNROUcIG0bF43Ax1DZT62qLWyyn/GCSV5VMj+TFOXjPs+kp 0fw2ECik/q3BG1VQ2gTjxNMcRWiYwj8p/5Nb/U05eh4euFwfxKEvsaPWMPQsnnSkHmjW SVZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KsPtTSBA; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si7641532edj.297.2021.01.25.03.25.27; Mon, 25 Jan 2021 03:25:27 -0800 (PST) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KsPtTSBA; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727761AbhAYLYm (ORCPT + 6 others); Mon, 25 Jan 2021 06:24:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727738AbhAYLMm (ORCPT ); Mon, 25 Jan 2021 06:12:42 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6722EC0617AA for ; Mon, 25 Jan 2021 03:08:49 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id i9so3925500wmq.1 for ; Mon, 25 Jan 2021 03:08:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2eRhKsVau0r7rYJc5kg9wODttGekaxkRDREYSkG2V3g=; b=KsPtTSBAe0tkUxNLmUgmnZTm9c5AfkV2YTEWV91OLlxWU6JSKf4TB8V0VJ52+PRz5M T5EtOM+4J3dfMdgyBbN7bdeDrXK6f1CJQo70cwWWD2/PdUJDA3fzcDOPJ2vYObqfF2iz xdypk+jPI41aBtU/ZlFSq13WhFnBQS9ToWK69kW6nrQMv7Nyla9UJoQMzKFheu3jHi5r 42J2XKlP8z3Virz2svl9Lpi88Feg2unVcV98HLW1m8p4U+ZKpsCGfTBD/OnfHXowYuYQ AxnygmlRoR39X1p70/TI395nERXf7alKLVOR4sbR/F+VOGSzOgjGLYQNZmWywJVK71mT JpBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2eRhKsVau0r7rYJc5kg9wODttGekaxkRDREYSkG2V3g=; b=GIwEzxPXgOoG81uSbtwauM5/gHuAJmxhuJfb9CQRQTB2ntyrfkNcBT69g4r2MHEZoD bqIbwurGVe2wUILhxbvZRvjYeP7QF1udFiWoLzJyB/FfUke9zcwHnyyka65xInlqMZIQ kLJzEpDYfszNf3glhZTRaqxz8Vlnz2CmttjSoUoa4RH6lD+n2u8clPjJ/4/aE6MDttAC fKex/VU7VUNVXJajDsUDhPAwg2hqZ94zNNPDD3IzIF4GSijARluXwm8FMLnla0MT5Zd9 tIsYH8VPM1qBvYkAd/+bxNEy4Cn/hEgFSLE0rAeCHYoBm8C9G8SR0ADiXKzz9Kb/RSm2 ildQ== X-Gm-Message-State: AOAM5335VzBhsWPAAj7f5XDMEEvrjgrilXEpmrFTlPfIGXHNzrTlE9SW CpW2xgIrn5lU9mhJSc6c0p3gkQ== X-Received: by 2002:a1c:3c04:: with SMTP id j4mr1104717wma.147.1611572928161; Mon, 25 Jan 2021 03:08:48 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:47 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , Arnd Bergmann , Greg Kroah-Hartman , Zhou Wang Subject: [PATCH v11 05/10] uacce: Enable IOMMU_DEV_FEAT_IOPF Date: Mon, 25 Jan 2021 12:06:46 +0100 Message-Id: <20210125110650.3232195-6-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org The IOPF (I/O Page Fault) feature is now enabled independently from the SVA feature, because some IOPF implementations are device-specific and do not require IOMMU support for PCIe PRI or Arm SMMU stall. Enable IOPF unconditionally when enabling SVA for now. In the future, if a device driver implementing a uacce interface doesn't need IOPF support, it will need to tell the uacce module, for example with a new flag. Acked-by: Zhangfei Gao Signed-off-by: Jean-Philippe Brucker --- Cc: Arnd Bergmann Cc: Greg Kroah-Hartman Cc: Zhangfei Gao Cc: Zhou Wang --- drivers/misc/uacce/uacce.c | 39 +++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) -- 2.30.0 diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index d07af4edfcac..6db7a98486ec 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -385,6 +385,33 @@ static void uacce_release(struct device *dev) kfree(uacce); } +static unsigned int uacce_enable_sva(struct device *parent, unsigned int flags) +{ + if (!(flags & UACCE_DEV_SVA)) + return flags; + + flags &= ~UACCE_DEV_SVA; + + if (iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_IOPF)) + return flags; + + if (iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_SVA)) { + iommu_dev_disable_feature(parent, IOMMU_DEV_FEAT_IOPF); + return flags; + } + + return flags | UACCE_DEV_SVA; +} + +static void uacce_disable_sva(struct uacce_device *uacce) +{ + if (!(uacce->flags & UACCE_DEV_SVA)) + return; + + iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_IOPF); +} + /** * uacce_alloc() - alloc an accelerator * @parent: pointer of uacce parent device @@ -404,11 +431,7 @@ struct uacce_device *uacce_alloc(struct device *parent, if (!uacce) return ERR_PTR(-ENOMEM); - if (flags & UACCE_DEV_SVA) { - ret = iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_SVA); - if (ret) - flags &= ~UACCE_DEV_SVA; - } + flags = uacce_enable_sva(parent, flags); uacce->parent = parent; uacce->flags = flags; @@ -432,8 +455,7 @@ struct uacce_device *uacce_alloc(struct device *parent, return uacce; err_with_uacce: - if (flags & UACCE_DEV_SVA) - iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + uacce_disable_sva(uacce); kfree(uacce); return ERR_PTR(ret); } @@ -487,8 +509,7 @@ void uacce_remove(struct uacce_device *uacce) mutex_unlock(&uacce->queues_lock); /* disable sva now since no opened queues */ - if (uacce->flags & UACCE_DEV_SVA) - iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + uacce_disable_sva(uacce); if (uacce->cdev) cdev_device_del(uacce->cdev, &uacce->dev); From patchwork Mon Jan 25 11:06:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370190 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp3900277jam; Mon, 25 Jan 2021 03:28:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJynDUgxeKX/zaZAhlV41kYSwTuXN8+OZmw/5VQIG1TNVYXK/N0YuaeOhgk0biTsbgd/LWnX X-Received: by 2002:a05:6402:95a:: with SMTP id h26mr54244edz.288.1611574122712; Mon, 25 Jan 2021 03:28:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611574122; cv=none; d=google.com; s=arc-20160816; b=Ykv+uEWGX4mKpgeDF7p84/RwJ7O6+lZ+jCZuzULdmFV++8CpIXquNEOP4MuHL1bG5l S4jY6J9+/8CQ39+lK0e1z6KduLmMRPEtDaaOeOxMCxUSm3JRdCt19GFOu9Tj3FA4aoSC matWnhlda2BcUcEakvjA5VoI88vBQXeCiZbi8gdOk8DRaPAjp/prWyjm754J694P50Ew 4iQ2pKt9wSdevFrP+g0tdq+eFeWbi2DeqAcTovKEn0JM6jJXmfQd3zAY00pEuMv0SCXc Y0Bzgc3C8KtjnGFEDWJUTE8PtpxGAQYE32zyWMQdw9HKOYsHHD1Sj/JxIc8bhmezMeY5 iJhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=j4gQVr3WKH1e0317BI6PolZ4g0GU2TEiJ+uV7JsOhgI=; b=mS+PwQzwq0M5w2t4Wh2rlpUGYKXWy8vttXAQi8RbPiGt3Dd914pcBGfQRfMoMkXdnO vTtgidUmRbYQOVifRYqNtHtIFPo/6hWUIKXMxKCifxJ/sRd8pgSahZoWg0cuBLU6r3sH dVWhXqhRwWwoEKD1xh54M598GFDvO/w3l+IKrolwfaG8pFQjzeCNXNo3D7/x6VV0H/r8 wke0K4G8TjIKkfP3A2wC0kuPGOsLs+cgw0aC27GLTYaOyIc4leirzAMVg1JKHP298IF0 RVhBh+mxiRi05Ta6UIzeWObH+6HI6N23OnxPlBLlQ1pSwfInnC9kufon8ftrT75bZCYt 1OrQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aNSSkAbb; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id p31si7609602edd.505.2021.01.25.03.28.42; Mon, 25 Jan 2021 03:28:42 -0800 (PST) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aNSSkAbb; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727778AbhAYL1X (ORCPT + 6 others); Mon, 25 Jan 2021 06:27:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727753AbhAYLMg (ORCPT ); Mon, 25 Jan 2021 06:12:36 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A83CAC0617AB for ; Mon, 25 Jan 2021 03:08:50 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id 6so11881991wri.3 for ; Mon, 25 Jan 2021 03:08:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j4gQVr3WKH1e0317BI6PolZ4g0GU2TEiJ+uV7JsOhgI=; b=aNSSkAbba01LOdMn/CcQTcAnpXvTV4R+5GgiIm9APXL7xhYZQhr5E4qj4dyw7w7Atq ed0l3Pu7K80yjF1RzvioOnmbtZnI4T4ll3/qJGz271MhtREv//33YfwTx5O8xNaYXfJl /+3L0fdhc8YRNZ0l4+Wi4fqClryKZQVsWU3HdQQym34mU8yuY/zEWdM0ssy2zlyo7K1s 75LRds4/ll1SmWromsmEWUqHYBImP5tglVw6DkMphoyK4URrO02RMBhZCKuGdhyc8Kg4 BIeYDBcdQTNjZC20fiMnXXOsKV8uiPr6gt3dSK8ACWC/02ATjpXU/GdsPb4ljqpVpREv /5hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j4gQVr3WKH1e0317BI6PolZ4g0GU2TEiJ+uV7JsOhgI=; b=sLOwinRxDaP6lS/BwFZnkx4ceVxEmtFrRyHtaeNXoWniftgOayNBD1H+b+b5fnLDnv +rNjv7Vh8RKrV94ZgmUUJez5UwJVClLp0dbx+IHX3GyPdZIj4pPcyTgsd4WTWykAEHgH /iRDCF4TxZYvJGA4DJJj8YhCrXySXKwPZmKq2JBCMytojVeORDInplSNjbcgpZBsS8DQ Du30YmuB0Ny5NRYbUZAPCIiI/O7Gk1z6mk3vi9KdoIU5WUfgdCVdmQvg/C5yloVewHp3 HPtJguJ22AuQqNCWWjoIXdge46fe8tHjDqnVVkAnIFG4TmRt83SGMVqQsG6iDrVli8yQ ceNg== X-Gm-Message-State: AOAM532uUfKK/88nviUKwGFtvyk5CP8IOsobWogtT5MisTt5wulP4cBK d0EIWj2kEx2F+R+z3kjT313qRQ== X-Received: by 2002:a5d:6311:: with SMTP id i17mr290139wru.195.1611572929375; Mon, 25 Jan 2021 03:08:49 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:48 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v11 06/10] iommu: Add a page fault handler Date: Mon, 25 Jan 2021 12:06:47 +0100 Message-Id: <20210125110650.3232195-7-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Some systems allow devices to handle I/O Page Faults in the core mm. For example systems implementing the PCIe PRI extension or Arm SMMU stall model. Infrastructure for reporting these recoverable page faults was added to the IOMMU core by commit 0c830e6b3282 ("iommu: Introduce device fault report API"). Add a page fault handler for host SVA. IOMMU driver can now instantiate several fault workqueues and link them to IOPF-capable devices. Drivers can choose between a single global workqueue, one per IOMMU device, one per low-level fault queue, one per domain, etc. When it receives a fault event, most commonly in an IRQ handler, the IOMMU driver reports the fault using iommu_report_device_fault(), which calls the registered handler. The page fault handler then calls the mm fault handler, and reports either success or failure with iommu_page_response(). After the handler succeeds, the hardware retries the access. The iopf_param pointer could be embedded into iommu_fault_param. But putting iopf_param into the iommu_param structure allows us not to care about ordering between calls to iopf_queue_add_device() and iommu_register_device_fault_handler(). Reviewed-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/Makefile | 1 + drivers/iommu/iommu-sva-lib.h | 53 ++++ include/linux/iommu.h | 2 + drivers/iommu/io-pgfault.c | 461 ++++++++++++++++++++++++++++++++++ 4 files changed, 517 insertions(+) create mode 100644 drivers/iommu/io-pgfault.c -- 2.30.0 diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 61bd30cd8369..60fafc23dee6 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -28,3 +28,4 @@ obj-$(CONFIG_S390_IOMMU) += s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o obj-$(CONFIG_IOMMU_SVA_LIB) += iommu-sva-lib.o +obj-$(CONFIG_IOMMU_SVA_LIB) += io-pgfault.o diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index b40990aef3fd..031155010ca8 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -12,4 +12,57 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max); void iommu_sva_free_pasid(struct mm_struct *mm); struct mm_struct *iommu_sva_find(ioasid_t pasid); +/* I/O Page fault */ +struct device; +struct iommu_fault; +struct iopf_queue; + +#ifdef CONFIG_IOMMU_SVA_LIB +int iommu_queue_iopf(struct iommu_fault *fault, void *cookie); + +int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev); +int iopf_queue_remove_device(struct iopf_queue *queue, + struct device *dev); +int iopf_queue_flush_dev(struct device *dev); +struct iopf_queue *iopf_queue_alloc(const char *name); +void iopf_queue_free(struct iopf_queue *queue); +int iopf_queue_discard_partial(struct iopf_queue *queue); + +#else /* CONFIG_IOMMU_SVA_LIB */ +static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) +{ + return -ENODEV; +} + +static inline int iopf_queue_add_device(struct iopf_queue *queue, + struct device *dev) +{ + return -ENODEV; +} + +static inline int iopf_queue_remove_device(struct iopf_queue *queue, + struct device *dev) +{ + return -ENODEV; +} + +static inline int iopf_queue_flush_dev(struct device *dev) +{ + return -ENODEV; +} + +static inline struct iopf_queue *iopf_queue_alloc(const char *name) +{ + return NULL; +} + +static inline void iopf_queue_free(struct iopf_queue *queue) +{ +} + +static inline int iopf_queue_discard_partial(struct iopf_queue *queue) +{ + return -ENODEV; +} +#endif /* CONFIG_IOMMU_SVA_LIB */ #endif /* _IOMMU_SVA_LIB_H */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 00348e4c3c26..edc9be443a74 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -366,6 +366,7 @@ struct iommu_fault_param { * struct dev_iommu - Collection of per-device IOMMU data * * @fault_param: IOMMU detected device fault reporting data + * @iopf_param: I/O Page Fault queue and data * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to * @priv: IOMMU Driver private data @@ -376,6 +377,7 @@ struct iommu_fault_param { struct dev_iommu { struct mutex lock; struct iommu_fault_param *fault_param; + struct iopf_device_param *iopf_param; struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c new file mode 100644 index 000000000000..1df8c1dcae77 --- /dev/null +++ b/drivers/iommu/io-pgfault.c @@ -0,0 +1,461 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Handle device page faults + * + * Copyright (C) 2020 ARM Ltd. + */ + +#include +#include +#include +#include +#include + +#include "iommu-sva-lib.h" + +/** + * struct iopf_queue - IO Page Fault queue + * @wq: the fault workqueue + * @devices: devices attached to this queue + * @lock: protects the device list + */ +struct iopf_queue { + struct workqueue_struct *wq; + struct list_head devices; + struct mutex lock; +}; + +/** + * struct iopf_device_param - IO Page Fault data attached to a device + * @dev: the device that owns this param + * @queue: IOPF queue + * @queue_list: index into queue->devices + * @partial: faults that are part of a Page Request Group for which the last + * request hasn't been submitted yet. + */ +struct iopf_device_param { + struct device *dev; + struct iopf_queue *queue; + struct list_head queue_list; + struct list_head partial; +}; + +struct iopf_fault { + struct iommu_fault fault; + struct list_head list; +}; + +struct iopf_group { + struct iopf_fault last_fault; + struct list_head faults; + struct work_struct work; + struct device *dev; +}; + +static int iopf_complete_group(struct device *dev, struct iopf_fault *iopf, + enum iommu_page_response_code status) +{ + struct iommu_page_response resp = { + .version = IOMMU_PAGE_RESP_VERSION_1, + .pasid = iopf->fault.prm.pasid, + .grpid = iopf->fault.prm.grpid, + .code = status, + }; + + if ((iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID) && + (iopf->fault.prm.flags & IOMMU_FAULT_PAGE_RESPONSE_NEEDS_PASID)) + resp.flags = IOMMU_PAGE_RESP_PASID_VALID; + + return iommu_page_response(dev, &resp); +} + +static enum iommu_page_response_code +iopf_handle_single(struct iopf_fault *iopf) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags = 0; + unsigned int fault_flags = FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm = &iopf->fault.prm; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm = iommu_sva_find(prm->pasid); + if (IS_ERR_OR_NULL(mm)) + return status; + + mmap_read_lock(mm); + + vma = find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |= VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |= VM_WRITE; + fault_flags |= FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |= VM_EXEC; + fault_flags |= FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |= FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret = handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status = ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + +static void iopf_handle_group(struct work_struct *work) +{ + struct iopf_group *group; + struct iopf_fault *iopf, *next; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS; + + group = container_of(work, struct iopf_group, work); + + list_for_each_entry_safe(iopf, next, &group->faults, list) { + /* + * For the moment, errors are sticky: don't handle subsequent + * faults in the group if there is an error. + */ + if (status == IOMMU_PAGE_RESP_SUCCESS) + status = iopf_handle_single(iopf); + + if (!(iopf->fault.prm.flags & + IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) + kfree(iopf); + } + + iopf_complete_group(group->dev, &group->last_fault, status); + kfree(group); +} + +/** + * iommu_queue_iopf - IO Page Fault handler + * @fault: fault event + * @cookie: struct device, passed to iommu_register_device_fault_handler. + * + * Add a fault to the device workqueue, to be handled by mm. + * + * This module doesn't handle PCI PASID Stop Marker; IOMMU drivers must discard + * them before reporting faults. A PASID Stop Marker (LRW = 0b100) doesn't + * expect a response. It may be generated when disabling a PASID (issuing a + * PASID stop request) by some PCI devices. + * + * The PASID stop request is issued by the device driver before unbind(). Once + * it completes, no page request is generated for this PASID anymore and + * outstanding ones have been pushed to the IOMMU (as per PCIe 4.0r1.0 - 6.20.1 + * and 10.4.1.2 - Managing PASID TLP Prefix Usage). Some PCI devices will wait + * for all outstanding page requests to come back with a response before + * completing the PASID stop request. Others do not wait for page responses, and + * instead issue this Stop Marker that tells us when the PASID can be + * reallocated. + * + * It is safe to discard the Stop Marker because it is an optimization. + * a. Page requests, which are posted requests, have been flushed to the IOMMU + * when the stop request completes. + * b. The IOMMU driver flushes all fault queues on unbind() before freeing the + * PASID. + * + * So even though the Stop Marker might be issued by the device *after* the stop + * request completes, outstanding faults will have been dealt with by the time + * the PASID is freed. + * + * Return: 0 on success and <0 on error. + */ +int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) +{ + int ret; + struct iopf_group *group; + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + + struct device *dev = cookie; + struct dev_iommu *param = dev->iommu; + + lockdep_assert_held(¶m->lock); + + if (fault->type != IOMMU_FAULT_PAGE_REQ) + /* Not a recoverable page fault */ + return -EOPNOTSUPP; + + /* + * As long as we're holding param->lock, the queue can't be unlinked + * from the device and therefore cannot disappear. + */ + iopf_param = param->iopf_param; + if (!iopf_param) + return -ENODEV; + + if (!(fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) { + iopf = kzalloc(sizeof(*iopf), GFP_KERNEL); + if (!iopf) + return -ENOMEM; + + iopf->fault = *fault; + + /* Non-last request of a group. Postpone until the last one */ + list_add(&iopf->list, &iopf_param->partial); + + return 0; + } + + group = kzalloc(sizeof(*group), GFP_KERNEL); + if (!group) { + /* + * The caller will send a response to the hardware. But we do + * need to clean up before leaving, otherwise partial faults + * will be stuck. + */ + ret = -ENOMEM; + goto cleanup_partial; + } + + group->dev = dev; + group->last_fault.fault = *fault; + INIT_LIST_HEAD(&group->faults); + list_add(&group->last_fault.list, &group->faults); + INIT_WORK(&group->work, iopf_handle_group); + + /* See if we have partial faults for this group */ + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { + if (iopf->fault.prm.grpid == fault->prm.grpid) + /* Insert *before* the last fault */ + list_move(&iopf->list, &group->faults); + } + + queue_work(iopf_param->queue->wq, &group->work); + return 0; + +cleanup_partial: + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { + if (iopf->fault.prm.grpid == fault->prm.grpid) { + list_del(&iopf->list); + kfree(iopf); + } + } + return ret; +} +EXPORT_SYMBOL_GPL(iommu_queue_iopf); + +/** + * iopf_queue_flush_dev - Ensure that all queued faults have been processed + * @dev: the endpoint whose faults need to be flushed. + * + * The IOMMU driver calls this before releasing a PASID, to ensure that all + * pending faults for this PASID have been handled, and won't hit the address + * space of the next process that uses this PASID. The driver must make sure + * that no new fault is added to the queue. In particular it must flush its + * low-level queue before calling this function. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_flush_dev(struct device *dev) +{ + int ret = 0; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param) + return -ENODEV; + + mutex_lock(¶m->lock); + iopf_param = param->iopf_param; + if (iopf_param) + flush_workqueue(iopf_param->queue->wq); + else + ret = -ENODEV; + mutex_unlock(¶m->lock); + + return ret; +} +EXPORT_SYMBOL_GPL(iopf_queue_flush_dev); + +/** + * iopf_queue_discard_partial - Remove all pending partial fault + * @queue: the queue whose partial faults need to be discarded + * + * When the hardware queue overflows, last page faults in a group may have been + * lost and the IOMMU driver calls this to discard all partial faults. The + * driver shouldn't be adding new faults to this queue concurrently. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_discard_partial(struct iopf_queue *queue) +{ + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + + if (!queue) + return -EINVAL; + + mutex_lock(&queue->lock); + list_for_each_entry(iopf_param, &queue->devices, queue_list) { + list_for_each_entry_safe(iopf, next, &iopf_param->partial, + list) { + list_del(&iopf->list); + kfree(iopf); + } + } + mutex_unlock(&queue->lock); + return 0; +} +EXPORT_SYMBOL_GPL(iopf_queue_discard_partial); + +/** + * iopf_queue_add_device - Add producer to the fault queue + * @queue: IOPF queue + * @dev: device to add + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev) +{ + int ret = -EBUSY; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param) + return -ENODEV; + + iopf_param = kzalloc(sizeof(*iopf_param), GFP_KERNEL); + if (!iopf_param) + return -ENOMEM; + + INIT_LIST_HEAD(&iopf_param->partial); + iopf_param->queue = queue; + iopf_param->dev = dev; + + mutex_lock(&queue->lock); + mutex_lock(¶m->lock); + if (!param->iopf_param) { + list_add(&iopf_param->queue_list, &queue->devices); + param->iopf_param = iopf_param; + ret = 0; + } + mutex_unlock(¶m->lock); + mutex_unlock(&queue->lock); + + if (ret) + kfree(iopf_param); + + return ret; +} +EXPORT_SYMBOL_GPL(iopf_queue_add_device); + +/** + * iopf_queue_remove_device - Remove producer from fault queue + * @queue: IOPF queue + * @dev: device to remove + * + * Caller makes sure that no more faults are reported for this device. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_remove_device(struct iopf_queue *queue, struct device *dev) +{ + int ret = -EINVAL; + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param || !queue) + return -EINVAL; + + mutex_lock(&queue->lock); + mutex_lock(¶m->lock); + iopf_param = param->iopf_param; + if (iopf_param && iopf_param->queue == queue) { + list_del(&iopf_param->queue_list); + param->iopf_param = NULL; + ret = 0; + } + mutex_unlock(¶m->lock); + mutex_unlock(&queue->lock); + if (ret) + return ret; + + /* Just in case some faults are still stuck */ + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) + kfree(iopf); + + kfree(iopf_param); + + return 0; +} +EXPORT_SYMBOL_GPL(iopf_queue_remove_device); + +/** + * iopf_queue_alloc - Allocate and initialize a fault queue + * @name: a unique string identifying the queue (for workqueue) + * + * Return: the queue on success and NULL on error. + */ +struct iopf_queue *iopf_queue_alloc(const char *name) +{ + struct iopf_queue *queue; + + queue = kzalloc(sizeof(*queue), GFP_KERNEL); + if (!queue) + return NULL; + + /* + * The WQ is unordered because the low-level handler enqueues faults by + * group. PRI requests within a group have to be ordered, but once + * that's dealt with, the high-level function can handle groups out of + * order. + */ + queue->wq = alloc_workqueue("iopf_queue/%s", WQ_UNBOUND, 0, name); + if (!queue->wq) { + kfree(queue); + return NULL; + } + + INIT_LIST_HEAD(&queue->devices); + mutex_init(&queue->lock); + + return queue; +} +EXPORT_SYMBOL_GPL(iopf_queue_alloc); + +/** + * iopf_queue_free - Free IOPF queue + * @queue: queue to free + * + * Counterpart to iopf_queue_alloc(). The driver must not be queuing faults or + * adding/removing devices on this queue anymore. + */ +void iopf_queue_free(struct iopf_queue *queue) +{ + struct iopf_device_param *iopf_param, *next; + + if (!queue) + return; + + list_for_each_entry_safe(iopf_param, next, &queue->devices, queue_list) + iopf_queue_remove_device(queue, iopf_param->dev); + + destroy_workqueue(queue->wq); + kfree(queue); +} +EXPORT_SYMBOL_GPL(iopf_queue_free); From patchwork Mon Jan 25 11:06:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370572 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp4608987jam; Mon, 25 Jan 2021 20:43:49 -0800 (PST) X-Google-Smtp-Source: ABdhPJwtqXNNG9ZpeXRAPcgSHlf6o31mKbKoJVcFdg2356Ka/wBdcchpyyotkrhPOzfH5Syvx4AG X-Received: by 2002:a17:906:eb1b:: with SMTP id mb27mr2439836ejb.332.1611636229449; Mon, 25 Jan 2021 20:43:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611636229; cv=none; d=google.com; s=arc-20160816; b=yhg/QzophZQoPXm7POcayAa/wuRHt2+QMvjhyP2YIZkKaunZBrVfer4hiuH/uoUA0z uVige1tzE5Woad5+Bh8pJICWPgvBOVLPtCNzgFkX2LWPrdIUJqB2+tgSkQCZ11VzZVYH KYkb8VhlPELFQ7kyjSoS4QPz1LXghNoBB3vBYr9GOwYmBRakCRgrMs1Rza3nrqhxBFWj MxH6KGVAInN1KVfLGfTAl8cFFBT+AKYP1urbLiWE/06+ixVCAN8cBJFeJahJ5x88kf3P 2DT7tVCNj1BssrVCKNsf7G6wfweG53IjG4AAZEbrWCWjZTcwyak0oI1NSTXBiX2xSW2J nSsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Iza7ai+1pJVkOiREPpK3tUFt/+4KEOcT4kmdIoRu1W4=; b=WBh0D/80uMb3D3NwTzbIlhesH/OHw6vvIQ3k8OfdL2kLyPYvFNymITNNhD8FiqUMiJ G+tpsvC/N0a38kwm+fCimGtsDXslOUi+9W5beccUpIgu1hMdrShsSn6GM7lGFlAmEXP2 CPVGiqcIRVRAInA6Jpy8N3pF8eOv6PcWSdXPxExT026idkbqmFGIO61lwDuCTtj+HLJF TkU82bRTvyFWSbm5t9HPhOv57xBUYfeCJvtvbLQYZXoQNesBPrySm5oHph+3qIlEtKzz poQok74ZUj9mNkZE1VbJnOXYdNfjAR1KJl71eZB7l6jZJ4KlR4MuGNSFxHGbogwO/VnQ Qaag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fzAYzsIv; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b40si8719986edf.251.2021.01.25.20.43.49; Mon, 25 Jan 2021 20:43:49 -0800 (PST) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fzAYzsIv; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726983AbhAZEje (ORCPT + 6 others); Mon, 25 Jan 2021 23:39:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728313AbhAYMtz (ORCPT ); Mon, 25 Jan 2021 07:49:55 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4B7DC061355 for ; Mon, 25 Jan 2021 03:08:52 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id d16so11199498wro.11 for ; Mon, 25 Jan 2021 03:08:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Iza7ai+1pJVkOiREPpK3tUFt/+4KEOcT4kmdIoRu1W4=; b=fzAYzsIviLsnsdHkwnDD4zLK+G98O29ATNRd/oDswbowaY7Zr+nLaOcFcmXmf/bj2w sN5e/n3UT20Fr7KGxcyNYL303yAvJlogHiVtLgMj77MlPdSuYi0HnX/67z3hcp3eSnvq 6kNOn44rjrxRPk6uUBmXRSa0IA+4zGDWl9YWMyRArq4MyJGnCsb8/+uHR9Idp9Kiu6Lg 66pDSi3HhXCfpnVf2sDoKPzg4CMYQh1IAjWuFBrDf/8dfeBlJOKz1mZAKZXCVMehWrzE W2wpnBuYldvz0puu5+hXREtPgOQTBgA611k1XcR4jfdmMFLTxChzyjB04qECL/Q97yvB 9gqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Iza7ai+1pJVkOiREPpK3tUFt/+4KEOcT4kmdIoRu1W4=; b=S2Q7MQLZQsZXFJT8xZSpG+uAvYpcMmXx7x/5VlOtFr740MQssMduwhgmXABhIlNS9S 1Ka648wEItj5W1Pkf5SWs+stR6NWkoBiqljP4EVUEd35SNN1z66Xd3xWAqNEac0/20yh gorg48gLOHfrwxMhMv8FUtY2+1iMmhHRgISt9217kOPdjuSWTp21uIJHVarL1QyV1GR6 9KU17GnP/yGAXK3Xolp95HLrz0IvjoIJP6dXzNbOclWCYUmreKO59PyohQ4JWFqRYsZw wmQq0a8dvEsomvPIrBMVWJ8RPDGlpLKJPbCqL64WjxLIIIiS0jxJKnEmr8mF1g03QWzi CKTQ== X-Gm-Message-State: AOAM530icCmRd9J2ase6UQR7pBnpBtNyxeQhxneAabgJT9amXL+OAhjq r0owZ+HiFvSf8ZAk8qQZl183LA== X-Received: by 2002:adf:b1da:: with SMTP id r26mr348225wra.198.1611572931725; Mon, 25 Jan 2021 03:08:51 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:51 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , Rob Herring Subject: [PATCH v11 08/10] dt-bindings: document stall property for IOMMU masters Date: Mon, 25 Jan 2021 12:06:49 +0100 Message-Id: <20210125110650.3232195-9-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org On ARM systems, some platform devices behind an IOMMU may support stall, which is the ability to recover from page faults. Let the firmware tell us when a device supports stall. Reviewed-by: Rob Herring Signed-off-by: Jean-Philippe Brucker --- .../devicetree/bindings/iommu/iommu.txt | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) -- 2.30.0 diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt b/Documentation/devicetree/bindings/iommu/iommu.txt index 3c36334e4f94..26ba9e530f13 100644 --- a/Documentation/devicetree/bindings/iommu/iommu.txt +++ b/Documentation/devicetree/bindings/iommu/iommu.txt @@ -92,6 +92,24 @@ Optional properties: tagging DMA transactions with an address space identifier. By default, this is 0, which means that the device only has one address space. +- dma-can-stall: When present, the master can wait for a transaction to + complete for an indefinite amount of time. Upon translation fault some + IOMMUs, instead of aborting the translation immediately, may first + notify the driver and keep the transaction in flight. This allows the OS + to inspect the fault and, for example, make physical pages resident + before updating the mappings and completing the transaction. Such IOMMU + accepts a limited number of simultaneous stalled transactions before + having to either put back-pressure on the master, or abort new faulting + transactions. + + Firmware has to opt-in stalling, because most buses and masters don't + support it. In particular it isn't compatible with PCI, where + transactions have to complete before a time limit. More generally it + won't work in systems and masters that haven't been designed for + stalling. For example the OS, in order to handle a stalled transaction, + may attempt to retrieve pages from secondary storage in a stalled + domain, leading to a deadlock. + Notes: ====== From patchwork Mon Jan 25 11:06:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370573 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp4609009jam; Mon, 25 Jan 2021 20:43:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJzQmTzm9KbRjdL67Cq9N+zNv4tefQp59dV04FBuaR5LvNRaMRqJts0jsI1weUgoa7+wdJUC X-Received: by 2002:a17:907:1010:: with SMTP id ox16mr2225321ejb.467.1611636231373; Mon, 25 Jan 2021 20:43:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611636231; cv=none; d=google.com; s=arc-20160816; b=iUf7oiibrEOvvicniPxhA9Xyd1f7/oGXgbFWNWTVO5rlGqmv6SPG8YRJ74x3REfnPM 4KVyEcKAa8ZZdIgi5u6dRSbZqrKjSbBLbAOCJ3xHbRTLVXrRYlDGgxcHlcUFHbYbb+iQ hK8XfhaqrsvpYF5duORLepzIlTrYiSyye0qZsbvJklGvu4thJ8pYjmbqvmjyVa9JHZIR kFf7Yi0YrVtRG+RnkUVDEVZjIGpWs/jwXoa3d/CHNaoSYdEG4QP6z4Ap+RKmQ87AtMN+ bxwuQ4yJGzYqWEQMNWDj1xjuB211MYgQXM1kiBc+kb3TznGtL4yYWTKGrD868JdD4aJb fIfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=P/EAy3wFo2SkAvzlEqANsfgzLeO0MB9eAURTmLNsvNA=; b=MGuUTzrzAJPm+Ed1FykbxWgI/Dr4K09Yo2G5uczCYX5+b2C6tn1AJcsmv+Zg9bfAv7 qiAXDaPcR0tpoIELz/DRUUU99Q7EMVV0X7UmwpA/97zgmvTouDF2bVJOodofgk412Fmn YrTfxqYNLsQoeeWfKPq14nNxBhTZNAp2VilxRRUG8dIPvyNs8RF50YkyAxQLqgNBqlM9 aTFROq1Fp+hMhHr7UT6GDHVyA25VF/rMN1D/j6bW2ggiSnFH2QBRIEk6MZW50gvo3C1m 6tMzkJXShIYfWeHbuJaaX/zJ4y6CqxQDq2DvNFBTxh9aAOx6UfWL15f8fXbv4DwuWpoa 0ZCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hn7k9YBr; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b40si8719986edf.251.2021.01.25.20.43.51; Mon, 25 Jan 2021 20:43:51 -0800 (PST) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hn7k9YBr; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727002AbhAZEjp (ORCPT + 6 others); Mon, 25 Jan 2021 23:39:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728319AbhAYMt6 (ORCPT ); Mon, 25 Jan 2021 07:49:58 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42273C061356 for ; Mon, 25 Jan 2021 03:08:54 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id b5so11864894wrr.10 for ; Mon, 25 Jan 2021 03:08:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P/EAy3wFo2SkAvzlEqANsfgzLeO0MB9eAURTmLNsvNA=; b=hn7k9YBrpbYNaKrAacJHgdQijWZghUPi3Iv6zU06UxFqaoM6CS1UbqtknoP0o2EYtg B4SlQ+8/0UDOp6kFcPYo7zeG7dDFCiQ42Bf8cd62mIK1oyjO5yqUyY5/Z9NO1CCEjIqF /AlTjnPryDQ833VP2jsLn7eScdSvT5H1N/afnru2UDmPA/nsLXS7u3+txOfIzwwlT2ED ixEkaN2yIDalNKbCKVn5d/Ji12naseZlZUxo+FJcEb0k1M0QCSwvdalS8aIAj+bOeTx2 Bh2aW/dDFeqvnxAjNunComt/RSTHyfjg+DFVN85ttExE9Q2MlJKK4QEsbi23b+YkcRQH lsJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P/EAy3wFo2SkAvzlEqANsfgzLeO0MB9eAURTmLNsvNA=; b=D8dSU41pXH3hwVzJlz2eszSQDGzfIcIcXy9AHQaUeaZdk9lQsDrTdbOT0Kx0aMy4aE KIqoKpheB4/TdzxyORW+Sg8tTRKd7GfLkfKmArqghSJbKx/rVrBBwwJZIrZAE6pVxPEN o9nSotRx4Ms89juzJLS+DaS/F8qE+VhqPU18/Ar+zKij22mYcT3RVJ1at7JFxpbVWDG6 K1wDHPJ8wbxr/SWdgEFbCPU0sFQXvJGxMpvpLvF1YvtfV81GTdP8MjYqj5XjUXQNa0bZ +Lua4wJMHpEUYozTp3HnTW+6cUU0SOt+4c2nuSv7HDSdwY7GzGYysGQYq313p0vyRbtB kXhg== X-Gm-Message-State: AOAM532PeTwCWHYhlDzzPlLse6iZqxiVT0W60oC0G+Xz3a2VDWaxHVmN gFWtD8590N8HrQiwtkd2nFI7dg== X-Received: by 2002:adf:eed0:: with SMTP id a16mr293691wrp.107.1611572932965; Mon, 25 Jan 2021 03:08:52 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:52 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v11 09/10] ACPI/IORT: Enable stall support for platform devices Date: Mon, 25 Jan 2021 12:06:50 +0100 Message-Id: <20210125110650.3232195-10-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Copy the "Stall supported" bit, that tells whether a named component supports stall, into the dma-can-stall device property. Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- drivers/acpi/arm64/iort.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 2.30.0 diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index c9a8bbb74b09..42820d7eb869 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -968,13 +968,15 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data) static void iort_named_component_init(struct device *dev, struct acpi_iort_node *node) { - struct property_entry props[2] = {}; + struct property_entry props[3] = {}; struct acpi_iort_named_component *nc; nc = (struct acpi_iort_named_component *)node->node_data; props[0] = PROPERTY_ENTRY_U32("pasid-num-bits", FIELD_GET(ACPI_IORT_NC_PASID_BITS, nc->node_flags)); + if (nc->node_flags & ACPI_IORT_NC_STALL_SUPPORTED) + props[1] = PROPERTY_ENTRY_BOOL("dma-can-stall"); if (device_add_properties(dev, props)) dev_warn(dev, "Could not add device properties\n"); From patchwork Mon Jan 25 11:06:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 370574 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp4609789jam; Mon, 25 Jan 2021 20:45:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJxg2GJyO3IameKRQ5DK9uv9ndE97PqHzwwEjVJyK76BFpXx4L0ogoQRUQh9Jh056YBku6S7 X-Received: by 2002:a17:906:688c:: with SMTP id n12mr2429351ejr.111.1611636232269; Mon, 25 Jan 2021 20:43:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611636232; cv=none; d=google.com; s=arc-20160816; b=HEShfVKXY5xnE9XLKKH8QKu6CRzQDLwvpoIZmP3feD+kQz7/1RttEXCedulmijQ2DL zv6JFFF1wUfN2tGPrGmZSqjMxibqlbo8EzHkc8Ofw51w4KVD0pbPYJHOUgm1bauoCOip 9jfiAeP+dxRFAx1WNbFmk/LbGNi4xs+cGXUm3USLZNwul5GQNidXwNmSoQHE5e4hKHPe /CBohOXBhmVQhPAPt/6YmxN9MNB6Tqx34n0ZRlXM1Wg6C5TiMxQV7Oj3e6ZCLlrmvo++ 64WNbBbSdPca0mKTic7IXp4/ZDE16tElMAEnjMmaLOztFnrJIoQEq9QaD2sMIttGY8bD h8JQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hAgkyBWdOaKduInm933KULZarY4dk8dJFcqwaBpEgH0=; b=Tm8oBABzP+tYJu4nEvHXMRepbh85rqNYhS5ENU+KoCnOl6EpsOPXc9CONTQrUAbzsX xk6UxB8tOymMpWDSjVDQzNUOjnq47Q8UIZU0hC1pFvQ6tAoLVbp7MRF3PE6ADy5O2diJ vUvG9KFY4UMbTMoy56/E9xpmdgczKOh1l/CDzZ+4yKj+8UAJ91UiFcizgxtLW3tZbJfV lAe8HqIL1+/SVaIZau+oeA6Zz/TStZhkRs5dNQElqEuycnNq65h5Mo2L4eJnxDl/aWIP rUHplVKJtrWIZjoYUTshD9213IDn897Q1L+iZRh8DNS2mO9WzlkHW7EOCrdXieP5xj59 1cuw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jk9jg9RJ; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b40si8719986edf.251.2021.01.25.20.43.51; Mon, 25 Jan 2021 20:43:52 -0800 (PST) Received-SPF: pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jk9jg9RJ; spf=pass (google.com: domain of devicetree-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=devicetree-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726725AbhAZEkA (ORCPT + 6 others); Mon, 25 Jan 2021 23:40:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728424AbhAYMwE (ORCPT ); Mon, 25 Jan 2021 07:52:04 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9357EC06121D for ; Mon, 25 Jan 2021 03:08:55 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id f16so1362888wmq.5 for ; Mon, 25 Jan 2021 03:08:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hAgkyBWdOaKduInm933KULZarY4dk8dJFcqwaBpEgH0=; b=jk9jg9RJgLQJfo5RG8VKGf1xaSt+18wP2OoaGLgtLQGWPVKNLSfupzGpmfO30j7iXh MoCiBNhroPNFvjtAVxNV6wWQtmb2dBbjD+DN3ij3hCT+wOXxwX/6HC/xDfqmZ14tiQJi AaZ4ipsIUtpC86zMN7VOH+Apr34Vb9zPBjy1Anetu2/Da8YM1TIqq2t/min+PxofUXpt a7V109GMf/MZjEsaUc5m2CM/DsWfC4IH5mwmNiZ4YJwMRT9KlE20RLsA7FD2fs/n/Suk kA1/B4oYy2z75BwSC7KcKIl5/yBmOfVXT5QwFQC7pMX6sqzekcf+HlSwrCn0G0oPiEIu 5LlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hAgkyBWdOaKduInm933KULZarY4dk8dJFcqwaBpEgH0=; b=CsCSS8Svvuw+cZCzpq+ZA2RFTc4sV8D5EJqcHprgb+qrkbS2tow8i3uuwbzi5E1Eks nY4jX4cNM5UOjPw8a3GrD0u9l6F7ppKBKwCFli4BfD5GcQA+zYGmZDvNB9Olwgzx5Fnw TZB5AEXqsNY0BW4alblavaU9wmZ2cJhrKOpc+Fk8bvfdpUM4STHpcqLUBjIncDvdX+KO CNRvOhhhmQGf+ghjlbe9A9cOrKM6+7BuCoWS9l5YljTPx7R5ErlM3gbNCs1nZ468xW3U 7SZOmWyJ+willpakc5A5T7ejrrS0Jxvn4JC2PZXOf6I08QTI4Nec5+xc7bY72wfq+u0A Tzjw== X-Gm-Message-State: AOAM532k23xBV57nHOpa5q468LnXCbdZngrqtm/qNAF5PaOp8ROHIz+z avoBcC2Ga02QN0yfMAhtmYYWJA== X-Received: by 2002:a05:600c:2110:: with SMTP id u16mr15392234wml.65.1611572934266; Mon, 25 Jan 2021 03:08:54 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id u6sm16636014wro.75.2021.01.25.03.08.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 03:08:53 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v11 10/10] iommu/arm-smmu-v3: Add stall support for platform devices Date: Mon, 25 Jan 2021 12:06:51 +0100 Message-Id: <20210125110650.3232195-11-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210125110650.3232195-1-jean-philippe@linaro.org> References: <20210125110650.3232195-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org The SMMU provides a Stall model for handling page faults in platform devices. It is similar to PCIe PRI, but doesn't require devices to have their own translation cache. Instead, faulting transactions are parked and the OS is given a chance to fix the page tables and retry the transaction. Enable stall for devices that support it (opt-in by firmware). When an event corresponds to a translation error, call the IOMMU fault handler. If the fault is recoverable, it will call us back to terminate or continue the stall. To use stall device drivers need to enable IOMMU_DEV_FEAT_IOPF, which initializes the fault queue for the device. Tested-by: Zhangfei Gao Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 43 ++++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 59 +++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 187 +++++++++++++++++- 3 files changed, 274 insertions(+), 15 deletions(-) -- 2.30.0 Reviewed-by: Jonathan Cameron diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 7b15b7580c6e..59af0bbd2f7b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -354,6 +354,13 @@ #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) #define CMDQ_PRI_1_RESP GENMASK_ULL(13, 12) +#define CMDQ_RESUME_0_RESP_TERM 0UL +#define CMDQ_RESUME_0_RESP_RETRY 1UL +#define CMDQ_RESUME_0_RESP_ABORT 2UL +#define CMDQ_RESUME_0_RESP GENMASK_ULL(13, 12) +#define CMDQ_RESUME_0_SID GENMASK_ULL(63, 32) +#define CMDQ_RESUME_1_STAG GENMASK_ULL(15, 0) + #define CMDQ_SYNC_0_CS GENMASK_ULL(13, 12) #define CMDQ_SYNC_0_CS_NONE 0 #define CMDQ_SYNC_0_CS_IRQ 1 @@ -370,6 +377,25 @@ #define EVTQ_0_ID GENMASK_ULL(7, 0) +#define EVT_ID_TRANSLATION_FAULT 0x10 +#define EVT_ID_ADDR_SIZE_FAULT 0x11 +#define EVT_ID_ACCESS_FAULT 0x12 +#define EVT_ID_PERMISSION_FAULT 0x13 + +#define EVTQ_0_SSV (1UL << 11) +#define EVTQ_0_SSID GENMASK_ULL(31, 12) +#define EVTQ_0_SID GENMASK_ULL(63, 32) +#define EVTQ_1_STAG GENMASK_ULL(15, 0) +#define EVTQ_1_STALL (1UL << 31) +#define EVTQ_1_PnU (1UL << 33) +#define EVTQ_1_InD (1UL << 34) +#define EVTQ_1_RnW (1UL << 35) +#define EVTQ_1_S2 (1UL << 39) +#define EVTQ_1_CLASS GENMASK_ULL(41, 40) +#define EVTQ_1_TT_READ (1UL << 44) +#define EVTQ_2_ADDR GENMASK_ULL(63, 0) +#define EVTQ_3_IPA GENMASK_ULL(51, 12) + /* PRI queue */ #define PRIQ_ENT_SZ_SHIFT 4 #define PRIQ_ENT_DWORDS ((1 << PRIQ_ENT_SZ_SHIFT) >> 3) @@ -464,6 +490,13 @@ struct arm_smmu_cmdq_ent { enum pri_resp resp; } pri; + #define CMDQ_OP_RESUME 0x44 + struct { + u32 sid; + u16 stag; + u8 resp; + } resume; + #define CMDQ_OP_CMD_SYNC 0x46 struct { u64 msiaddr; @@ -522,6 +555,7 @@ struct arm_smmu_cmdq_batch { struct arm_smmu_evtq { struct arm_smmu_queue q; + struct iopf_queue *iopf; u32 max_stalls; }; @@ -659,7 +693,9 @@ struct arm_smmu_master { struct arm_smmu_stream *streams; unsigned int num_streams; bool ats_enabled; + bool stall_enabled; bool sva_enabled; + bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; }; @@ -678,6 +714,7 @@ struct arm_smmu_domain { struct io_pgtable_ops *pgtbl_ops; bool non_strict; + bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -719,6 +756,7 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master *master); bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); +bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata); void arm_smmu_sva_unbind(struct iommu_sva *handle); @@ -750,6 +788,11 @@ static inline int arm_smmu_master_disable_sva(struct arm_smmu_master *master) return -ENODEV; } +static inline bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master) +{ + return false; +} + static inline struct iommu_sva * arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index bb251cab61f3..ee66d1f4cb81 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -435,9 +435,13 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) return true; } -static bool arm_smmu_iopf_supported(struct arm_smmu_master *master) +bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master) { - return false; + /* We're not keeping track of SIDs in fault events */ + if (master->num_streams != 1) + return false; + + return master->stall_enabled; } bool arm_smmu_master_sva_supported(struct arm_smmu_master *master) @@ -445,8 +449,8 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master *master) if (!(master->smmu->features & ARM_SMMU_FEAT_SVA)) return false; - /* SSID and IOPF support are mandatory for the moment */ - return master->ssid_bits && arm_smmu_iopf_supported(master); + /* SSID support is mandatory for the moment */ + return master->ssid_bits; } bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master) @@ -459,13 +463,55 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master) return enabled; } +static int arm_smmu_master_sva_enable_iopf(struct arm_smmu_master *master) +{ + int ret; + struct device *dev = master->dev; + + /* + * Drivers for devices supporting PRI or stall should enable IOPF first. + * Others have device-specific fault handlers and don't need IOPF. + */ + if (!arm_smmu_master_iopf_supported(master)) + return 0; + + if (!master->iopf_enabled) + return -EINVAL; + + ret = iopf_queue_add_device(master->smmu->evtq.iopf, dev); + if (ret) + return ret; + + ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); + if (ret) { + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + return ret; + } + return 0; +} + +static void arm_smmu_master_sva_disable_iopf(struct arm_smmu_master *master) +{ + struct device *dev = master->dev; + + if (!master->iopf_enabled) + return; + + iommu_unregister_device_fault_handler(dev); + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); +} + int arm_smmu_master_enable_sva(struct arm_smmu_master *master) { + int ret; + mutex_lock(&sva_lock); - master->sva_enabled = true; + ret = arm_smmu_master_sva_enable_iopf(master); + if (!ret) + master->sva_enabled = true; mutex_unlock(&sva_lock); - return 0; + return ret; } int arm_smmu_master_disable_sva(struct arm_smmu_master *master) @@ -476,6 +522,7 @@ int arm_smmu_master_disable_sva(struct arm_smmu_master *master) mutex_unlock(&sva_lock); return -EBUSY; } + arm_smmu_master_sva_disable_iopf(master); master->sva_enabled = false; mutex_unlock(&sva_lock); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3afec6ed8075..39a59d297c3c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -32,6 +32,7 @@ #include #include "arm-smmu-v3.h" +#include "../../iommu-sva-lib.h" static bool disable_bypass = true; module_param(disable_bypass, bool, 0444); @@ -315,6 +316,11 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) } cmd[1] |= FIELD_PREP(CMDQ_PRI_1_RESP, ent->pri.resp); break; + case CMDQ_OP_RESUME: + cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_SID, ent->resume.sid); + cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_RESP, ent->resume.resp); + cmd[1] |= FIELD_PREP(CMDQ_RESUME_1_STAG, ent->resume.stag); + break; case CMDQ_OP_CMD_SYNC: if (ent->sync.msiaddr) { cmd[0] |= FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_IRQ); @@ -878,6 +884,44 @@ static int arm_smmu_cmdq_batch_submit(struct arm_smmu_device *smmu, return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmds, cmds->num, true); } +static int arm_smmu_page_response(struct device *dev, + struct iommu_fault_event *unused, + struct iommu_page_response *resp) +{ + struct arm_smmu_cmdq_ent cmd = {0}; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + int sid = master->streams[0].id; + + if (master->stall_enabled) { + cmd.opcode = CMDQ_OP_RESUME; + cmd.resume.sid = sid; + cmd.resume.stag = resp->grpid; + switch (resp->code) { + case IOMMU_PAGE_RESP_INVALID: + case IOMMU_PAGE_RESP_FAILURE: + cmd.resume.resp = CMDQ_RESUME_0_RESP_ABORT; + break; + case IOMMU_PAGE_RESP_SUCCESS: + cmd.resume.resp = CMDQ_RESUME_0_RESP_RETRY; + break; + default: + return -EINVAL; + } + } else { + return -ENODEV; + } + + arm_smmu_cmdq_issue_cmd(master->smmu, &cmd); + /* + * Don't send a SYNC, it doesn't do anything for RESUME or PRI_RESP. + * RESUME consumption guarantees that the stalled transaction will be + * terminated... at some point in the future. PRI_RESP is fire and + * forget. + */ + + return 0; +} + /* Context descriptor manipulation functions */ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) { @@ -988,7 +1032,6 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, u64 val; bool cd_live; __le64 *cdptr; - struct arm_smmu_device *smmu = smmu_domain->smmu; if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) return -E2BIG; @@ -1033,8 +1076,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - /* STALL_MODEL==0b10 && CD.S==0 is ILLEGAL */ - if (smmu->features & ARM_SMMU_FEAT_STALL_FORCE) + if (smmu_domain->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1278,7 +1320,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, FIELD_PREP(STRTAB_STE_1_STRW, strw)); if (smmu->features & ARM_SMMU_FEAT_STALLS && - !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) + !master->stall_enabled) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); val |= (s1_cfg->cdcfg.cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) | @@ -1355,7 +1397,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) return 0; } -__maybe_unused static struct arm_smmu_master * arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) { @@ -1382,9 +1423,96 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) } /* IRQ and event handlers */ +static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) +{ + int ret; + u32 reason; + u32 perm = 0; + struct arm_smmu_master *master; + bool ssid_valid = evt[0] & EVTQ_0_SSV; + u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]); + struct iommu_fault_event fault_evt = { }; + struct iommu_fault *flt = &fault_evt.fault; + + /* Stage-2 is always pinned at the moment */ + if (evt[1] & EVTQ_1_S2) + return -EFAULT; + + master = arm_smmu_find_master(smmu, sid); + if (!master) + return -EINVAL; + + if (evt[1] & EVTQ_1_RnW) + perm |= IOMMU_FAULT_PERM_READ; + else + perm |= IOMMU_FAULT_PERM_WRITE; + + if (evt[1] & EVTQ_1_InD) + perm |= IOMMU_FAULT_PERM_EXEC; + + if (evt[1] & EVTQ_1_PnU) + perm |= IOMMU_FAULT_PERM_PRIV; + + switch (FIELD_GET(EVTQ_0_ID, evt[0])) { + case EVT_ID_TRANSLATION_FAULT: + case EVT_ID_ADDR_SIZE_FAULT: + case EVT_ID_ACCESS_FAULT: + reason = IOMMU_FAULT_REASON_PTE_FETCH; + break; + case EVT_ID_PERMISSION_FAULT: + reason = IOMMU_FAULT_REASON_PERMISSION; + break; + default: + return -EOPNOTSUPP; + } + + if (evt[1] & EVTQ_1_STALL) { + flt->type = IOMMU_FAULT_PAGE_REQ; + flt->prm = (struct iommu_fault_page_request) { + .flags = IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE, + .grpid = FIELD_GET(EVTQ_1_STAG, evt[1]), + .perm = perm, + .addr = FIELD_GET(EVTQ_2_ADDR, evt[2]), + }; + + if (ssid_valid) { + flt->prm.flags |= IOMMU_FAULT_PAGE_REQUEST_PASID_VALID; + flt->prm.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]); + } + } else { + flt->type = IOMMU_FAULT_DMA_UNRECOV; + flt->event = (struct iommu_fault_unrecoverable) { + .reason = reason, + .flags = IOMMU_FAULT_UNRECOV_ADDR_VALID | + IOMMU_FAULT_UNRECOV_FETCH_ADDR_VALID, + .perm = perm, + .addr = FIELD_GET(EVTQ_2_ADDR, evt[2]), + .fetch_addr = FIELD_GET(EVTQ_3_IPA, evt[3]), + }; + + if (ssid_valid) { + flt->event.flags |= IOMMU_FAULT_UNRECOV_PASID_VALID; + flt->event.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]); + } + } + + ret = iommu_report_device_fault(master->dev, &fault_evt); + if (ret && flt->type == IOMMU_FAULT_PAGE_REQ) { + /* Nobody cared, abort the access */ + struct iommu_page_response resp = { + .pasid = flt->prm.pasid, + .grpid = flt->prm.grpid, + .code = IOMMU_PAGE_RESP_FAILURE, + }; + arm_smmu_page_response(master->dev, &fault_evt, &resp); + } + + return ret; +} + static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { - int i; + int i, ret; struct arm_smmu_device *smmu = dev; struct arm_smmu_queue *q = &smmu->evtq.q; struct arm_smmu_ll_queue *llq = &q->llq; @@ -1394,6 +1522,10 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) while (!queue_remove_raw(q, evt)) { u8 id = FIELD_GET(EVTQ_0_ID, evt[0]); + ret = arm_smmu_handle_evt(smmu, evt); + if (!ret) + continue; + dev_info(smmu->dev, "event 0x%02x received:\n", id); for (i = 0; i < ARRAY_SIZE(evt); ++i) dev_info(smmu->dev, "\t0x%016llx\n", @@ -1928,6 +2060,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, cfg->s1cdmax = master->ssid_bits; + smmu_domain->stall_enabled = master->stall_enabled; + ret = arm_smmu_alloc_cd_tables(smmu_domain); if (ret) goto out_free_asid; @@ -2275,6 +2409,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu_domain->s1_cfg.s1cdmax, master->ssid_bits); ret = -EINVAL; goto out_unlock; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && + smmu_domain->stall_enabled != master->stall_enabled) { + dev_err(dev, "cannot attach to stall-%s domain\n", + smmu_domain->stall_enabled ? "enabled" : "disabled"); + ret = -EINVAL; + goto out_unlock; } master->domain = smmu_domain; @@ -2510,6 +2650,11 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->ssid_bits = min_t(u8, master->ssid_bits, CTXDESC_LINEAR_CDMAX); + if ((smmu->features & ARM_SMMU_FEAT_STALLS && + device_property_read_bool(dev, "dma-can-stall")) || + smmu->features & ARM_SMMU_FEAT_STALL_FORCE) + master->stall_enabled = true; + return &smmu->iommu; err_free_master: @@ -2527,7 +2672,8 @@ static void arm_smmu_release_device(struct device *dev) return; master = dev_iommu_priv_get(dev); - WARN_ON(arm_smmu_master_sva_enabled(master)); + if (WARN_ON(arm_smmu_master_sva_enabled(master))) + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); @@ -2655,6 +2801,8 @@ static bool arm_smmu_dev_has_feature(struct device *dev, return false; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + return arm_smmu_master_iopf_supported(master); case IOMMU_DEV_FEAT_SVA: return arm_smmu_master_sva_supported(master); default: @@ -2671,6 +2819,8 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev, return false; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + return master->iopf_enabled; case IOMMU_DEV_FEAT_SVA: return arm_smmu_master_sva_enabled(master); default: @@ -2681,6 +2831,8 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev, static int arm_smmu_dev_enable_feature(struct device *dev, enum iommu_dev_features feat) { + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + if (!arm_smmu_dev_has_feature(dev, feat)) return -ENODEV; @@ -2688,8 +2840,11 @@ static int arm_smmu_dev_enable_feature(struct device *dev, return -EBUSY; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + master->iopf_enabled = true; + return 0; case IOMMU_DEV_FEAT_SVA: - return arm_smmu_master_enable_sva(dev_iommu_priv_get(dev)); + return arm_smmu_master_enable_sva(master); default: return -EINVAL; } @@ -2698,12 +2853,17 @@ static int arm_smmu_dev_enable_feature(struct device *dev, static int arm_smmu_dev_disable_feature(struct device *dev, enum iommu_dev_features feat) { + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + if (!arm_smmu_dev_feature_enabled(dev, feat)) return -EINVAL; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + master->iopf_enabled = false; + return 0; case IOMMU_DEV_FEAT_SVA: - return arm_smmu_master_disable_sva(dev_iommu_priv_get(dev)); + return arm_smmu_master_disable_sva(master); default: return -EINVAL; } @@ -2734,6 +2894,7 @@ static struct iommu_ops arm_smmu_ops = { .sva_bind = arm_smmu_sva_bind, .sva_unbind = arm_smmu_sva_unbind, .sva_get_pasid = arm_smmu_sva_get_pasid, + .page_response = arm_smmu_page_response, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; @@ -2831,6 +2992,13 @@ static int arm_smmu_init_queues(struct arm_smmu_device *smmu) if (ret) return ret; + if ((smmu->features & ARM_SMMU_FEAT_SVA) && + (smmu->features & ARM_SMMU_FEAT_STALLS)) { + smmu->evtq.iopf = iopf_queue_alloc(dev_name(smmu->dev)); + if (!smmu->evtq.iopf) + return -ENOMEM; + } + /* priq */ if (!(smmu->features & ARM_SMMU_FEAT_PRI)) return 0; @@ -3746,6 +3914,7 @@ static int arm_smmu_device_remove(struct platform_device *pdev) iommu_device_unregister(&smmu->iommu); iommu_device_sysfs_remove(&smmu->iommu); arm_smmu_device_disable(smmu); + iopf_queue_free(smmu->evtq.iopf); return 0; }