From patchwork Wed Jan 27 15:43:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371560 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp385826jam; Wed, 27 Jan 2021 07:47:36 -0800 (PST) X-Google-Smtp-Source: ABdhPJxk6bUjqRRR7NuI0WMRsAEv6cJ+hJz5hJX2OuYWFlcv+B0/bb+ba9AVRGH7xpzzwIOmP/Vo X-Received: by 2002:a17:906:4b48:: with SMTP id j8mr7147055ejv.112.1611762456435; Wed, 27 Jan 2021 07:47:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611762456; cv=none; d=google.com; s=arc-20160816; b=hh54cfNIMfwnzJmuFygTkey4DCWFfkN17r7hg0lAOZbpg7t4yqzuP1jO0+k8g4uT3P fwJVzfUqNwEVRA/qK4SelscfpVsvl8dIDjUCaRCcZ88BTPRhTNz7TCas6mbOXIaD3xr9 QDC+qitsZUeWVCFKC9Gm3ceefRj3jwCjKpb2JHsbZIkaETJa2GemmM3sxKzN9VU8j+0W 8ZHoA/gnudWmvQG7M1tzwttknq9yWEsrFbIJ2owaOvpJgLk2BIgrAbxKbaBwDkP9u/62 W4Je04v8phRH3e7tyEiEjj9SUR6Tt9VjVkwzuXSC7A2C3dC9ZukRpP1w/YL//Mh/L8DP g+kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0PBUJkI6WkOQ4c7ebH11XhAeKbkF/R51HzjghE20E74=; b=Pn4et33S3+GLBcVjt29jfZ/NXRl0vAwiRb/NFF3bCf5qJ/h+DNRib/oLZn91hNTjAN Jl42ZYQR46Sio+3xajzwaIQWAA8Ay8koS04PGhnldltM/A5kIZUic2jhr3Y2af2HkO6t X9MzKMgJKZFHh/YAFRo6eUot1MNwQ7BfIDP3mLAburIe43MqKdFGk34Z98rkdYdKGxEz oD2H3LpChK0iNGocKy8ZFKKlNI7HPFG+dYxvfFI9P2SQCK5BsDrcOkM0/YMZVlOg53ZK a/VtbHrKRUPLCX1TEtH8T5ExK+7bfwr9gceSyr8YyxNPWO3IdVeHydqi0XITXekZPmQj jivg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BQJC3Huz; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cy17si1201994edb.193.2021.01.27.07.47.36; Wed, 27 Jan 2021 07:47:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BQJC3Huz; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236566AbhA0PrA (ORCPT + 4 others); Wed, 27 Jan 2021 10:47:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236555AbhA0PpQ (ORCPT ); Wed, 27 Jan 2021 10:45:16 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 184C4C061573 for ; Wed, 27 Jan 2021 07:44:36 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id m187so1919136wme.2 for ; Wed, 27 Jan 2021 07:44:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0PBUJkI6WkOQ4c7ebH11XhAeKbkF/R51HzjghE20E74=; b=BQJC3HuzpSzPBhYU8dAoIE4ZFOsaKvTTyDozAsFM/UlXGrWhBWsG1EBwunnAFsQ7fi Cx6syUteFUWRkMbIUgIP/7mUtZnmYFakuGW4SsO2HYY8yC3V29+hRjP6BBmROUg7LaQI aqsoQl5WG6wx9FDljC24Hkkr2MSU5/DJAJ6zVS02cIGN54trZqTVDl7jf2gKnTFxcFsR brLgpNCFXSoX9613SmJxNT49S2bJcbAxoyN26EEtgxLXXUnadPwjEgY/e506opSx3820 uNXoF0qCGnGL5/tkm1T76snDHbiUJrhNkBBpAuFEKzUgjQ+DiexDYpHVV90v9UMZ47Il uBZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0PBUJkI6WkOQ4c7ebH11XhAeKbkF/R51HzjghE20E74=; b=LC4kQPunoMD+gxRjMV1opMs533/ULRKdkf2Chj9cn+yyRucZNy1eJM+/rg24YzMbVi hDsN33oztxdg5xIsplMFJkjQBUbbvOXWhrhFQurLHEowaD+AUlAXMpFjIfkd/9ZPKd7W WoybKtBQZNIrWmgqhqrd98o3WVeyNSOCOoXQQL1kGdP3tYqpltN/G2okZEGOCkA8kevd 9fRVasqnqoYFyHsDxIg3+btmVq8903ZghEumR4XUiQnEsID68hCND37mhmgaaOKcrx6c c9OPCnYOQJJS7nMQg0hakt1DltkI9O0KW/E8Dlw1b3MPcWyGPCYpDzVO2glL9GD4Q8OW e4Tg== X-Gm-Message-State: AOAM533pmydvqhuMe4eQn1Yg58XfUIFDQF1C7SvfHOkoBWDFbliW6U/P NzUImTORzLIv/jLdSVX6vD95kw== X-Received: by 2002:a1c:cc14:: with SMTP id h20mr4755144wmb.180.1611762274755; Wed, 27 Jan 2021 07:44:34 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:34 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v12 01/10] iommu: Fix comment for struct iommu_fwspec Date: Wed, 27 Jan 2021 16:43:14 +0100 Message-Id: <20210127154322.3959196-2-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Commit 986d5ecc5699 ("iommu: Move fwspec->iommu_priv to struct dev_iommu") removed iommu_priv from fwspec and commit 5702ee24182f ("ACPI/IORT: Check ATS capability in root complex nodes") added @flags. Update the struct doc. Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- include/linux/iommu.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.30.0 diff --git a/include/linux/iommu.h b/include/linux/iommu.h index b3f0e2018c62..bdf3f34a4457 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -570,7 +570,7 @@ struct iommu_group *fsl_mc_device_group(struct device *dev); * struct iommu_fwspec - per-device IOMMU instance data * @ops: ops for this device's IOMMU * @iommu_fwnode: firmware handle for this device's IOMMU - * @iommu_priv: IOMMU driver private data for this device + * @flags: IOMMU_FWSPEC_* flags * @num_pasid_bits: number of PASID bits supported by this device * @num_ids: number of associated device IDs * @ids: IDs which this device may present to the IOMMU From patchwork Wed Jan 27 15:43:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371561 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp385837jam; Wed, 27 Jan 2021 07:47:36 -0800 (PST) X-Google-Smtp-Source: ABdhPJz/fm/9WzO51W0aDeeTmDKO2YUPRxi3hobStbGLgpcJTxutZdJF3QuBf/n6pa0enutNr34g X-Received: by 2002:a05:6402:3487:: with SMTP id v7mr9843814edc.68.1611762456832; Wed, 27 Jan 2021 07:47:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611762456; cv=none; d=google.com; s=arc-20160816; b=ThNBrF/JDQBqNUQYKOelVKWshWiCsmM2qYTAuRYf9yQxRkDtFD2bfOdZbDumkETyjR wsO3/qYVqRxZ6dv4QVDl/tVg/Mc68AI1BPnMMzHEiB2V3efBpgnx7DrC2mrhqdYaAHMd Q6CL3+bJulQh6dXKmi4HreiJCseFjGf+hzwTrAYSu+MffC8sTbQo4nLFkMehW0zcC5rf JFyULZlFUFG0yDeVyRNaF/WSa1cXLFSEnUgDW4obks4c049VagR//7Wh+40T44Xh3tjM fNmG8neXOUVEnn0IVtRXJNxzpIGCyL6kXRPrL54N59QJQ7JuTdSezbKb3Z6tqnQZQiil YUng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=D32J+akdetMmA5Myd4Zx5B3eWdLb4pZJ7wlpCkerL8M=; b=M7FjfZCmrPArH6nvc6yHyajUmn0oGxdJAlcANo9N5b/kQJ32PpnF0EEeAaJ62b3NmF mezFS8cGie0qcKodtz4P7M1j1wEhvaGUHZwb8eKMfCZOMghgt8zQpFdjN8Lvop2X4Txj XakbiVZ0UzmUY2CxWR4u4Wom/dimW7WEUdyuD/d0yG8rUa8gxQshRo1dmTms/eOdCtK7 Y2o94TIFzdRPHJP61+516MZYt6xlwSEtac/YgVXd77zt69dJCXtVCnYL1LK4Qv+qxHcg Yd0E9T5fF2lOHlaHP5kZTxV9hoQj2ViC8b6VOJuG3+wk3fMD3i5I4yUbjD9efqMdhKtD FXwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="UH/OoMLS"; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cy17si1201994edb.193.2021.01.27.07.47.36; Wed, 27 Jan 2021 07:47:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="UH/OoMLS"; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236666AbhA0PrC (ORCPT + 4 others); Wed, 27 Jan 2021 10:47:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236580AbhA0PpR (ORCPT ); Wed, 27 Jan 2021 10:45:17 -0500 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54802C06178A for ; Wed, 27 Jan 2021 07:44:37 -0800 (PST) Received: by mail-wm1-x32d.google.com with SMTP id 190so1923040wmz.0 for ; Wed, 27 Jan 2021 07:44:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D32J+akdetMmA5Myd4Zx5B3eWdLb4pZJ7wlpCkerL8M=; b=UH/OoMLSzaN+FsOIAJC5gSRoEPcxuWqa3X8jFrP04820j5DyxMxcbcEASbEFTLSXhf 0GbsQXfuVhptl0Se2XE0XAs8Zavr6sEKa2rel8nhxvwz8E6FblS4+tq7tjVQqwaG5r9b Q4JSoMMnhu5zc9L6nMwEKawp/1O1SoINlbiuSmMTVFjA6k7Xrr6gMXIw3KB3jaQMaxMo sA9iPSfbVPKxUwMegdGRfLLOmLkAjzbukFGhVZZMHzsBnMub1mMrtEn+gV6cjVLeklC+ 8hCfYS+bjCENMG1iK1Sk4oFenjeVv0OZ5a07jNWA999LGtsOmmNr5ryx/vn7jghMVJ/O eRJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D32J+akdetMmA5Myd4Zx5B3eWdLb4pZJ7wlpCkerL8M=; b=gZ6ZUKe9dTls8xPJrpFmNOy71P1Vyz0Q2NNfCaatUwTKBOeV4I8oJ7CJ2DUIH+RATl 8gj2KDaNSNkyX/3kJI2uSGuGfCJWBGPccRyo7RE3Xks8GXELZ126yforpgBNbl7RoDfw w8Xue7+LoVRr97SXd4yyzyxM+3XHyV6T7fgLKX5d5p3FtjkKU8cwIerRFECRnB/090XO DkT6ke/wxddn//1CgxMuyeRsWQDEJlgYGmtgQmzBeeOa7S6VBr+gn64suaP0F4VL8ziX ySyMZ7mS6QjJzZi1SBVXxdvFy1zmVDxf0gQ5njHX9G37EJ39FIQzmApDw00TwKCm9Zhe zg2A== X-Gm-Message-State: AOAM531l0+fWPNJDV/aOWemK7A9HoeGQ24U3np/akakrpsIHbIm/hxzx aWk9K5U6HN743DvrhxDxtXdzfQ== X-Received: by 2002:a7b:ce11:: with SMTP id m17mr4703993wmc.158.1611762276108; Wed, 27 Jan 2021 07:44:36 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:35 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v12 02/10] iommu/arm-smmu-v3: Use device properties for pasid-num-bits Date: Wed, 27 Jan 2021 16:43:15 +0100 Message-Id: <20210127154322.3959196-3-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org The pasid-num-bits property shouldn't need a dedicated fwspec field, it's a job for device properties. Add properties for IORT, and access the number of PASID bits using device_property_read_u32(). Suggested-by: Robin Murphy Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- include/linux/iommu.h | 2 -- drivers/acpi/arm64/iort.c | 13 +++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 ++- drivers/iommu/of_iommu.c | 5 ----- 4 files changed, 9 insertions(+), 14 deletions(-) -- 2.30.0 diff --git a/include/linux/iommu.h b/include/linux/iommu.h index bdf3f34a4457..b7ea11fc1a93 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -571,7 +571,6 @@ struct iommu_group *fsl_mc_device_group(struct device *dev); * @ops: ops for this device's IOMMU * @iommu_fwnode: firmware handle for this device's IOMMU * @flags: IOMMU_FWSPEC_* flags - * @num_pasid_bits: number of PASID bits supported by this device * @num_ids: number of associated device IDs * @ids: IDs which this device may present to the IOMMU */ @@ -579,7 +578,6 @@ struct iommu_fwspec { const struct iommu_ops *ops; struct fwnode_handle *iommu_fwnode; u32 flags; - u32 num_pasid_bits; unsigned int num_ids; u32 ids[]; }; diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index d4eac6d7e9fb..c9a8bbb74b09 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -968,15 +968,16 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data) static void iort_named_component_init(struct device *dev, struct acpi_iort_node *node) { + struct property_entry props[2] = {}; struct acpi_iort_named_component *nc; - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - - if (!fwspec) - return; nc = (struct acpi_iort_named_component *)node->node_data; - fwspec->num_pasid_bits = FIELD_GET(ACPI_IORT_NC_PASID_BITS, - nc->node_flags); + props[0] = PROPERTY_ENTRY_U32("pasid-num-bits", + FIELD_GET(ACPI_IORT_NC_PASID_BITS, + nc->node_flags)); + + if (device_add_properties(dev, props)) + dev_warn(dev, "Could not add device properties\n"); } static int iort_nc_iommu_map(struct device *dev, struct acpi_iort_node *node) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index baebaac34a83..88dd9feb32f4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2392,7 +2392,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) } } - master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits); + device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits); + master->ssid_bits = min(smmu->ssid_bits, master->ssid_bits); /* * Note that PASID must be enabled before, and disabled after ATS: diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c index e505b9130a1c..a9d2df001149 100644 --- a/drivers/iommu/of_iommu.c +++ b/drivers/iommu/of_iommu.c @@ -210,11 +210,6 @@ const struct iommu_ops *of_iommu_configure(struct device *dev, of_pci_iommu_init, &info); } else { err = of_iommu_configure_device(master_np, dev, id); - - fwspec = dev_iommu_fwspec_get(dev); - if (!err && fwspec) - of_property_read_u32(master_np, "pasid-num-bits", - &fwspec->num_pasid_bits); } /* From patchwork Wed Jan 27 15:43:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371562 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp386009jam; Wed, 27 Jan 2021 07:47:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJxcml7Gp5gDhlXAZGZowAgwOlS7Gs5tyya0w6DKJ/HuzdO3xTqcoQNmRmTAnpeBajykhsDk X-Received: by 2002:a50:b765:: with SMTP id g92mr9687453ede.317.1611762469950; Wed, 27 Jan 2021 07:47:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611762469; cv=none; d=google.com; s=arc-20160816; b=V94b7S1tb+x4G0MgymMDqpVR8wapx6W8vdj8p2Efj9kGFLEGa3Qi3snqbziOpNqcAU pGDyfRGxoKshnmiLAdaXMB86TPZeMf/xXCoc9apW1cxCdd82ecKUfmmtk6e2VBSV3wQ0 J9/pIh8g9EBuUzGrh1AN8L6bCcj7+vV80ps/2p89A5uvN7bsLl3DvELjJeimmwjecpV1 FYeZyvt95LccV9n015ZyJdglQSqEAQh4K5dXCvdX1CEI5/jYvtBTzUf/CToEFEMruXrP Kok+8CJROwluIDUB2DqgaOWBzjvojSmD2IV70yHgrknP5/PJsNKrsTXRzxsW80kiolq8 jzMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uiSdj3ahcV4jfiq23D9UAVEBduTBnIEfUfM+Rzx87y0=; b=zFK1712shjcK6gE+jYpdM5p4Jg532qsz5VsGzhhCitAJCRZnTBlT9PBcwY4tMChhHD WcE4Us210LegbO+kfY/H0tK0MxG8/AAST1jUGnteVuSirwgvu7fjWxCworXYmO9swZr2 MgBjwOlRBjSs5bzSIvfM0DiPG9FC3ig96hKfT3PELkYc6fYXdqv4Q2QBKgHQ6qREHAKW TW5B0cgyZOCASKFrEw2902aZv+ULGK1rskpaq3tsNHp7D0NIBQBprAtqQ9a1cHwWA3ug m8N96EEXc5rmX+LRD3n3nXJiBfYxv2ROk/9JK2l7bgfK6ekJ7zoL1q8WFvrrBeQ7w40O kpSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Mrx2fWY2; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cy17si1201994edb.193.2021.01.27.07.47.49; Wed, 27 Jan 2021 07:47:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Mrx2fWY2; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236679AbhA0PrL (ORCPT + 4 others); Wed, 27 Jan 2021 10:47:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236597AbhA0PpT (ORCPT ); Wed, 27 Jan 2021 10:45:19 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1304C061793 for ; Wed, 27 Jan 2021 07:44:38 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id i9so2081190wmq.1 for ; Wed, 27 Jan 2021 07:44:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uiSdj3ahcV4jfiq23D9UAVEBduTBnIEfUfM+Rzx87y0=; b=Mrx2fWY2rsbi1BLTyPkIpFFj1S95FTrb66KJsxbFJgvM+8jkmHybwZuT/uj2GyWSLn dTZlf3d5ggBEtQJVU1JMBsZJaRmAfVfM5257W62qCe/W8//8I4bOuDWGMwebfaSU5SOG xtztN8LNapip2/77JVFc1aRqgfg1tTvKUweHlQ0c4sB9B0Z5BMSK5/gPD6hi+kJ43i7A 14KnhgsBvK7j5+kIOT8IeBEBNpRdSgZatwC8vPLINLrQYzrqDsxn2UFQ/6s98efXxZtp pVNIwL2mjFs+mTlq8X/6x03Xrt1e0+vBfm/XstFBcxptX6POl5gEQ29G8nnJ/4ROC+Jj xpTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uiSdj3ahcV4jfiq23D9UAVEBduTBnIEfUfM+Rzx87y0=; b=Kk1H5my0g6OQVFKgutWA5ZxomUhoyFTe2RjgkiC2VOOVkxFcYKjwC/8BCb2wYacobx 7+BUffgl5I/jbVQMZsLf4JQUZV3wsXwP6iImvmByMjg6urH2miescXdOu94E9pMsmDub ZTQ0i1FCcULXvLORAN7jQBeRS48iuD+4PQbT93F1Df9nHoFr8KnT8nBqqTnvtxiXm0ly yFTtU+Sc2KL+2BKlaPh9+KKN5CSaRfYnGhDKVwABCHtcTw68JAktVPdcGQIgkcPnzNw0 ViG6/lJQ6IMhAsFKiu4We7+b3Ahha2Kavbx9sA7VDR5SdIVSdF8AMFZZUnctRecptTxR u6ZA== X-Gm-Message-State: AOAM532+EEktOyewEd6CXjBSqc/ZJ0ltkOa6cFxZR3a+FMLAoka6RXIa 67rUnfqTmHIKDwAXrLS2e7v+gA== X-Received: by 2002:a1c:2341:: with SMTP id j62mr4832905wmj.34.1611762277458; Wed, 27 Jan 2021 07:44:37 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:36 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , Arnd Bergmann , David Woodhouse , Greg Kroah-Hartman , Zhou Wang Subject: [PATCH v12 03/10] iommu: Separate IOMMU_DEV_FEAT_IOPF from IOMMU_DEV_FEAT_SVA Date: Wed, 27 Jan 2021 16:43:16 +0100 Message-Id: <20210127154322.3959196-4-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Some devices manage I/O Page Faults (IOPF) themselves instead of relying on PCIe PRI or Arm SMMU stall. Allow their drivers to enable SVA without mandating IOMMU-managed IOPF. The other device drivers now need to first enable IOMMU_DEV_FEAT_IOPF before enabling IOMMU_DEV_FEAT_SVA. Enabling IOMMU_DEV_FEAT_IOPF on its own doesn't have any effect visible to the device driver, it is used in combination with other features. Signed-off-by: Jean-Philippe Brucker --- Cc: Arnd Bergmann Cc: David Woodhouse Cc: Greg Kroah-Hartman Cc: Joerg Roedel Cc: Lu Baolu Cc: Will Deacon Cc: Zhangfei Gao Cc: Zhou Wang --- include/linux/iommu.h | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) -- 2.30.0 Reviewed-by: Eric Auger diff --git a/include/linux/iommu.h b/include/linux/iommu.h index b7ea11fc1a93..00348e4c3c26 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -156,10 +156,24 @@ struct iommu_resv_region { enum iommu_resv_type type; }; -/* Per device IOMMU features */ +/** + * enum iommu_dev_features - Per device IOMMU features + * @IOMMU_DEV_FEAT_AUX: Auxiliary domain feature + * @IOMMU_DEV_FEAT_SVA: Shared Virtual Addresses + * @IOMMU_DEV_FEAT_IOPF: I/O Page Faults such as PRI or Stall. Generally + * enabling %IOMMU_DEV_FEAT_SVA requires + * %IOMMU_DEV_FEAT_IOPF, but some devices manage I/O Page + * Faults themselves instead of relying on the IOMMU. When + * supported, this feature must be enabled before and + * disabled after %IOMMU_DEV_FEAT_SVA. + * + * Device drivers query whether a feature is supported using + * iommu_dev_has_feature(), and enable it using iommu_dev_enable_feature(). + */ enum iommu_dev_features { - IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */ - IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */ + IOMMU_DEV_FEAT_AUX, + IOMMU_DEV_FEAT_SVA, + IOMMU_DEV_FEAT_IOPF, }; #define IOMMU_PASID_INVALID (-1U) From patchwork Wed Jan 27 15:43:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371566 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp386194jam; Wed, 27 Jan 2021 07:48:03 -0800 (PST) X-Google-Smtp-Source: ABdhPJxC+Sff/RjV7aaJ/SVJVmlsn8z5HYID0COeBWdJoPntCHS8o/kW9EAi/Ka4cs3AgZn7UICi X-Received: by 2002:a17:906:38c3:: with SMTP id r3mr7390593ejd.193.1611762482886; Wed, 27 Jan 2021 07:48:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611762482; cv=none; d=google.com; s=arc-20160816; b=rwDPgYlp3ZaMZO3/Tzwp6Tfd7cgfPJiFvqOAEuAokC7A3q550f1OwaKqxy/UHqDONT ohEf0bVHpP1rIMCD1+M03HJoPXxfvrXLbxfi54uMHKOmk8RqgT4gCsLPlNRYWnun0a0A Xoqd5KAO74SNbMov5lWQejhBs2wfaGQK4QTCuXiqgYOWJ0VgvWf1mW1nwqqU1fEtdvAT ezo+y1ZwP5j6mDTg+bWGSiEROwEBBLTqFm/+z5ARPibXSn4kM7boN9fCuhef2w7yC36k zytsqJeUw55Ot+Bq1VMsMKkDn4ZjpusmOU4i3o2YE13Nxc4F2dwZ97S/pvfgK5+uJSvp 62sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0HAbfAEz8TnDqiVsICUxE3hnnLknlvV/GMLCZp7lxlM=; b=zSceF7rU+fgmc5KcASJ5+Yc+v8nBoIOgEc5TrvPrvQfgL+Nw7EmtblREkXOZQIR/ZK Di6KbsPYg49SMl6lVjtlQ6+UFVhKVW2QMgJu+slZBGk+IQH4voHHH0nvkaqTqy9jOCnf K0g4B+oLgq0v5mgmiRc6llK4/xoxBX/KjXeqpTOUXFIvyGUgQFU5OJq1aIRWOar262zz rcQlU4y9PHa1bFEz7hdxm4wo/0qgziQQtWEZSrkrRAxK14UyewU3BUjMxFRRznI9MrF0 o8GA26y5b1YVSieycOD/EcWIzw1n055P/8CTR3anYYBwug6E38wi3EqPY6Ste5ZV3sKP knvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ad/j4aBI"; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cy17si1201994edb.193.2021.01.27.07.48.02; Wed, 27 Jan 2021 07:48:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ad/j4aBI"; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236411AbhA0Prk (ORCPT + 4 others); Wed, 27 Jan 2021 10:47:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236616AbhA0Ppg (ORCPT ); Wed, 27 Jan 2021 10:45:36 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04200C061797 for ; Wed, 27 Jan 2021 07:44:40 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id g10so2413707wrx.1 for ; Wed, 27 Jan 2021 07:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0HAbfAEz8TnDqiVsICUxE3hnnLknlvV/GMLCZp7lxlM=; b=ad/j4aBI/T3WQEdCN+D6UAssRoPtamKNCApTqJ4FK8vTOMCdUIsdNHY5IzN6j1WbgY +8re948rFqrdclQrOU+JQ7TK+1NZcyhl5Aw0t9A0R1Lxy2hKotqfrnQ/TcKW9OYoflRv aCDaEnJAJO/9z9KFs4OidYWk8o/9ptYrG9b6u4M/reWw2gwF1+9/jnnx8yRw4vkuau5W 1o19RF+k1Mb91K5dJ4y+iYEQG9SvHHUf4d0+zb3Lxt4NOo2rs2euYSb+wQGCwe6OEE3C 1352Mdoft0AYtZ72LLDVh3bk+vfGO2y3Vj5eB8jtgQZy7Kor4+bbXI2vnPnyWLGJuU2b VWjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0HAbfAEz8TnDqiVsICUxE3hnnLknlvV/GMLCZp7lxlM=; b=I5lA31P/5Z8vxmDm5B1ReML4qMxkAqVpe6lm7ecJQejoi5Dzw9m7Gmi0QCmCjNVkc2 /YMbLdkgFwPDkWTSUsBay/m5UKkn41NqWxIiZYZsLEpS2fGOVfnzZ2m/0OA0AlD8Y0Hm pmBDtO+AXUi5SKIX6mGkVSSn9hChLU9Vdj0Qmdja2iU+chxXQ3R116kLRz6kBYpcRRDh chbTnrvv448tfP3ZLKcBjZdCIpVs+hOQzX/c67IsGj8QK0rCvSkb6a9K8l1gtjiAz9DD UJmdaeewS64mpvXj35k5hHgNRjrN+0A40RUlgE5W4Do1HvFmNgkT/4jtaffXUqmPKxYg ZGzQ== X-Gm-Message-State: AOAM5325cD0wrRHBrbUEPok9HG06tOsUEVzoHg0aWxQ1ymGDFWT0nD+0 wcEXkxCD+J6K5s8wOGoM4vX0oQ== X-Received: by 2002:adf:fc8a:: with SMTP id g10mr12217200wrr.189.1611762278786; Wed, 27 Jan 2021 07:44:38 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:38 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , David Woodhouse Subject: [PATCH v12 04/10] iommu/vt-d: Support IOMMU_DEV_FEAT_IOPF Date: Wed, 27 Jan 2021 16:43:17 +0100 Message-Id: <20210127154322.3959196-5-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Allow drivers to query and enable IOMMU_DEV_FEAT_IOPF, which amounts to checking whether PRI is enabled. Reviewed-by: Lu Baolu Signed-off-by: Jean-Philippe Brucker --- Cc: David Woodhouse Cc: Lu Baolu --- drivers/iommu/intel/iommu.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) -- 2.30.0 diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index f665322a0991..c777bd94df5d 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -5330,6 +5330,8 @@ static int siov_find_pci_dvsec(struct pci_dev *pdev) static bool intel_iommu_dev_has_feat(struct device *dev, enum iommu_dev_features feat) { + struct device_domain_info *info = get_domain_info(dev); + if (feat == IOMMU_DEV_FEAT_AUX) { int ret; @@ -5344,13 +5346,13 @@ intel_iommu_dev_has_feat(struct device *dev, enum iommu_dev_features feat) return !!siov_find_pci_dvsec(to_pci_dev(dev)); } - if (feat == IOMMU_DEV_FEAT_SVA) { - struct device_domain_info *info = get_domain_info(dev); + if (feat == IOMMU_DEV_FEAT_IOPF) + return info && info->pri_supported; + if (feat == IOMMU_DEV_FEAT_SVA) return info && (info->iommu->flags & VTD_FLAG_SVM_CAPABLE) && info->pasid_supported && info->pri_supported && info->ats_supported; - } return false; } @@ -5361,6 +5363,9 @@ intel_iommu_dev_enable_feat(struct device *dev, enum iommu_dev_features feat) if (feat == IOMMU_DEV_FEAT_AUX) return intel_iommu_enable_auxd(dev); + if (feat == IOMMU_DEV_FEAT_IOPF) + return intel_iommu_dev_has_feat(dev, feat) ? 0 : -ENODEV; + if (feat == IOMMU_DEV_FEAT_SVA) { struct device_domain_info *info = get_domain_info(dev); From patchwork Wed Jan 27 15:43:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371565 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp386190jam; Wed, 27 Jan 2021 07:48:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJxZ2IvSVT/5w5ruXsX3RnrXTxw5AqT7sye36VHMx8mVvnLTr8lzQRpFZEi/aTioKSMGCZOh X-Received: by 2002:a05:6402:1c8b:: with SMTP id cy11mr9949770edb.256.1611762482513; Wed, 27 Jan 2021 07:48:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611762482; cv=none; d=google.com; s=arc-20160816; b=H4v8bJcw9UmPXipnnNql0eVnuCm0iLEfiY6HnqqMXlMjehpVWfkrbbQB9fhdQIXKU8 oFlDvGdfmZ/EBp24QsAmGbHCY0imXX/pwKYK0ofzjcmRLcFi1B9HmtOEmJ3yIkzQ/VCX ooI89mmfXl7RDHJ6Ckn0ij6cFxusarb+A6O1Dd8pr+mCBAfQs0Az60I9uJ09q4kcLmZM wuwExEtGe1LMJqoKMloYh+alCfiATepULfp/9ySvefgLKK9AVnnVWiCR86Zie8pJDGmc 2xW/emxh0v7nF7t2bocFQF5inu3gVylj8HlNocaOFFkNxI8sls+x184o3qq6voklLdL8 Eo3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2eRhKsVau0r7rYJc5kg9wODttGekaxkRDREYSkG2V3g=; b=HA2Upai17zp88bLXxFSVJINfRnd5MRj2IEKHoTGmCb87sKpAwx8pYRJXpcxSDTPOLZ wLAQJ8LrhLNaWghZgde/wg66Wh8iTHhdI8g7g9zTZ9DYl2ewDi+DqpMOzFpf4E1qU9W5 6bL+kPuM0zXRCfpHS5Eu0tPPL0dqxEntQstSznWLUkSArh2CSjl4RqJybTWCIgdNSOtp ZNCQeyfRzgtZLvpCau7Wii1sbFqY1GLZOB4whrphoPXrR6Urpi64qWUquFneId303tBS US1tnlKIuf6HkUN3oceQoladF7gBPo455iG/SJNCCPTWosISxWjozQKgLQiwa2aCLLzc CWPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ivODR+o+; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cy17si1201994edb.193.2021.01.27.07.48.02; Wed, 27 Jan 2021 07:48:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ivODR+o+; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236459AbhA0Prg (ORCPT + 4 others); Wed, 27 Jan 2021 10:47:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236411AbhA0Ppl (ORCPT ); Wed, 27 Jan 2021 10:45:41 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E898C0617A9 for ; Wed, 27 Jan 2021 07:44:41 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id s24so3550912wmj.0 for ; Wed, 27 Jan 2021 07:44:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2eRhKsVau0r7rYJc5kg9wODttGekaxkRDREYSkG2V3g=; b=ivODR+o+zeC+VLJ0NgPRRPEDqKLo115ma74DNu8zGnxMHACyPpvmyq2NbKEY54zWcC HMm/wagqjHGCTD9o5sOyQaYdwXrhGcQkH2bd8KUAVmSLDxtP1uQgJJQRingKBA0YhTVP sWQ+mMRICXpCezAP6Oj2gsHkGf+PQTmlM6VqnZ2A0kBbV37ha72LnnyjtE1Y1DSOhWUn uSlXLg2b3WTL1E3LYGyJUyJOQ3ycX5g0wj/1R/jq2nm0TPNsCHwArjYP5pnIHZKPQdek rkX5+FHD4bTJn/ZGAgi+VVGF9SZRHo5GHcqKmqn+22Vc1eUWA9hrfnlUvtNucqx/emx/ 3qUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2eRhKsVau0r7rYJc5kg9wODttGekaxkRDREYSkG2V3g=; b=ajuReMqu7H+xcrV41q12nQSvrv+Vivixk65P/4xebVhXh4Nw45I3+oIaDL4ashihl3 9QHseo4Q9pjytQtNVMs4Kc8JFC4fEXdRaNPaOysxGh4pKm44UEKZD+Fq5DXFP1WB5d0U kT5sfdL4wzhm36ghT0hTdrvVcI7X2X28LtV2QxHWT/Depb2+h9X5E6XZCJGKAwGgE9rM YcJCT5PfcqXdmK7gLDRsKZvd7rGBD001PbFvIFlViezYvh29ylONT27qSt74+nDA38RL 3JhJmRiNPeDjrSeRz/Nadu5Ub7+qB/6KlJYw+kIi8nmJ9OmgY6wb1UyrWs/LpNJ+V5P7 5QZQ== X-Gm-Message-State: AOAM530ylROp8VHlVkxw5mNRPzO7M42pVjuCgwjOdZDx9cSMManpuhG5 WMP/7lNDod3XujQTj5CQWdLD2Q== X-Received: by 2002:a1c:7e4e:: with SMTP id z75mr4860472wmc.168.1611762280149; Wed, 27 Jan 2021 07:44:40 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:39 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , Arnd Bergmann , Greg Kroah-Hartman , Zhou Wang Subject: [PATCH v12 05/10] uacce: Enable IOMMU_DEV_FEAT_IOPF Date: Wed, 27 Jan 2021 16:43:18 +0100 Message-Id: <20210127154322.3959196-6-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org The IOPF (I/O Page Fault) feature is now enabled independently from the SVA feature, because some IOPF implementations are device-specific and do not require IOMMU support for PCIe PRI or Arm SMMU stall. Enable IOPF unconditionally when enabling SVA for now. In the future, if a device driver implementing a uacce interface doesn't need IOPF support, it will need to tell the uacce module, for example with a new flag. Acked-by: Zhangfei Gao Signed-off-by: Jean-Philippe Brucker --- Cc: Arnd Bergmann Cc: Greg Kroah-Hartman Cc: Zhangfei Gao Cc: Zhou Wang --- drivers/misc/uacce/uacce.c | 39 +++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) -- 2.30.0 diff --git a/drivers/misc/uacce/uacce.c b/drivers/misc/uacce/uacce.c index d07af4edfcac..6db7a98486ec 100644 --- a/drivers/misc/uacce/uacce.c +++ b/drivers/misc/uacce/uacce.c @@ -385,6 +385,33 @@ static void uacce_release(struct device *dev) kfree(uacce); } +static unsigned int uacce_enable_sva(struct device *parent, unsigned int flags) +{ + if (!(flags & UACCE_DEV_SVA)) + return flags; + + flags &= ~UACCE_DEV_SVA; + + if (iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_IOPF)) + return flags; + + if (iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_SVA)) { + iommu_dev_disable_feature(parent, IOMMU_DEV_FEAT_IOPF); + return flags; + } + + return flags | UACCE_DEV_SVA; +} + +static void uacce_disable_sva(struct uacce_device *uacce) +{ + if (!(uacce->flags & UACCE_DEV_SVA)) + return; + + iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_IOPF); +} + /** * uacce_alloc() - alloc an accelerator * @parent: pointer of uacce parent device @@ -404,11 +431,7 @@ struct uacce_device *uacce_alloc(struct device *parent, if (!uacce) return ERR_PTR(-ENOMEM); - if (flags & UACCE_DEV_SVA) { - ret = iommu_dev_enable_feature(parent, IOMMU_DEV_FEAT_SVA); - if (ret) - flags &= ~UACCE_DEV_SVA; - } + flags = uacce_enable_sva(parent, flags); uacce->parent = parent; uacce->flags = flags; @@ -432,8 +455,7 @@ struct uacce_device *uacce_alloc(struct device *parent, return uacce; err_with_uacce: - if (flags & UACCE_DEV_SVA) - iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + uacce_disable_sva(uacce); kfree(uacce); return ERR_PTR(ret); } @@ -487,8 +509,7 @@ void uacce_remove(struct uacce_device *uacce) mutex_unlock(&uacce->queues_lock); /* disable sva now since no opened queues */ - if (uacce->flags & UACCE_DEV_SVA) - iommu_dev_disable_feature(uacce->parent, IOMMU_DEV_FEAT_SVA); + uacce_disable_sva(uacce); if (uacce->cdev) cdev_device_del(uacce->cdev, &uacce->dev); From patchwork Wed Jan 27 15:43:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 372672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AAD1C433DB for ; Wed, 27 Jan 2021 15:48:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 09EE3207C4 for ; Wed, 27 Jan 2021 15:47:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236530AbhA0Pr5 (ORCPT ); Wed, 27 Jan 2021 10:47:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234675AbhA0PqG (ORCPT ); Wed, 27 Jan 2021 10:46:06 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE0DEC0617AA for ; Wed, 27 Jan 2021 07:44:42 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id i9so2081401wmq.1 for ; Wed, 27 Jan 2021 07:44:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j4gQVr3WKH1e0317BI6PolZ4g0GU2TEiJ+uV7JsOhgI=; b=ol4vJkBgXRtcpP/o9t3QV+OtPJ9KaPd2eWbhuPVP3Hdq6nqRC6QkmN3WjeacbQq/Z1 6pUy+qSwDJSaifOeKGd/pKb6B9fJ6BFUcMZQSvXJcX1yiGRPH6RBekQytSEkmeaxmJ5a 0mwUs0nGA1Vm9mcq6PXN6MJtTi6Kbrit2qIZLlmUdlKjq3JYIfzsyUZw7add226d4VC0 W3YlN1e8YbbOMUYGSUT8Ka8E2plJ/3xqPx9dKD9uXt0MH8fwvEKAUeOhernmJ6aQ4V7O Xv3mK0qaAJsqWb3/rQPspBfoRfWnXdX4gvwYksHyfqIpgIf6nz6aNcyRi8sLnYkdxC8J WuHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j4gQVr3WKH1e0317BI6PolZ4g0GU2TEiJ+uV7JsOhgI=; b=YAgGcuJ5SvvlztrUFKuBlTHlgMU69C4AbIOP6HolOnrvyjfPG7w6T9WYD1nCEvXMQ2 QtpsT29VnMWNAIqyld0fUhhSG/+OR1rreJFE4WYAsp5I68iBwYCE2J6pg+XlQ8iWaTov UDelAID5JbuwjoTSHcDPhGophzaShyGfC+d8TY+TxsG9K5ra5nm/NFa8fO0eKqKQB2aJ JVrC/QMXWV+eGImwtTO5J5TAlrKoZLOmd7uXP2fceZGn+febHfEXRkgHZe6KGy1NxJt6 SxUtQUDRdJxZUbwphGbu4HtG2Dw0QKlJQcd28U8Kq8judtSHMfr7RYQM0KFR6BjDw2X2 ynRA== X-Gm-Message-State: AOAM531kl3LHuCQVcecxo/y5XtW32+uxgm1U3DHg/79KpSAupI3fhCOb f+qGTB97cC+hObJuRfZ5sfz7DQ== X-Google-Smtp-Source: ABdhPJzr4NX+4nvGnjPZ/Bbb+0hw4BX4gdX74vLnuAyOdj3wSRMcHHvzSiPq3uWp52uPWIcrbPBReg== X-Received: by 2002:a1c:7d0c:: with SMTP id y12mr4760833wmc.184.1611762281404; Wed, 27 Jan 2021 07:44:41 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:40 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v12 06/10] iommu: Add a page fault handler Date: Wed, 27 Jan 2021 16:43:19 +0100 Message-Id: <20210127154322.3959196-7-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Some systems allow devices to handle I/O Page Faults in the core mm. For example systems implementing the PCIe PRI extension or Arm SMMU stall model. Infrastructure for reporting these recoverable page faults was added to the IOMMU core by commit 0c830e6b3282 ("iommu: Introduce device fault report API"). Add a page fault handler for host SVA. IOMMU driver can now instantiate several fault workqueues and link them to IOPF-capable devices. Drivers can choose between a single global workqueue, one per IOMMU device, one per low-level fault queue, one per domain, etc. When it receives a fault event, most commonly in an IRQ handler, the IOMMU driver reports the fault using iommu_report_device_fault(), which calls the registered handler. The page fault handler then calls the mm fault handler, and reports either success or failure with iommu_page_response(). After the handler succeeds, the hardware retries the access. The iopf_param pointer could be embedded into iommu_fault_param. But putting iopf_param into the iommu_param structure allows us not to care about ordering between calls to iopf_queue_add_device() and iommu_register_device_fault_handler(). Reviewed-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/Makefile | 1 + drivers/iommu/iommu-sva-lib.h | 53 ++++ include/linux/iommu.h | 2 + drivers/iommu/io-pgfault.c | 461 ++++++++++++++++++++++++++++++++++ 4 files changed, 517 insertions(+) create mode 100644 drivers/iommu/io-pgfault.c diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 61bd30cd8369..60fafc23dee6 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -28,3 +28,4 @@ obj-$(CONFIG_S390_IOMMU) += s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o obj-$(CONFIG_IOMMU_SVA_LIB) += iommu-sva-lib.o +obj-$(CONFIG_IOMMU_SVA_LIB) += io-pgfault.o diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index b40990aef3fd..031155010ca8 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -12,4 +12,57 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max); void iommu_sva_free_pasid(struct mm_struct *mm); struct mm_struct *iommu_sva_find(ioasid_t pasid); +/* I/O Page fault */ +struct device; +struct iommu_fault; +struct iopf_queue; + +#ifdef CONFIG_IOMMU_SVA_LIB +int iommu_queue_iopf(struct iommu_fault *fault, void *cookie); + +int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev); +int iopf_queue_remove_device(struct iopf_queue *queue, + struct device *dev); +int iopf_queue_flush_dev(struct device *dev); +struct iopf_queue *iopf_queue_alloc(const char *name); +void iopf_queue_free(struct iopf_queue *queue); +int iopf_queue_discard_partial(struct iopf_queue *queue); + +#else /* CONFIG_IOMMU_SVA_LIB */ +static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) +{ + return -ENODEV; +} + +static inline int iopf_queue_add_device(struct iopf_queue *queue, + struct device *dev) +{ + return -ENODEV; +} + +static inline int iopf_queue_remove_device(struct iopf_queue *queue, + struct device *dev) +{ + return -ENODEV; +} + +static inline int iopf_queue_flush_dev(struct device *dev) +{ + return -ENODEV; +} + +static inline struct iopf_queue *iopf_queue_alloc(const char *name) +{ + return NULL; +} + +static inline void iopf_queue_free(struct iopf_queue *queue) +{ +} + +static inline int iopf_queue_discard_partial(struct iopf_queue *queue) +{ + return -ENODEV; +} +#endif /* CONFIG_IOMMU_SVA_LIB */ #endif /* _IOMMU_SVA_LIB_H */ diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 00348e4c3c26..edc9be443a74 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -366,6 +366,7 @@ struct iommu_fault_param { * struct dev_iommu - Collection of per-device IOMMU data * * @fault_param: IOMMU detected device fault reporting data + * @iopf_param: I/O Page Fault queue and data * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to * @priv: IOMMU Driver private data @@ -376,6 +377,7 @@ struct iommu_fault_param { struct dev_iommu { struct mutex lock; struct iommu_fault_param *fault_param; + struct iopf_device_param *iopf_param; struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; void *priv; diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c new file mode 100644 index 000000000000..1df8c1dcae77 --- /dev/null +++ b/drivers/iommu/io-pgfault.c @@ -0,0 +1,461 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Handle device page faults + * + * Copyright (C) 2020 ARM Ltd. + */ + +#include +#include +#include +#include +#include + +#include "iommu-sva-lib.h" + +/** + * struct iopf_queue - IO Page Fault queue + * @wq: the fault workqueue + * @devices: devices attached to this queue + * @lock: protects the device list + */ +struct iopf_queue { + struct workqueue_struct *wq; + struct list_head devices; + struct mutex lock; +}; + +/** + * struct iopf_device_param - IO Page Fault data attached to a device + * @dev: the device that owns this param + * @queue: IOPF queue + * @queue_list: index into queue->devices + * @partial: faults that are part of a Page Request Group for which the last + * request hasn't been submitted yet. + */ +struct iopf_device_param { + struct device *dev; + struct iopf_queue *queue; + struct list_head queue_list; + struct list_head partial; +}; + +struct iopf_fault { + struct iommu_fault fault; + struct list_head list; +}; + +struct iopf_group { + struct iopf_fault last_fault; + struct list_head faults; + struct work_struct work; + struct device *dev; +}; + +static int iopf_complete_group(struct device *dev, struct iopf_fault *iopf, + enum iommu_page_response_code status) +{ + struct iommu_page_response resp = { + .version = IOMMU_PAGE_RESP_VERSION_1, + .pasid = iopf->fault.prm.pasid, + .grpid = iopf->fault.prm.grpid, + .code = status, + }; + + if ((iopf->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID) && + (iopf->fault.prm.flags & IOMMU_FAULT_PAGE_RESPONSE_NEEDS_PASID)) + resp.flags = IOMMU_PAGE_RESP_PASID_VALID; + + return iommu_page_response(dev, &resp); +} + +static enum iommu_page_response_code +iopf_handle_single(struct iopf_fault *iopf) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags = 0; + unsigned int fault_flags = FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm = &iopf->fault.prm; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm = iommu_sva_find(prm->pasid); + if (IS_ERR_OR_NULL(mm)) + return status; + + mmap_read_lock(mm); + + vma = find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |= VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |= VM_WRITE; + fault_flags |= FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |= VM_EXEC; + fault_flags |= FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |= FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret = handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status = ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + +static void iopf_handle_group(struct work_struct *work) +{ + struct iopf_group *group; + struct iopf_fault *iopf, *next; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS; + + group = container_of(work, struct iopf_group, work); + + list_for_each_entry_safe(iopf, next, &group->faults, list) { + /* + * For the moment, errors are sticky: don't handle subsequent + * faults in the group if there is an error. + */ + if (status == IOMMU_PAGE_RESP_SUCCESS) + status = iopf_handle_single(iopf); + + if (!(iopf->fault.prm.flags & + IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) + kfree(iopf); + } + + iopf_complete_group(group->dev, &group->last_fault, status); + kfree(group); +} + +/** + * iommu_queue_iopf - IO Page Fault handler + * @fault: fault event + * @cookie: struct device, passed to iommu_register_device_fault_handler. + * + * Add a fault to the device workqueue, to be handled by mm. + * + * This module doesn't handle PCI PASID Stop Marker; IOMMU drivers must discard + * them before reporting faults. A PASID Stop Marker (LRW = 0b100) doesn't + * expect a response. It may be generated when disabling a PASID (issuing a + * PASID stop request) by some PCI devices. + * + * The PASID stop request is issued by the device driver before unbind(). Once + * it completes, no page request is generated for this PASID anymore and + * outstanding ones have been pushed to the IOMMU (as per PCIe 4.0r1.0 - 6.20.1 + * and 10.4.1.2 - Managing PASID TLP Prefix Usage). Some PCI devices will wait + * for all outstanding page requests to come back with a response before + * completing the PASID stop request. Others do not wait for page responses, and + * instead issue this Stop Marker that tells us when the PASID can be + * reallocated. + * + * It is safe to discard the Stop Marker because it is an optimization. + * a. Page requests, which are posted requests, have been flushed to the IOMMU + * when the stop request completes. + * b. The IOMMU driver flushes all fault queues on unbind() before freeing the + * PASID. + * + * So even though the Stop Marker might be issued by the device *after* the stop + * request completes, outstanding faults will have been dealt with by the time + * the PASID is freed. + * + * Return: 0 on success and <0 on error. + */ +int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) +{ + int ret; + struct iopf_group *group; + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + + struct device *dev = cookie; + struct dev_iommu *param = dev->iommu; + + lockdep_assert_held(¶m->lock); + + if (fault->type != IOMMU_FAULT_PAGE_REQ) + /* Not a recoverable page fault */ + return -EOPNOTSUPP; + + /* + * As long as we're holding param->lock, the queue can't be unlinked + * from the device and therefore cannot disappear. + */ + iopf_param = param->iopf_param; + if (!iopf_param) + return -ENODEV; + + if (!(fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) { + iopf = kzalloc(sizeof(*iopf), GFP_KERNEL); + if (!iopf) + return -ENOMEM; + + iopf->fault = *fault; + + /* Non-last request of a group. Postpone until the last one */ + list_add(&iopf->list, &iopf_param->partial); + + return 0; + } + + group = kzalloc(sizeof(*group), GFP_KERNEL); + if (!group) { + /* + * The caller will send a response to the hardware. But we do + * need to clean up before leaving, otherwise partial faults + * will be stuck. + */ + ret = -ENOMEM; + goto cleanup_partial; + } + + group->dev = dev; + group->last_fault.fault = *fault; + INIT_LIST_HEAD(&group->faults); + list_add(&group->last_fault.list, &group->faults); + INIT_WORK(&group->work, iopf_handle_group); + + /* See if we have partial faults for this group */ + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { + if (iopf->fault.prm.grpid == fault->prm.grpid) + /* Insert *before* the last fault */ + list_move(&iopf->list, &group->faults); + } + + queue_work(iopf_param->queue->wq, &group->work); + return 0; + +cleanup_partial: + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { + if (iopf->fault.prm.grpid == fault->prm.grpid) { + list_del(&iopf->list); + kfree(iopf); + } + } + return ret; +} +EXPORT_SYMBOL_GPL(iommu_queue_iopf); + +/** + * iopf_queue_flush_dev - Ensure that all queued faults have been processed + * @dev: the endpoint whose faults need to be flushed. + * + * The IOMMU driver calls this before releasing a PASID, to ensure that all + * pending faults for this PASID have been handled, and won't hit the address + * space of the next process that uses this PASID. The driver must make sure + * that no new fault is added to the queue. In particular it must flush its + * low-level queue before calling this function. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_flush_dev(struct device *dev) +{ + int ret = 0; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param) + return -ENODEV; + + mutex_lock(¶m->lock); + iopf_param = param->iopf_param; + if (iopf_param) + flush_workqueue(iopf_param->queue->wq); + else + ret = -ENODEV; + mutex_unlock(¶m->lock); + + return ret; +} +EXPORT_SYMBOL_GPL(iopf_queue_flush_dev); + +/** + * iopf_queue_discard_partial - Remove all pending partial fault + * @queue: the queue whose partial faults need to be discarded + * + * When the hardware queue overflows, last page faults in a group may have been + * lost and the IOMMU driver calls this to discard all partial faults. The + * driver shouldn't be adding new faults to this queue concurrently. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_discard_partial(struct iopf_queue *queue) +{ + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + + if (!queue) + return -EINVAL; + + mutex_lock(&queue->lock); + list_for_each_entry(iopf_param, &queue->devices, queue_list) { + list_for_each_entry_safe(iopf, next, &iopf_param->partial, + list) { + list_del(&iopf->list); + kfree(iopf); + } + } + mutex_unlock(&queue->lock); + return 0; +} +EXPORT_SYMBOL_GPL(iopf_queue_discard_partial); + +/** + * iopf_queue_add_device - Add producer to the fault queue + * @queue: IOPF queue + * @dev: device to add + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev) +{ + int ret = -EBUSY; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param) + return -ENODEV; + + iopf_param = kzalloc(sizeof(*iopf_param), GFP_KERNEL); + if (!iopf_param) + return -ENOMEM; + + INIT_LIST_HEAD(&iopf_param->partial); + iopf_param->queue = queue; + iopf_param->dev = dev; + + mutex_lock(&queue->lock); + mutex_lock(¶m->lock); + if (!param->iopf_param) { + list_add(&iopf_param->queue_list, &queue->devices); + param->iopf_param = iopf_param; + ret = 0; + } + mutex_unlock(¶m->lock); + mutex_unlock(&queue->lock); + + if (ret) + kfree(iopf_param); + + return ret; +} +EXPORT_SYMBOL_GPL(iopf_queue_add_device); + +/** + * iopf_queue_remove_device - Remove producer from fault queue + * @queue: IOPF queue + * @dev: device to remove + * + * Caller makes sure that no more faults are reported for this device. + * + * Return: 0 on success and <0 on error. + */ +int iopf_queue_remove_device(struct iopf_queue *queue, struct device *dev) +{ + int ret = -EINVAL; + struct iopf_fault *iopf, *next; + struct iopf_device_param *iopf_param; + struct dev_iommu *param = dev->iommu; + + if (!param || !queue) + return -EINVAL; + + mutex_lock(&queue->lock); + mutex_lock(¶m->lock); + iopf_param = param->iopf_param; + if (iopf_param && iopf_param->queue == queue) { + list_del(&iopf_param->queue_list); + param->iopf_param = NULL; + ret = 0; + } + mutex_unlock(¶m->lock); + mutex_unlock(&queue->lock); + if (ret) + return ret; + + /* Just in case some faults are still stuck */ + list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) + kfree(iopf); + + kfree(iopf_param); + + return 0; +} +EXPORT_SYMBOL_GPL(iopf_queue_remove_device); + +/** + * iopf_queue_alloc - Allocate and initialize a fault queue + * @name: a unique string identifying the queue (for workqueue) + * + * Return: the queue on success and NULL on error. + */ +struct iopf_queue *iopf_queue_alloc(const char *name) +{ + struct iopf_queue *queue; + + queue = kzalloc(sizeof(*queue), GFP_KERNEL); + if (!queue) + return NULL; + + /* + * The WQ is unordered because the low-level handler enqueues faults by + * group. PRI requests within a group have to be ordered, but once + * that's dealt with, the high-level function can handle groups out of + * order. + */ + queue->wq = alloc_workqueue("iopf_queue/%s", WQ_UNBOUND, 0, name); + if (!queue->wq) { + kfree(queue); + return NULL; + } + + INIT_LIST_HEAD(&queue->devices); + mutex_init(&queue->lock); + + return queue; +} +EXPORT_SYMBOL_GPL(iopf_queue_alloc); + +/** + * iopf_queue_free - Free IOPF queue + * @queue: queue to free + * + * Counterpart to iopf_queue_alloc(). The driver must not be queuing faults or + * adding/removing devices on this queue anymore. + */ +void iopf_queue_free(struct iopf_queue *queue) +{ + struct iopf_device_param *iopf_param, *next; + + if (!queue) + return; + + list_for_each_entry_safe(iopf_param, next, &queue->devices, queue_list) + iopf_queue_remove_device(queue, iopf_param->dev); + + destroy_workqueue(queue->wq); + kfree(queue); +} +EXPORT_SYMBOL_GPL(iopf_queue_free); From patchwork Wed Jan 27 15:43:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371564 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp386185jam; Wed, 27 Jan 2021 07:48:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJyDbQTsn+SrnZeX5FT8dQkBi7NThof9vTZfzPZT8HIaAK/ovBIBEc6VSFuyPfRidvQBG5hb X-Received: by 2002:a50:d58c:: with SMTP id v12mr9422863edi.71.1611762481891; Wed, 27 Jan 2021 07:48:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611762481; cv=none; d=google.com; s=arc-20160816; b=qbHgOa7wJm6DCrpQE1JRXCB/KKvUrqDbRJm5QZpi57Dqv452l+l9HWEBx+prXTlrsH gdCOANjuHG7HcN5AWK4vUc5IpPVQHW8LRPqTA0GQYKFvWkNr49LuZh7SJxdWrhWhKHm9 hSyyxP4XZM4LCSijv5bwuHBRHC0tE7Cnrl1VlJaO5tuYuBw8k8tRCPUQ5Lp0BfH7D5Oq 3n3DXrQQLFBrrf+nLbqXlE8O+L3n8JaS9XmmX6eh+EPuJ+BE2b746EncS5r1O+ji2e63 IRxOZIf/mitHsGpnipWmra59sw6UpmqGnzFpPpK7QptP59VwhoufH/yGWxx41ny1KV0o TXGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tYnI/Xt3oYzBq3RcE9O/zctEuyA4AvchKpT/A0KDE3A=; b=xHbaUR0Q6t3O8JqLywhjV/d4AmtNVaJBKBkFhnTHZXzCuyVzUatQ9SoecYXj5Did04 mfsDwBNkgLGb+i5MHrb9I6R8CzWeJunmmJZEYI0y5/8E9I3G8qSvyt/5tBHlsr/QXo7B w9Tq8/OTvzYM34IoPWTMV7XETcUQxYTIXvTj97OhKNvO9Ds6Jf7X6x3sUUiz00ive6ee ObYuXMX4V9u1R2ABhvwSxjOk9RTKGdYrRqDFWmx6e6vhNsNjURmlnr/ucW0SyWIhvtkm Fq3MjH0c1eAAKscntqzSGIEGBMnIWSSU6wZhD2t5C6lSSPyEB6gQD20YLWd6bfF9b7rt B78A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ruYUr3as; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cy17si1201994edb.193.2021.01.27.07.48.01; Wed, 27 Jan 2021 07:48:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ruYUr3as; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234782AbhA0Prb (ORCPT + 4 others); Wed, 27 Jan 2021 10:47:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236644AbhA0Pp6 (ORCPT ); Wed, 27 Jan 2021 10:45:58 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D6BFC061352 for ; Wed, 27 Jan 2021 07:44:44 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id 190so1923350wmz.0 for ; Wed, 27 Jan 2021 07:44:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tYnI/Xt3oYzBq3RcE9O/zctEuyA4AvchKpT/A0KDE3A=; b=ruYUr3asCJkiWMZqX/2pPM1Py9KGdSwjvw86YXdm8jjE5F4yusgIfV1RKx3Cs/cq/F e5+A6cmTJMkM27OdjClhQau+TH66YdIs+Oy3VVgjKpMDeBDJcMVC/ZsHnYq7uwMvdMts EbceoCB7MCfhN91HROZuPB4t8n/6RZ6m44l+FS7wghy/zp+PtFLLUuYd2cTZ7akA6flQ CpujCePN2Ift9R3fC3ZSlU0I/OA8fkq5tYOBm6sDEcLw67jgx7+69ZD7CQpKXs3T2DLx cx/2lrNnk60w+/WhoxqeZTE7NQaN14rXdyro9XdKK7YWEMKPeY6R5RnhIb7NUTl0jX9V zm3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tYnI/Xt3oYzBq3RcE9O/zctEuyA4AvchKpT/A0KDE3A=; b=qIb4bFTYGcAUDLrOVFMtBLKJ1d5wnZJsixX054Fd7Zf3tNup9uqm8NHp4fXkLJOFhi uwsDyIL5y5pZZHyBWnlvbYk6GjOtx9Lm07z0S576VupvXiOq478v8EHb7Zn3g1YtgeFw UR3cpFUU1cW9VkZfSSMYlNBXf3473ZEAu7tAXwy2P44in7Wka+4taoV7zJIs/9Gz2Ou6 KMizPotRc6SHWDcZNpGM2thAh7wHaovNJB8A/UzvgZVGZpy/J2LZYbk5HhhfKBuWFu2t r0Xiujfwl3Cj1u76gsjgP9g9Gl8bbIKS55SUBJ8raza68U9a5oJNx04n7cr9R5vz1qFe FEGA== X-Gm-Message-State: AOAM5333699+6q1k62LjDqCkjDhS1IPY+HDX+ivX1S/XK72dNIE6tahg elcHUepSfMOOvj5kQf5X+S0ByA== X-Received: by 2002:a1c:a145:: with SMTP id k66mr4719492wme.18.1611762282841; Wed, 27 Jan 2021 07:44:42 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:42 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v12 07/10] iommu/arm-smmu-v3: Maintain a SID->device structure Date: Wed, 27 Jan 2021 16:43:20 +0100 Message-Id: <20210127154322.3959196-8-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org When handling faults from the event or PRI queue, we need to find the struct device associated with a SID. Add a rb_tree to keep track of SIDs. Acked-by: Jonathan Cameron Reviewed-by: Eric Auger Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 13 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 161 ++++++++++++++++---- 2 files changed, 144 insertions(+), 30 deletions(-) -- 2.30.0 diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index f985817c967a..7b15b7580c6e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -639,6 +639,15 @@ struct arm_smmu_device { /* IOMMU core code handle */ struct iommu_device iommu; + + struct rb_root streams; + struct mutex streams_mutex; +}; + +struct arm_smmu_stream { + u32 id; + struct arm_smmu_master *master; + struct rb_node node; }; /* SMMU private data for each master */ @@ -647,8 +656,8 @@ struct arm_smmu_master { struct device *dev; struct arm_smmu_domain *domain; struct list_head domain_head; - u32 *sids; - unsigned int num_sids; + struct arm_smmu_stream *streams; + unsigned int num_streams; bool ats_enabled; bool sva_enabled; struct list_head bonds; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 88dd9feb32f4..3afec6ed8075 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -909,8 +909,8 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i = 0; i < master->num_sids; i++) { - cmd.cfgi.sid = master->sids[i]; + for (i = 0; i < master->num_streams; i++) { + cmd.cfgi.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } } @@ -1355,6 +1355,32 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) return 0; } +__maybe_unused +static struct arm_smmu_master * +arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) +{ + struct rb_node *node; + struct arm_smmu_stream *stream; + struct arm_smmu_master *master = NULL; + + mutex_lock(&smmu->streams_mutex); + node = smmu->streams.rb_node; + while (node) { + stream = rb_entry(node, struct arm_smmu_stream, node); + if (stream->id < sid) { + node = node->rb_right; + } else if (stream->id > sid) { + node = node->rb_left; + } else { + master = stream->master; + break; + } + } + mutex_unlock(&smmu->streams_mutex); + + return master; +} + /* IRQ and event handlers */ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { @@ -1588,8 +1614,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - for (i = 0; i < master->num_sids; i++) { - cmd.atc.sid = master->sids[i]; + for (i = 0; i < master->num_streams; i++) { + cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_issue_cmd(master->smmu, &cmd); } @@ -1632,8 +1658,8 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!master->ats_enabled) continue; - for (i = 0; i < master->num_sids; i++) { - cmd.atc.sid = master->sids[i]; + for (i = 0; i < master->num_streams; i++) { + cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } @@ -2065,13 +2091,13 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) int i, j; struct arm_smmu_device *smmu = master->smmu; - for (i = 0; i < master->num_sids; ++i) { - u32 sid = master->sids[i]; + for (i = 0; i < master->num_streams; ++i) { + u32 sid = master->streams[i].id; __le64 *step = arm_smmu_get_step_for_sid(smmu, sid); /* Bridged PCI devices may end up with duplicated IDs */ for (j = 0; j < i; j++) - if (master->sids[j] == sid) + if (master->streams[j].id == sid) break; if (j < i) continue; @@ -2345,11 +2371,101 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid) return sid < limit; } +static int arm_smmu_insert_master(struct arm_smmu_device *smmu, + struct arm_smmu_master *master) +{ + int i; + int ret = 0; + struct arm_smmu_stream *new_stream, *cur_stream; + struct rb_node **new_node, *parent_node = NULL; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev); + + master->streams = kcalloc(fwspec->num_ids, + sizeof(*master->streams), GFP_KERNEL); + if (!master->streams) + return -ENOMEM; + master->num_streams = fwspec->num_ids; + + mutex_lock(&smmu->streams_mutex); + for (i = 0; i < fwspec->num_ids && !ret; i++) { + u32 sid = fwspec->ids[i]; + + new_stream = &master->streams[i]; + new_stream->id = sid; + new_stream->master = master; + + /* + * Check the SIDs are in range of the SMMU and our stream table + */ + if (!arm_smmu_sid_in_range(smmu, sid)) { + ret = -ERANGE; + break; + } + + /* Ensure l2 strtab is initialised */ + if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) { + ret = arm_smmu_init_l2_strtab(smmu, sid); + if (ret) + break; + } + + /* Insert into SID tree */ + new_node = &(smmu->streams.rb_node); + while (*new_node) { + cur_stream = rb_entry(*new_node, struct arm_smmu_stream, + node); + parent_node = *new_node; + if (cur_stream->id > new_stream->id) { + new_node = &((*new_node)->rb_left); + } else if (cur_stream->id < new_stream->id) { + new_node = &((*new_node)->rb_right); + } else { + dev_warn(master->dev, + "stream %u already in tree\n", + cur_stream->id); + ret = -EINVAL; + break; + } + } + + if (!ret) { + rb_link_node(&new_stream->node, parent_node, new_node); + rb_insert_color(&new_stream->node, &smmu->streams); + } + } + + if (ret) { + for (; i > 0; i--) + rb_erase(&master->streams[i].node, &smmu->streams); + kfree(master->streams); + } + mutex_unlock(&smmu->streams_mutex); + + return ret; +} + +static void arm_smmu_remove_master(struct arm_smmu_master *master) +{ + int i; + struct arm_smmu_device *smmu = master->smmu; + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(master->dev); + + if (!smmu || !master->streams) + return; + + mutex_lock(&smmu->streams_mutex); + for (i = 0; i < fwspec->num_ids; i++) + rb_erase(&master->streams[i].node, &smmu->streams); + mutex_unlock(&smmu->streams_mutex); + + kfree(master->streams); +} + static struct iommu_ops arm_smmu_ops; static struct iommu_device *arm_smmu_probe_device(struct device *dev) { - int i, ret; + int ret; struct arm_smmu_device *smmu; struct arm_smmu_master *master; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); @@ -2370,27 +2486,12 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - master->sids = fwspec->ids; - master->num_sids = fwspec->num_ids; INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); - /* Check the SIDs are in range of the SMMU and our stream table */ - for (i = 0; i < master->num_sids; i++) { - u32 sid = master->sids[i]; - - if (!arm_smmu_sid_in_range(smmu, sid)) { - ret = -ERANGE; - goto err_free_master; - } - - /* Ensure l2 strtab is initialised */ - if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) { - ret = arm_smmu_init_l2_strtab(smmu, sid); - if (ret) - goto err_free_master; - } - } + ret = arm_smmu_insert_master(smmu, master); + if (ret) + goto err_free_master; device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits); master->ssid_bits = min(smmu->ssid_bits, master->ssid_bits); @@ -2429,6 +2530,7 @@ static void arm_smmu_release_device(struct device *dev) WARN_ON(arm_smmu_master_sva_enabled(master)); arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); kfree(master); iommu_fwspec_free(dev); } @@ -2852,6 +2954,9 @@ static int arm_smmu_init_structures(struct arm_smmu_device *smmu) { int ret; + mutex_init(&smmu->streams_mutex); + smmu->streams = RB_ROOT; + ret = arm_smmu_init_queues(smmu); if (ret) return ret; From patchwork Wed Jan 27 15:43:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59CE4C433DB for ; Wed, 27 Jan 2021 15:50:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 08E90207C4 for ; Wed, 27 Jan 2021 15:50:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236004AbhA0Pu1 (ORCPT ); Wed, 27 Jan 2021 10:50:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236645AbhA0PqX (ORCPT ); Wed, 27 Jan 2021 10:46:23 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A567C061353 for ; Wed, 27 Jan 2021 07:44:45 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id i9so2081526wmq.1 for ; Wed, 27 Jan 2021 07:44:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Iza7ai+1pJVkOiREPpK3tUFt/+4KEOcT4kmdIoRu1W4=; b=rbK1aLxadekkDLedlYkhXNNhT2RXl9ArKYIMq5P+xZUhGoeMuNK3hNiCz/AdZv7dFV OSkzpHq8cAXoa/SXmQczZVVsMDUa3zgVBPkpfFJsnMVHytFJ4lPtF7iVgQDdouVwknB6 PKAJ3I0JTJnun9bq1lGNt8cZHMDadB4iIeVL949UJaWXlk8wdS6to9Ss+zGxoSi5GdbL rQVmX44XNm3Yoa6Lp0sLLAnANdFnGsOPlibkhusNtF+Vv0m8ZbNcxG4L4D1R1D/UxyLB XIv8tC7yp0wBkuUigCeJ+XCpBK2QPUanxRD8fKUUvRyaioR65KVTDBWpW69znMpePx9P xlLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Iza7ai+1pJVkOiREPpK3tUFt/+4KEOcT4kmdIoRu1W4=; b=IDICIoyTfLqrfz1jEXP4aIin5niP2sbl7rtFuH6ioZV23lpawAkN9Ze2wZzfU1RVrV KjlqtX9n4hfGxm7m5+TbdvvyBw4b2sWex+33vNquGOEr60ouqvdnzA/XH5kfm17eC0eR oCQ/xsuEJ3sbMt3ULHfcCy01LTrCh+iyvhGsVOMA5ir393YAAp/R+7MvsKD0nlsZBk6V Qsh6zExDZyQessrXjkjTPYJFD+EDAPs3JdckE3MgqaFYfKsXj7czVimGMGo7yoq2SrhH gSuTWG1viTm8dFFLh4MgBb+cw4rrIwJpFyMN+WqufEwN6XTHbl+QdWUTegHx1prFXBUK wyrg== X-Gm-Message-State: AOAM533PU7y+tGWUc4FHc2eZDZJwaspDp6QgO7C4YOhK9IZou0n64Zjn MrXqmv0c5ffk/ZXXthCpH1OoLA== X-Google-Smtp-Source: ABdhPJxueLPbEIo5H0DTOaZMTDZgKa24SKwcOb0gi9WAMioQDQ/yigHLDYzBi1GocdkersH1im+GCg== X-Received: by 2002:a1c:2c0b:: with SMTP id s11mr4542706wms.13.1611762284206; Wed, 27 Jan 2021 07:44:44 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:43 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker , Rob Herring Subject: [PATCH v12 08/10] dt-bindings: document stall property for IOMMU masters Date: Wed, 27 Jan 2021 16:43:21 +0100 Message-Id: <20210127154322.3959196-9-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org On ARM systems, some platform devices behind an IOMMU may support stall, which is the ability to recover from page faults. Let the firmware tell us when a device supports stall. Reviewed-by: Rob Herring Signed-off-by: Jean-Philippe Brucker Reviewed-by: Eric Auger --- .../devicetree/bindings/iommu/iommu.txt | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt b/Documentation/devicetree/bindings/iommu/iommu.txt index 3c36334e4f94..26ba9e530f13 100644 --- a/Documentation/devicetree/bindings/iommu/iommu.txt +++ b/Documentation/devicetree/bindings/iommu/iommu.txt @@ -92,6 +92,24 @@ Optional properties: tagging DMA transactions with an address space identifier. By default, this is 0, which means that the device only has one address space. +- dma-can-stall: When present, the master can wait for a transaction to + complete for an indefinite amount of time. Upon translation fault some + IOMMUs, instead of aborting the translation immediately, may first + notify the driver and keep the transaction in flight. This allows the OS + to inspect the fault and, for example, make physical pages resident + before updating the mappings and completing the transaction. Such IOMMU + accepts a limited number of simultaneous stalled transactions before + having to either put back-pressure on the master, or abort new faulting + transactions. + + Firmware has to opt-in stalling, because most buses and masters don't + support it. In particular it isn't compatible with PCI, where + transactions have to complete before a time limit. More generally it + won't work in systems and masters that haven't been designed for + stalling. For example the OS, in order to handle a stalled transaction, + may attempt to retrieve pages from secondary storage in a stalled + domain, leading to a deadlock. + Notes: ====== From patchwork Wed Jan 27 15:43:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 371568 Delivered-To: patch@linaro.org Received: by 2002:a02:a60d:0:0:0:0:0 with SMTP id c13csp386342jam; Wed, 27 Jan 2021 07:48:15 -0800 (PST) X-Google-Smtp-Source: ABdhPJymzw9nOQX7M8XjoDjOrTSl/v4ss5ctu+5ifhahA8s5Vv2gkO/m0DxZdZsb5MU87XbrT6f6 X-Received: by 2002:a17:907:9810:: with SMTP id ji16mr7065545ejc.394.1611762495421; Wed, 27 Jan 2021 07:48:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611762495; cv=none; d=google.com; s=arc-20160816; b=yfLGy+8YNPCN8QB1/k0XsCLxBgOYqeE3xg9rd+P5sMfDNMf8oQTa90chNAqHJ26ic2 cSUHTfhTfCUSFntQiVuMUTm+KUZL4bn/ukM2ef7Nsah2pwC5IFosk3vzWlwy21Q0aHQV Mnx+u/t1V1Uov1hJSUm2w2DscPCxZ/zq+68ekRxipspj0W/uNO9RhT1E8nNwJaFNGoue +lufqOUJupfBruyLpz5QAPXew8pvHNxmvY4AmvLfLHgZfNv4vChDd/v/DJi7JGyToNBk 14XO3olxXWGw/wQt+pdPhMlJ3Dw2zWj6BuzYLBGwCmQz6T3hgybpvoledkyp2Zde5/1S /4rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=P/EAy3wFo2SkAvzlEqANsfgzLeO0MB9eAURTmLNsvNA=; b=ZCHRj+qrFsqZFDAJ7dxPArxKqPQWrts3LOJSZaZsZB17jc+9GtUsyvziHqeB0ikShg tZG8DJwtAOdmGl8ySEu6j7NuvYybhUi1GWoSSVMa2KyXByuUOkUT+LdVwKbXtuL71lVL 31CjeqBQpbY4dl6c5lJllrIpIFtYf4PhksZokGV2fgbJRu/QcxJ6zO2IlhzuiyP5c/VF tkgMq9Wbv+RyDh+KP4WqqA4hK4FKfKqBA7uFV0x2AvcbZPdBYYD8gafct/jAEAcbEI7R 5XM+l+coMnYNyPjOKQZRHKZb3XqTF7nZ7HL8zdfdeFZ680FpbI3iS0gJVE1+T3TDYLiT qLWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mb07EcDa; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cy17si1201994edb.193.2021.01.27.07.48.15; Wed, 27 Jan 2021 07:48:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mb07EcDa; spf=pass (google.com: domain of linux-acpi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-acpi-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236206AbhA0PsE (ORCPT + 4 others); Wed, 27 Jan 2021 10:48:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236651AbhA0Pqj (ORCPT ); Wed, 27 Jan 2021 10:46:39 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3079C061356 for ; Wed, 27 Jan 2021 07:44:46 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id f16so1910318wmq.5 for ; Wed, 27 Jan 2021 07:44:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P/EAy3wFo2SkAvzlEqANsfgzLeO0MB9eAURTmLNsvNA=; b=mb07EcDaQibTEwHyo1+my09q6bkkLrDK+31eVBj3xeA3p0boN8Cnn6DHCFEjErg6ay 1qVr03v/rFH29yu2BvCS2DbIo97LHYGyuv88KKQNM2MTQ2+U4Bl3vu4LuBfUSrrG7uak 6aDNhLCw2Q8ymqA8LvqcjEjg/PNki0B2ltIM9AHHsXFIhulJH5o+h060GC3ZenURtkBN 8ysP23TmFVJIj6bNYAQip6zL0bPvbTVeQPIGR30xGFngU0oajhJjG2BRFpYrUvvg5WBo NmGSP9P24o0UZJ3BVjroLLiiBev9KxkbokaPnFyaRq/BegljrmRMi+x5tle1qwRvtGKF m2jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P/EAy3wFo2SkAvzlEqANsfgzLeO0MB9eAURTmLNsvNA=; b=MQeNnLTTogDo4oeudoZRnHUeOEUMRNq4+7K8+/IQmafNucBIpV9bICt/aJXQfL2KQr 3DHOshInodBSsnbkFx/bLRwhPu6wmxckjhQ8P8hcnpSyBUAFx3kTRuauiMYPmzquj4hf BJDko7KLM37W2N05xrax2vLIvkMOtMFqiXgkAPT39wiKE6AD92Q/xP3h1Bc7+G/SmLuG IPSXv3zGusOrIVQTM+ogWwkyQH9wfGcBE0fhX0dXaxNAXXrP2RuAAn44xJRddBsJ6UQm yWG2Jlczuvl80phZKBu8JmnAg2bMPgE7XxJK/z6QbYqsIzUMr5Z7eKWK+Ywju4vknQAr hf7w== X-Gm-Message-State: AOAM530o/aOp7xK3qxJXL91COoMnaZ/+NmSa0hjZs9eZZMILNse0I0yY hUYX1X3LuGtc7Cb7KvKUF40Gwg== X-Received: by 2002:a1c:e407:: with SMTP id b7mr4745128wmh.46.1611762285528; Wed, 27 Jan 2021 07:44:45 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:44 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v12 09/10] ACPI/IORT: Enable stall support for platform devices Date: Wed, 27 Jan 2021 16:43:22 +0100 Message-Id: <20210127154322.3959196-10-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Copy the "Stall supported" bit, that tells whether a named component supports stall, into the dma-can-stall device property. Acked-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- drivers/acpi/arm64/iort.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 2.30.0 diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index c9a8bbb74b09..42820d7eb869 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -968,13 +968,15 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data) static void iort_named_component_init(struct device *dev, struct acpi_iort_node *node) { - struct property_entry props[2] = {}; + struct property_entry props[3] = {}; struct acpi_iort_named_component *nc; nc = (struct acpi_iort_named_component *)node->node_data; props[0] = PROPERTY_ENTRY_U32("pasid-num-bits", FIELD_GET(ACPI_IORT_NC_PASID_BITS, nc->node_flags)); + if (nc->node_flags & ACPI_IORT_NC_STALL_SUPPORTED) + props[1] = PROPERTY_ENTRY_BOOL("dma-can-stall"); if (device_add_properties(dev, props)) dev_warn(dev, "Could not add device properties\n"); From patchwork Wed Jan 27 15:43:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 372671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04C2FC433E0 for ; Wed, 27 Jan 2021 15:49:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AAA80207B7 for ; Wed, 27 Jan 2021 15:49:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234201AbhA0Ptf (ORCPT ); Wed, 27 Jan 2021 10:49:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236355AbhA0Pre (ORCPT ); Wed, 27 Jan 2021 10:47:34 -0500 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BE7AC061355 for ; Wed, 27 Jan 2021 07:44:48 -0800 (PST) Received: by mail-wm1-x32d.google.com with SMTP id c127so2069325wmf.5 for ; Wed, 27 Jan 2021 07:44:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3j6gTVXiPvyZMvKzX3hwPYdWoLZVmtHXkBJNHleAv7c=; b=kRO+O5RjbL84aphvIQJPRTiQqrcO8k1kFYmNnvhY7ajKqpLv/Yp2M9KMHXxHRJ4z1I TPZ8u+c4VlYmNRMDpQJLrax52rvudCMsoNDPE4xSOueGMT1JEig+a3q8H1pQ/HdHRRCR 9nvRw8oVkJXlXm1IB1lZvHXVOi+lWt6bGbRJW7LCnODcLmfyCCZGJWO5/sTn1AUkcaKQ 3LcSrwn+rBnkx1gwRxrowhpZCg/SRdt08YFOp4HgBBd4sEa+RVxqGmlXOl1E9ccpVLBt Qvlli4zohTkaKHZiu1Pta9b4sUO9bMOcHkIMgAkuR2e4EkRUdB9CSrvJfWp77/XylQ8/ 0xtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3j6gTVXiPvyZMvKzX3hwPYdWoLZVmtHXkBJNHleAv7c=; b=bsmJ+qAuL3eZwuCPUHUsEOGWJeETdMrkXLjZzon2gBxsxopu745bJI8nqYDiCjkuzS 6aCbD+M9dTm+S+lEgDuoko+7EkB9syK85J00SgNeKK4EL7Jrz7ri0kBEAdwpy1H7kcL7 PlFCIz5rHZiFlq4DJ9sUlqwF7zgyV6zqPbrabfwiZisUJwrfsf1p3i1sb5C3gNWVE1e3 mKE5KCWAT4VRcKG83BKQghYmqV6/jqy1C/WFYN+EhX+OmxgWeBIx080cbJ0oKlOQ/6nz uqLIi8gK+4XNBzU8wuDLief4noTAsdPna1jjGc9zAEVxP7OY622Ye2c/Y7f6PMLhVlMQ q2zg== X-Gm-Message-State: AOAM533kYy7Xzzx9hl8USJsarz4DWt9Ixk5fJAKgYnJe5J2PU07EkieV GBdfZT3ZMoAVwze/g9G10IJzdQ== X-Google-Smtp-Source: ABdhPJz/m4On7ioP3U2GbVtBp43SXWB08VAxOMfBrLkJoTJYcLqOJiCiEWuLY18E4q8kslr97shSgQ== X-Received: by 2002:a7b:c188:: with SMTP id y8mr4557662wmi.173.1611762286844; Wed, 27 Jan 2021 07:44:46 -0800 (PST) Received: from localhost.localdomain ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id r13sm3046921wmh.9.2021.01.27.07.44.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jan 2021 07:44:46 -0800 (PST) From: Jean-Philippe Brucker To: joro@8bytes.org, will@kernel.org Cc: lorenzo.pieralisi@arm.com, robh+dt@kernel.org, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, Jonathan.Cameron@huawei.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-acpi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-accelerators@lists.ozlabs.org, baolu.lu@linux.intel.com, jacob.jun.pan@linux.intel.com, kevin.tian@intel.com, vdumpa@nvidia.com, zhangfei.gao@linaro.org, shameerali.kolothum.thodi@huawei.com, vivek.gautam@arm.com, Jean-Philippe Brucker Subject: [PATCH v12 10/10] iommu/arm-smmu-v3: Add stall support for platform devices Date: Wed, 27 Jan 2021 16:43:23 +0100 Message-Id: <20210127154322.3959196-11-jean-philippe@linaro.org> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210127154322.3959196-1-jean-philippe@linaro.org> References: <20210127154322.3959196-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org The SMMU provides a Stall model for handling page faults in platform devices. It is similar to PCIe PRI, but doesn't require devices to have their own translation cache. Instead, faulting transactions are parked and the OS is given a chance to fix the page tables and retry the transaction. Enable stall for devices that support it (opt-in by firmware). When an event corresponds to a translation error, call the IOMMU fault handler. If the fault is recoverable, it will call us back to terminate or continue the stall. To use stall device drivers need to enable IOMMU_DEV_FEAT_IOPF, which initializes the fault queue for the device. Tested-by: Zhangfei Gao Reviewed-by: Jonathan Cameron Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 43 ++++ .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 59 +++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 189 +++++++++++++++++- 3 files changed, 276 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 7b15b7580c6e..59af0bbd2f7b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -354,6 +354,13 @@ #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) #define CMDQ_PRI_1_RESP GENMASK_ULL(13, 12) +#define CMDQ_RESUME_0_RESP_TERM 0UL +#define CMDQ_RESUME_0_RESP_RETRY 1UL +#define CMDQ_RESUME_0_RESP_ABORT 2UL +#define CMDQ_RESUME_0_RESP GENMASK_ULL(13, 12) +#define CMDQ_RESUME_0_SID GENMASK_ULL(63, 32) +#define CMDQ_RESUME_1_STAG GENMASK_ULL(15, 0) + #define CMDQ_SYNC_0_CS GENMASK_ULL(13, 12) #define CMDQ_SYNC_0_CS_NONE 0 #define CMDQ_SYNC_0_CS_IRQ 1 @@ -370,6 +377,25 @@ #define EVTQ_0_ID GENMASK_ULL(7, 0) +#define EVT_ID_TRANSLATION_FAULT 0x10 +#define EVT_ID_ADDR_SIZE_FAULT 0x11 +#define EVT_ID_ACCESS_FAULT 0x12 +#define EVT_ID_PERMISSION_FAULT 0x13 + +#define EVTQ_0_SSV (1UL << 11) +#define EVTQ_0_SSID GENMASK_ULL(31, 12) +#define EVTQ_0_SID GENMASK_ULL(63, 32) +#define EVTQ_1_STAG GENMASK_ULL(15, 0) +#define EVTQ_1_STALL (1UL << 31) +#define EVTQ_1_PnU (1UL << 33) +#define EVTQ_1_InD (1UL << 34) +#define EVTQ_1_RnW (1UL << 35) +#define EVTQ_1_S2 (1UL << 39) +#define EVTQ_1_CLASS GENMASK_ULL(41, 40) +#define EVTQ_1_TT_READ (1UL << 44) +#define EVTQ_2_ADDR GENMASK_ULL(63, 0) +#define EVTQ_3_IPA GENMASK_ULL(51, 12) + /* PRI queue */ #define PRIQ_ENT_SZ_SHIFT 4 #define PRIQ_ENT_DWORDS ((1 << PRIQ_ENT_SZ_SHIFT) >> 3) @@ -464,6 +490,13 @@ struct arm_smmu_cmdq_ent { enum pri_resp resp; } pri; + #define CMDQ_OP_RESUME 0x44 + struct { + u32 sid; + u16 stag; + u8 resp; + } resume; + #define CMDQ_OP_CMD_SYNC 0x46 struct { u64 msiaddr; @@ -522,6 +555,7 @@ struct arm_smmu_cmdq_batch { struct arm_smmu_evtq { struct arm_smmu_queue q; + struct iopf_queue *iopf; u32 max_stalls; }; @@ -659,7 +693,9 @@ struct arm_smmu_master { struct arm_smmu_stream *streams; unsigned int num_streams; bool ats_enabled; + bool stall_enabled; bool sva_enabled; + bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; }; @@ -678,6 +714,7 @@ struct arm_smmu_domain { struct io_pgtable_ops *pgtbl_ops; bool non_strict; + bool stall_enabled; atomic_t nr_ats_masters; enum arm_smmu_domain_stage stage; @@ -719,6 +756,7 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master *master); bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master); int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); +bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); struct iommu_sva *arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata); void arm_smmu_sva_unbind(struct iommu_sva *handle); @@ -750,6 +788,11 @@ static inline int arm_smmu_master_disable_sva(struct arm_smmu_master *master) return -ENODEV; } +static inline bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master) +{ + return false; +} + static inline struct iommu_sva * arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm, void *drvdata) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index bb251cab61f3..ee66d1f4cb81 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -435,9 +435,13 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) return true; } -static bool arm_smmu_iopf_supported(struct arm_smmu_master *master) +bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master) { - return false; + /* We're not keeping track of SIDs in fault events */ + if (master->num_streams != 1) + return false; + + return master->stall_enabled; } bool arm_smmu_master_sva_supported(struct arm_smmu_master *master) @@ -445,8 +449,8 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master *master) if (!(master->smmu->features & ARM_SMMU_FEAT_SVA)) return false; - /* SSID and IOPF support are mandatory for the moment */ - return master->ssid_bits && arm_smmu_iopf_supported(master); + /* SSID support is mandatory for the moment */ + return master->ssid_bits; } bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master) @@ -459,13 +463,55 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master) return enabled; } +static int arm_smmu_master_sva_enable_iopf(struct arm_smmu_master *master) +{ + int ret; + struct device *dev = master->dev; + + /* + * Drivers for devices supporting PRI or stall should enable IOPF first. + * Others have device-specific fault handlers and don't need IOPF. + */ + if (!arm_smmu_master_iopf_supported(master)) + return 0; + + if (!master->iopf_enabled) + return -EINVAL; + + ret = iopf_queue_add_device(master->smmu->evtq.iopf, dev); + if (ret) + return ret; + + ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); + if (ret) { + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + return ret; + } + return 0; +} + +static void arm_smmu_master_sva_disable_iopf(struct arm_smmu_master *master) +{ + struct device *dev = master->dev; + + if (!master->iopf_enabled) + return; + + iommu_unregister_device_fault_handler(dev); + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); +} + int arm_smmu_master_enable_sva(struct arm_smmu_master *master) { + int ret; + mutex_lock(&sva_lock); - master->sva_enabled = true; + ret = arm_smmu_master_sva_enable_iopf(master); + if (!ret) + master->sva_enabled = true; mutex_unlock(&sva_lock); - return 0; + return ret; } int arm_smmu_master_disable_sva(struct arm_smmu_master *master) @@ -476,6 +522,7 @@ int arm_smmu_master_disable_sva(struct arm_smmu_master *master) mutex_unlock(&sva_lock); return -EBUSY; } + arm_smmu_master_sva_disable_iopf(master); master->sva_enabled = false; mutex_unlock(&sva_lock); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3afec6ed8075..76b2306ddff6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -32,6 +32,7 @@ #include #include "arm-smmu-v3.h" +#include "../../iommu-sva-lib.h" static bool disable_bypass = true; module_param(disable_bypass, bool, 0444); @@ -315,6 +316,11 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) } cmd[1] |= FIELD_PREP(CMDQ_PRI_1_RESP, ent->pri.resp); break; + case CMDQ_OP_RESUME: + cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_SID, ent->resume.sid); + cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_RESP, ent->resume.resp); + cmd[1] |= FIELD_PREP(CMDQ_RESUME_1_STAG, ent->resume.stag); + break; case CMDQ_OP_CMD_SYNC: if (ent->sync.msiaddr) { cmd[0] |= FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_IRQ); @@ -878,6 +884,44 @@ static int arm_smmu_cmdq_batch_submit(struct arm_smmu_device *smmu, return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmds, cmds->num, true); } +static int arm_smmu_page_response(struct device *dev, + struct iommu_fault_event *unused, + struct iommu_page_response *resp) +{ + struct arm_smmu_cmdq_ent cmd = {0}; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + int sid = master->streams[0].id; + + if (master->stall_enabled) { + cmd.opcode = CMDQ_OP_RESUME; + cmd.resume.sid = sid; + cmd.resume.stag = resp->grpid; + switch (resp->code) { + case IOMMU_PAGE_RESP_INVALID: + case IOMMU_PAGE_RESP_FAILURE: + cmd.resume.resp = CMDQ_RESUME_0_RESP_ABORT; + break; + case IOMMU_PAGE_RESP_SUCCESS: + cmd.resume.resp = CMDQ_RESUME_0_RESP_RETRY; + break; + default: + return -EINVAL; + } + } else { + return -ENODEV; + } + + arm_smmu_cmdq_issue_cmd(master->smmu, &cmd); + /* + * Don't send a SYNC, it doesn't do anything for RESUME or PRI_RESP. + * RESUME consumption guarantees that the stalled transaction will be + * terminated... at some point in the future. PRI_RESP is fire and + * forget. + */ + + return 0; +} + /* Context descriptor manipulation functions */ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid) { @@ -988,7 +1032,6 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, u64 val; bool cd_live; __le64 *cdptr; - struct arm_smmu_device *smmu = smmu_domain->smmu; if (WARN_ON(ssid >= (1 << smmu_domain->s1_cfg.s1cdmax))) return -E2BIG; @@ -1033,8 +1076,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; - /* STALL_MODEL==0b10 && CD.S==0 is ILLEGAL */ - if (smmu->features & ARM_SMMU_FEAT_STALL_FORCE) + if (smmu_domain->stall_enabled) val |= CTXDESC_CD_0_S; } @@ -1278,7 +1320,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, FIELD_PREP(STRTAB_STE_1_STRW, strw)); if (smmu->features & ARM_SMMU_FEAT_STALLS && - !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) + !master->stall_enabled) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); val |= (s1_cfg->cdcfg.cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) | @@ -1355,7 +1397,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) return 0; } -__maybe_unused static struct arm_smmu_master * arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) { @@ -1382,9 +1423,96 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) } /* IRQ and event handlers */ +static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) +{ + int ret; + u32 reason; + u32 perm = 0; + struct arm_smmu_master *master; + bool ssid_valid = evt[0] & EVTQ_0_SSV; + u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]); + struct iommu_fault_event fault_evt = { }; + struct iommu_fault *flt = &fault_evt.fault; + + /* Stage-2 is always pinned at the moment */ + if (evt[1] & EVTQ_1_S2) + return -EFAULT; + + master = arm_smmu_find_master(smmu, sid); + if (!master) + return -EINVAL; + + if (evt[1] & EVTQ_1_RnW) + perm |= IOMMU_FAULT_PERM_READ; + else + perm |= IOMMU_FAULT_PERM_WRITE; + + if (evt[1] & EVTQ_1_InD) + perm |= IOMMU_FAULT_PERM_EXEC; + + if (evt[1] & EVTQ_1_PnU) + perm |= IOMMU_FAULT_PERM_PRIV; + + switch (FIELD_GET(EVTQ_0_ID, evt[0])) { + case EVT_ID_TRANSLATION_FAULT: + case EVT_ID_ADDR_SIZE_FAULT: + case EVT_ID_ACCESS_FAULT: + reason = IOMMU_FAULT_REASON_PTE_FETCH; + break; + case EVT_ID_PERMISSION_FAULT: + reason = IOMMU_FAULT_REASON_PERMISSION; + break; + default: + return -EOPNOTSUPP; + } + + if (evt[1] & EVTQ_1_STALL) { + flt->type = IOMMU_FAULT_PAGE_REQ; + flt->prm = (struct iommu_fault_page_request) { + .flags = IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE, + .grpid = FIELD_GET(EVTQ_1_STAG, evt[1]), + .perm = perm, + .addr = FIELD_GET(EVTQ_2_ADDR, evt[2]), + }; + + if (ssid_valid) { + flt->prm.flags |= IOMMU_FAULT_PAGE_REQUEST_PASID_VALID; + flt->prm.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]); + } + } else { + flt->type = IOMMU_FAULT_DMA_UNRECOV; + flt->event = (struct iommu_fault_unrecoverable) { + .reason = reason, + .flags = IOMMU_FAULT_UNRECOV_ADDR_VALID | + IOMMU_FAULT_UNRECOV_FETCH_ADDR_VALID, + .perm = perm, + .addr = FIELD_GET(EVTQ_2_ADDR, evt[2]), + .fetch_addr = FIELD_GET(EVTQ_3_IPA, evt[3]), + }; + + if (ssid_valid) { + flt->event.flags |= IOMMU_FAULT_UNRECOV_PASID_VALID; + flt->event.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]); + } + } + + ret = iommu_report_device_fault(master->dev, &fault_evt); + if (ret && flt->type == IOMMU_FAULT_PAGE_REQ) { + /* Nobody cared, abort the access */ + struct iommu_page_response resp = { + .pasid = flt->prm.pasid, + .grpid = flt->prm.grpid, + .code = IOMMU_PAGE_RESP_FAILURE, + }; + arm_smmu_page_response(master->dev, &fault_evt, &resp); + } + + return ret; +} + static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { - int i; + int i, ret; struct arm_smmu_device *smmu = dev; struct arm_smmu_queue *q = &smmu->evtq.q; struct arm_smmu_ll_queue *llq = &q->llq; @@ -1394,6 +1522,10 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) while (!queue_remove_raw(q, evt)) { u8 id = FIELD_GET(EVTQ_0_ID, evt[0]); + ret = arm_smmu_handle_evt(smmu, evt); + if (!ret) + continue; + dev_info(smmu->dev, "event 0x%02x received:\n", id); for (i = 0; i < ARRAY_SIZE(evt); ++i) dev_info(smmu->dev, "\t0x%016llx\n", @@ -1928,6 +2060,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, cfg->s1cdmax = master->ssid_bits; + smmu_domain->stall_enabled = master->stall_enabled; + ret = arm_smmu_alloc_cd_tables(smmu_domain); if (ret) goto out_free_asid; @@ -2275,6 +2409,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu_domain->s1_cfg.s1cdmax, master->ssid_bits); ret = -EINVAL; goto out_unlock; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && + smmu_domain->stall_enabled != master->stall_enabled) { + dev_err(dev, "cannot attach to stall-%s domain\n", + smmu_domain->stall_enabled ? "enabled" : "disabled"); + ret = -EINVAL; + goto out_unlock; } master->domain = smmu_domain; @@ -2510,6 +2650,11 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->ssid_bits = min_t(u8, master->ssid_bits, CTXDESC_LINEAR_CDMAX); + if ((smmu->features & ARM_SMMU_FEAT_STALLS && + device_property_read_bool(dev, "dma-can-stall")) || + smmu->features & ARM_SMMU_FEAT_STALL_FORCE) + master->stall_enabled = true; + return &smmu->iommu; err_free_master: @@ -2527,7 +2672,8 @@ static void arm_smmu_release_device(struct device *dev) return; master = dev_iommu_priv_get(dev); - WARN_ON(arm_smmu_master_sva_enabled(master)); + if (WARN_ON(arm_smmu_master_sva_enabled(master))) + iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); @@ -2655,6 +2801,8 @@ static bool arm_smmu_dev_has_feature(struct device *dev, return false; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + return arm_smmu_master_iopf_supported(master); case IOMMU_DEV_FEAT_SVA: return arm_smmu_master_sva_supported(master); default: @@ -2671,6 +2819,8 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev, return false; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + return master->iopf_enabled; case IOMMU_DEV_FEAT_SVA: return arm_smmu_master_sva_enabled(master); default: @@ -2681,6 +2831,8 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev, static int arm_smmu_dev_enable_feature(struct device *dev, enum iommu_dev_features feat) { + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + if (!arm_smmu_dev_has_feature(dev, feat)) return -ENODEV; @@ -2688,8 +2840,11 @@ static int arm_smmu_dev_enable_feature(struct device *dev, return -EBUSY; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + master->iopf_enabled = true; + return 0; case IOMMU_DEV_FEAT_SVA: - return arm_smmu_master_enable_sva(dev_iommu_priv_get(dev)); + return arm_smmu_master_enable_sva(master); default: return -EINVAL; } @@ -2698,12 +2853,19 @@ static int arm_smmu_dev_enable_feature(struct device *dev, static int arm_smmu_dev_disable_feature(struct device *dev, enum iommu_dev_features feat) { + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + if (!arm_smmu_dev_feature_enabled(dev, feat)) return -EINVAL; switch (feat) { + case IOMMU_DEV_FEAT_IOPF: + if (master->sva_enabled) + return -EBUSY; + master->iopf_enabled = false; + return 0; case IOMMU_DEV_FEAT_SVA: - return arm_smmu_master_disable_sva(dev_iommu_priv_get(dev)); + return arm_smmu_master_disable_sva(master); default: return -EINVAL; } @@ -2734,6 +2896,7 @@ static struct iommu_ops arm_smmu_ops = { .sva_bind = arm_smmu_sva_bind, .sva_unbind = arm_smmu_sva_unbind, .sva_get_pasid = arm_smmu_sva_get_pasid, + .page_response = arm_smmu_page_response, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; @@ -2831,6 +2994,13 @@ static int arm_smmu_init_queues(struct arm_smmu_device *smmu) if (ret) return ret; + if ((smmu->features & ARM_SMMU_FEAT_SVA) && + (smmu->features & ARM_SMMU_FEAT_STALLS)) { + smmu->evtq.iopf = iopf_queue_alloc(dev_name(smmu->dev)); + if (!smmu->evtq.iopf) + return -ENOMEM; + } + /* priq */ if (!(smmu->features & ARM_SMMU_FEAT_PRI)) return 0; @@ -3746,6 +3916,7 @@ static int arm_smmu_device_remove(struct platform_device *pdev) iommu_device_unregister(&smmu->iommu); iommu_device_sysfs_remove(&smmu->iommu); arm_smmu_device_disable(smmu); + iopf_queue_free(smmu->evtq.iopf); return 0; }