| Message ID | 20220606061927.26049-2-nicolinc@nvidia.com |
|---|---|
| State | New |
| Headers | show |
| Series | Simplify vfio_iommu_type1 attach/detach routine | expand |
On Tue, Jun 07, 2022 at 11:23:27AM +0800, Baolu Lu wrote: > External email: Use caution opening links or attachments > > > On 2022/6/6 14:19, Nicolin Chen wrote: > > +/** > > + * iommu_attach_group - Attach an IOMMU group to an IOMMU domain > > + * @domain: IOMMU domain to attach > > + * @dev: IOMMU group that will be attached > > Nit: @group: ... Oh...Thanks!
> From: Nicolin Chen > Sent: Monday, June 6, 2022 2:19 PM > > Cases like VFIO wish to attach a device to an existing domain that was > not allocated specifically from the device. This raises a condition > where the IOMMU driver can fail the domain attach because the domain and > device are incompatible with each other. > > This is a soft failure that can be resolved by using a different domain. > > Provide a dedicated errno from the IOMMU driver during attach that the > reason attached failed is because of domain incompatability. EMEDIUMTYPE > is chosen because it is never used within the iommu subsystem today and > evokes a sense that the 'medium' aka the domain is incompatible. > > VFIO can use this to know attach is a soft failure and it should continue > searching. Otherwise the attach will be a hard failure and VFIO will > return the code to userspace. > > Update all drivers to return EMEDIUMTYPE in their failure paths that are > related to domain incompatability. Seems not all drivers are converted, e.g.: mtk_iommu_v1_attach_device(): /* Only allow the domain created internally. */ mtk_mapping = data->mapping; if (mtk_mapping->domain != domain) return 0; ** the current code sounds incorrect which should return an error s390_iommu_attach_device(): /* Allow only devices with identical DMA range limits */ } else if (domain->geometry.aperture_start != zdev->start_dma || domain->geometry.aperture_end != zdev->end_dma) { rc = -EINVAL; sprd_iommu_attach_device(): if (dom->sdev) { pr_err("There's already a device attached to this domain.\n"); return -EINVAL; } gart_iommu_attach_dev(): if (gart->active_domain && gart->active_domain != domain) { ret = -EBUSY; arm_smmu_attach_dev(): if (!fwspec || fwspec->ops != &arm_smmu_ops) { dev_err(dev, "cannot attach to SMMU, is it on the same bus?\n"); return -ENXIO; } **probably this check can be covered by next patch which moves bus ops check into iommu core? Thanks Kevin
Hi Kevin, On Wed, Jun 08, 2022 at 07:49:10AM +0000, Tian, Kevin wrote: > External email: Use caution opening links or attachments > > > > From: Nicolin Chen > > Sent: Monday, June 6, 2022 2:19 PM > > > > Cases like VFIO wish to attach a device to an existing domain that was > > not allocated specifically from the device. This raises a condition > > where the IOMMU driver can fail the domain attach because the domain and > > device are incompatible with each other. > > > > This is a soft failure that can be resolved by using a different domain. > > > > Provide a dedicated errno from the IOMMU driver during attach that the > > reason attached failed is because of domain incompatability. EMEDIUMTYPE > > is chosen because it is never used within the iommu subsystem today and > > evokes a sense that the 'medium' aka the domain is incompatible. > > > > VFIO can use this to know attach is a soft failure and it should continue > > searching. Otherwise the attach will be a hard failure and VFIO will > > return the code to userspace. > > > > Update all drivers to return EMEDIUMTYPE in their failure paths that are > > related to domain incompatability. > > Seems not all drivers are converted, e.g.: Thank you for going through all of them! > mtk_iommu_v1_attach_device(): > /* Only allow the domain created internally. */ > mtk_mapping = data->mapping; > if (mtk_mapping->domain != domain) > return 0; > ** the current code sounds incorrect which should return an error I agree. > s390_iommu_attach_device(): > /* Allow only devices with identical DMA range limits */ > } else if (domain->geometry.aperture_start != zdev->start_dma || > domain->geometry.aperture_end != zdev->end_dma) { > rc = -EINVAL; > > sprd_iommu_attach_device(): > if (dom->sdev) { > pr_err("There's already a device attached to this domain.\n"); > return -EINVAL; > } > > > gart_iommu_attach_dev(): > if (gart->active_domain && gart->active_domain != domain) { > ret = -EBUSY; Will add these. > arm_smmu_attach_dev(): > if (!fwspec || fwspec->ops != &arm_smmu_ops) { > dev_err(dev, "cannot attach to SMMU, is it on the same bus?\n"); > return -ENXIO; > } > **probably this check can be covered by next patch which moves bus ops > check into iommu core? I was thinking that it could be covered. Yet, we are about to drop that ops check, as Robin pointed out that we don't need that ops sanity for we don't have mixed-driver systems yet. So perhaps this would be a different ops check and should return -EMEDIUMTYPE too, I think. Thanks Nic
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 840831d5d2ad..ad499658a6b6 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1662,7 +1662,7 @@ static int attach_device(struct device *dev, if (domain->flags & PD_IOMMUV2_MASK) { struct iommu_domain *def_domain = iommu_get_dma_domain(dev); - ret = -EINVAL; + ret = -EMEDIUMTYPE; if (def_domain->type != IOMMU_DOMAIN_IDENTITY) goto out; diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c index 8af0242a90d9..e58dc310afd7 100644 --- a/drivers/iommu/apple-dart.c +++ b/drivers/iommu/apple-dart.c @@ -495,10 +495,10 @@ static int apple_dart_attach_dev(struct iommu_domain *domain, if (cfg->stream_maps[0].dart->force_bypass && domain->type != IOMMU_DOMAIN_IDENTITY) - return -EINVAL; + return -EMEDIUMTYPE; if (!cfg->stream_maps[0].dart->supports_bypass && domain->type == IOMMU_DOMAIN_IDENTITY) - return -EINVAL; + return -EMEDIUMTYPE; ret = apple_dart_finalize_domain(domain, cfg); if (ret) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 88817a3376ef..6c393cd84925 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2424,20 +2424,20 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) "cannot attach to SMMU %s (upstream of %s)\n", dev_name(smmu_domain->smmu->dev), dev_name(smmu->dev)); - ret = -ENXIO; + ret = -EMEDIUMTYPE; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { dev_err(dev, "cannot attach to incompatible domain (%u SSID bits != %u)\n", smmu_domain->s1_cfg.s1cdmax, master->ssid_bits); - ret = -EINVAL; + ret = -EMEDIUMTYPE; goto out_unlock; } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && smmu_domain->stall_enabled != master->stall_enabled) { dev_err(dev, "cannot attach to stall-%s domain\n", smmu_domain->stall_enabled ? "enabled" : "disabled"); - ret = -EINVAL; + ret = -EMEDIUMTYPE; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu/qcom_iommu.c b/drivers/iommu/arm/arm-smmu/qcom_iommu.c index 4c077c38fbd6..a8b63b855ffb 100644 --- a/drivers/iommu/arm/arm-smmu/qcom_iommu.c +++ b/drivers/iommu/arm/arm-smmu/qcom_iommu.c @@ -386,7 +386,7 @@ static int qcom_iommu_attach_dev(struct iommu_domain *domain, struct device *dev "attached to domain on IOMMU %s\n", dev_name(qcom_domain->iommu->dev), dev_name(qcom_iommu->dev)); - return -EINVAL; + return -EMEDIUMTYPE; } return 0; diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 44016594831d..0813b119d680 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4323,7 +4323,7 @@ static int prepare_domain_attach_device(struct iommu_domain *domain, return -ENODEV; if (dmar_domain->force_snooping && !ecap_sc_support(iommu->ecap)) - return -EOPNOTSUPP; + return -EMEDIUMTYPE; /* check if this iommu agaw is sufficient for max mapped address */ addr_width = agaw_to_width(iommu->agaw); @@ -4334,7 +4334,7 @@ static int prepare_domain_attach_device(struct iommu_domain *domain, dev_err(dev, "%s: iommu width (%d) is not " "sufficient for the mapped address (%llx)\n", __func__, addr_width, dmar_domain->max_addr); - return -EFAULT; + return -EMEDIUMTYPE; } dmar_domain->gaw = addr_width; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 847ad47a2dfd..19cf28d40ebe 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1972,6 +1972,17 @@ static int __iommu_attach_device(struct iommu_domain *domain, return ret; } +/** + * iommu_attach_device - Attach a device to an IOMMU domain + * @domain: IOMMU domain to attach + * @dev: Device that will be attached + * + * Returns 0 on success and error code on failure + * + * Specifically, -EMEDIUMTYPE is returned if the domain and the device are + * incompatible in some way. This indicates that a caller should try another + * existing IOMMU domain or allocate a new one. + */ int iommu_attach_device(struct iommu_domain *domain, struct device *dev) { struct iommu_group *group; @@ -2098,6 +2109,17 @@ static int __iommu_attach_group(struct iommu_domain *domain, return ret; } +/** + * iommu_attach_group - Attach an IOMMU group to an IOMMU domain + * @domain: IOMMU domain to attach + * @dev: IOMMU group that will be attached + * + * Returns 0 on success and error code on failure + * + * Specifically, -EMEDIUMTYPE is returned if the domain and the group are + * incompatible in some way. This indicates that a caller should try another + * existing IOMMU domain or allocate a new one. + */ int iommu_attach_group(struct iommu_domain *domain, struct iommu_group *group) { int ret; diff --git a/drivers/iommu/ipmmu-vmsa.c b/drivers/iommu/ipmmu-vmsa.c index 8fdb84b3642b..e491e410add5 100644 --- a/drivers/iommu/ipmmu-vmsa.c +++ b/drivers/iommu/ipmmu-vmsa.c @@ -630,7 +630,7 @@ static int ipmmu_attach_device(struct iommu_domain *io_domain, */ dev_err(dev, "Can't attach IPMMU %s to domain on IPMMU %s\n", dev_name(mmu->dev), dev_name(domain->mmu->dev)); - ret = -EINVAL; + ret = -EMEDIUMTYPE; } else dev_info(dev, "Reusing IPMMU context %u\n", domain->context_id); diff --git a/drivers/iommu/omap-iommu.c b/drivers/iommu/omap-iommu.c index d9cf2820c02e..bbc6c4cd7aae 100644 --- a/drivers/iommu/omap-iommu.c +++ b/drivers/iommu/omap-iommu.c @@ -1472,7 +1472,7 @@ omap_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) /* only a single client device can be attached to a domain */ if (omap_domain->dev) { dev_err(dev, "iommu domain is already attached\n"); - ret = -EBUSY; + ret = -EMEDIUMTYPE; goto out; } diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c index 25be4b822aa0..e3b812d8fa96 100644 --- a/drivers/iommu/virtio-iommu.c +++ b/drivers/iommu/virtio-iommu.c @@ -734,7 +734,7 @@ static int viommu_attach_dev(struct iommu_domain *domain, struct device *dev) ret = viommu_domain_finalise(vdev, domain); } else if (vdomain->viommu != vdev->viommu) { dev_err(dev, "cannot attach to foreign vIOMMU\n"); - ret = -EXDEV; + ret = -EMEDIUMTYPE; } mutex_unlock(&vdomain->mutex);
Cases like VFIO wish to attach a device to an existing domain that was not allocated specifically from the device. This raises a condition where the IOMMU driver can fail the domain attach because the domain and device are incompatible with each other. This is a soft failure that can be resolved by using a different domain. Provide a dedicated errno from the IOMMU driver during attach that the reason attached failed is because of domain incompatability. EMEDIUMTYPE is chosen because it is never used within the iommu subsystem today and evokes a sense that the 'medium' aka the domain is incompatible. VFIO can use this to know attach is a soft failure and it should continue searching. Otherwise the attach will be a hard failure and VFIO will return the code to userspace. Update all drivers to return EMEDIUMTYPE in their failure paths that are related to domain incompatability. Add kdocs describing this behavior. Suggested-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com> --- drivers/iommu/amd/iommu.c | 2 +- drivers/iommu/apple-dart.c | 4 ++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 6 +++--- drivers/iommu/arm/arm-smmu/qcom_iommu.c | 2 +- drivers/iommu/intel/iommu.c | 4 ++-- drivers/iommu/iommu.c | 22 +++++++++++++++++++++ drivers/iommu/ipmmu-vmsa.c | 2 +- drivers/iommu/omap-iommu.c | 2 +- drivers/iommu/virtio-iommu.c | 2 +- 9 files changed, 34 insertions(+), 12 deletions(-)