[v2,0/6] iommufd support allocating nested parent domain

IOMMU hardwares that support nested translation would have two stages
address translation (normally mentioned as stage-1 and stage-2). The page
table formats of the stage-1 and stage-2 can be different. e.g., VT-d has
different page table formats for stage-1 and stage-2.

Nested parent domain is the iommu domain used to represent the stage-2
translation. In IOMMUFD, both stage-1 and stage-2 translation are tracked
as HWPT (a.k.a. iommu domain). Stage-2 HWPT is parent of stage-1 HWPT as
stage-1 cannot work alone in nested translation. In the cases of stage-1 and
stage-2 page table format are different, the parent HWPT should use exactly
the stage-2 page table format. However, the existing kernel hides the format
selection in iommu drivers, so the domain allocated via IOMMU_HWPT_ALLOC can
use either stage-1 page table format or stage-2 page table format, there is
no guarantees for it.

To enforce the page table format of the nested parent domain, this series
introduces a new iommu op (domain_alloc_user) which can accept user flags
to allocate domain as userspace requires. It also converts IOMMUFD to use
the new domain_alloc_user op for domain allocation if supported, then extends
the IOMMU_HWPT_ALLOC ioctl to pass down a NEST_PARENT flag to allocate a HWPT
which can be used as parent. This series implements the new op in Intel iommu
driver to have a complete picture. It is a preparation for adding nesting
support in IOMMUFD/IOMMU.

Complete code can be found:
https://github.com/yiliu1765/iommufd/tree/iommufd_alloc_user_v2

Change log:

v2:
 - Require domain_alloc_user op if IOMMU_HWPT_ALLOC passes non-zero flags (Kevin)
 - IOMMUFD core should check kernel known flags while iommu driver needs
   to check supported flags as well (Jason)
 - Minor tweaks per Baolu's comment

v1: https://lore.kernel.org/linux-iommu/20230919092523.39286-1-yi.l.liu@intel.com/

Regards,
	Yi Liu

Yi Liu (6):
  iommu: Add new iommu op to create domains owned by userspace
  iommufd/hw_pagetable: Use domain_alloc_user op for domain allocation
  iommufd/hw_pagetable: Accepts user flags for domain allocation
  iommufd/hw_pagetable: Support allocating nested parent domain
  iommufd/selftest: Add domain_alloc_user() support in iommu mock
  iommu/vt-d: Add domain_alloc_user op

 drivers/iommu/intel/iommu.c                   | 28 +++++++++++++++++
 drivers/iommu/iommufd/device.c                |  2 +-
 drivers/iommu/iommufd/hw_pagetable.c          | 31 ++++++++++++++-----
 drivers/iommu/iommufd/iommufd_private.h       |  3 +-
 drivers/iommu/iommufd/selftest.c              | 19 ++++++++++++
 include/linux/iommu.h                         | 11 ++++++-
 include/uapi/linux/iommufd.h                  | 12 ++++++-
 tools/testing/selftests/iommu/iommufd.c       | 24 +++++++++++---
 .../selftests/iommu/iommufd_fail_nth.c        |  2 +-
 tools/testing/selftests/iommu/iommufd_utils.h | 11 +++++--
 10 files changed, 124 insertions(+), 19 deletions(-)

On 9/28/23 3:15 PM, Yi Liu wrote:
> This extends iommufd_hw_pagetable_alloc() to accepts user flags.
> 
> Reviewed-by: Kevin Tian<kevin.tian@intel.com>
> Signed-off-by: Yi Liu<yi.l.liu@intel.com>
> ---
>   drivers/iommu/iommufd/device.c          | 2 +-
>   drivers/iommu/iommufd/hw_pagetable.c    | 9 ++++++---
>   drivers/iommu/iommufd/iommufd_private.h | 3 ++-
>   3 files changed, 9 insertions(+), 5 deletions(-)

Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>

Best regards,
baolu

> From: Liu, Yi L <yi.l.liu@intel.com>
> Sent: Thursday, September 28, 2023 3:15 PM
> 
> This extends IOMMU_HWPT_ALLOC to allocate domains used as parent
> (stage-2)
> in nested translation.
> 
> Signed-off-by: Yi Liu <yi.l.liu@intel.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

On Thu, Sep 28, 2023 at 12:15:22AM -0700, Yi Liu wrote:

> Yi Liu (6):
>   iommu: Add new iommu op to create domains owned by userspace
>   iommufd/hw_pagetable: Use domain_alloc_user op for domain allocation
>   iommufd/hw_pagetable: Accepts user flags for domain allocation
>   iommufd/hw_pagetable: Support allocating nested parent domain
>   iommufd/selftest: Add domain_alloc_user() support in iommu mock
>   iommu/vt-d: Add domain_alloc_user op

I copy edited the commit messages, and moved the hunk adding
IOMMU_HWPT_ALLOC_NEST_PARENT from 'iommu: Add new iommu op to create
domains owned by userspace' into 'iommufd: Support allocating nested
parent domain'

Otherwise applied to iommufd for-next

Thanks,
Jason

On Mon, Oct 16, 2023 at 04:16:05PM +0800, Liu, Jingqi wrote:
 
> @@ -88,10 +90,19 @@ iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas,
>        refcount_inc(&ioas->obj.users);
>        hwpt->ioas = ioas;
> 
> -       hwpt->domain = iommu_domain_alloc(idev->dev->bus);
> -       if (!hwpt->domain) {
> -               rc = -ENOMEM;
> -               goto out_abort;
> +       if (ops->domain_alloc_user) {
> +               hwpt->domain = ops->domain_alloc_user(idev->dev, 0);
> 
> Seems a "flags" parameter needs to be passed to 'domain_alloc_user()'.
> Like this:
>         hwpt->domain = ops->domain_alloc_user(idev->dev, flags);

There's no "flags" parameter until the following PATCH-3:
https://lore.kernel.org/linux-iommu/20230928071528.26258-4-yi.l.liu@intel.com/

Thanks
Nicolin