mbox series

[v7,0/9] ACPI/IORT: Support for IORT RMR node

Message ID 20210805080724.480-1-shameerali.kolothum.thodi@huawei.com
Headers show
Series ACPI/IORT: Support for IORT RMR node | expand

Message

Shameerali Kolothum Thodi Aug. 5, 2021, 8:07 a.m. UTC
Hi,

The series adds support to IORT RMR nodes specified in IORT
Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
memory ranges that are used by endpoints and require a unity
mapping in SMMU.

We have faced issues with 3408iMR RAID controller cards which
fail to boot when SMMU is enabled. This is because these
controllers make use of host memory for various caching related
purposes and when SMMU is enabled the iMR firmware fails to 
access these memory regions as there is no mapping for them.
IORT RMR provides a way for UEFI to describe and report these
memory regions so that the kernel can make a unity mapping for
these in SMMU.

Change History: 

v6 --> v7

The only change from v6 is the fix pointed out by Steve to
the SMMUv2 SMR bypass install in patch #8.

Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
yet as the series still needs more review[1].

Feedback and tests on this series is very much appreciated.

v5 --> v6
- Addressed comments from Robin & Lorenzo.
  : Moved iort_parse_rmr() to acpi_iort_init() from
    iort_init_platform_devices().
  : Removed use of struct iort_rmr_entry during the initial
    parse. Using struct iommu_resv_region instead.
  : Report RMR address alignment and overlap errors, but continue.
  : Reworked arm_smmu_init_bypass_stes() (patch # 6).
- Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).
- Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based
  on Type of RMR region. Suggested by Jon N.

Thanks,
Shameer
[0] https://developer.arm.com/documentation/den0049/latest/
[1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7
------
v4 --> v5
 -Added a fw_data union to struct iommu_resv_region and removed
  struct iommu_rmr (Based on comments from Joerg/Robin).
 -Added iommu_put_rmrs() to release mem.
 -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by
  yet because of the above changes.

v3 -->v4
-Included the SMMUv2 SMR bypass install changes suggested by
 Steve(patch #7)
-As per Robin's comments, RMR reserve implementation is now
 more generic  (patch #8) and dropped v3 patches 8 and 10.
-Rebase to 5.13-rc1

RFC v2 --> v3
 -Dropped RFC tag as the ACPICA header changes are now ready to be
  part of 5.13[0]. But this series still has a dependency on that patch.
 -Added IORT E.b related changes(node flags, _DSM function 5 checks for
  PCIe).
 -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
  discussion here[1].
 -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)
------

Jon Nettleton (1):
  iommu/arm-smmu: Get associated RMR info and install bypass SMR

Shameer Kolothum (8):
  iommu: Introduce a union to struct iommu_resv_region
  ACPI/IORT: Add support for RMR node parsing
  iommu/dma: Introduce generic helper to retrieve RMR info
  ACPI/IORT: Add a helper to retrieve RMR memory regions
  iommu/arm-smmu-v3: Introduce strtab init helper
  iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force
    bypass
  iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
  iommu/dma: Reserve any RMR regions associated with a dev

 drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++
 drivers/iommu/dma-iommu.c                   |  89 +++++++++-
 include/linux/acpi_iort.h                   |   7 +
 include/linux/dma-iommu.h                   |  13 ++
 include/linux/iommu.h                       |  11 ++
 7 files changed, 393 insertions(+), 23 deletions(-)

-- 
2.17.1

Comments

Ard Biesheuvel Aug. 5, 2021, 1:22 p.m. UTC | #1
On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum
<shameerali.kolothum.thodi@huawei.com> wrote:
>

> Hi,

>

> The series adds support to IORT RMR nodes specified in IORT

> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe

> memory ranges that are used by endpoints and require a unity

> mapping in SMMU.

>

> We have faced issues with 3408iMR RAID controller cards which

> fail to boot when SMMU is enabled. This is because these

> controllers make use of host memory for various caching related

> purposes and when SMMU is enabled the iMR firmware fails to

> access these memory regions as there is no mapping for them.

> IORT RMR provides a way for UEFI to describe and report these

> memory regions so that the kernel can make a unity mapping for

> these in SMMU.

>


Does this mean we are ignoring the RMR memory ranges, and exposing the
entire physical address space to devices using the stream IDs in
question?


> Change History:

>

> v6 --> v7

>

> The only change from v6 is the fix pointed out by Steve to

> the SMMUv2 SMR bypass install in patch #8.

>

> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and

> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags

> yet as the series still needs more review[1].

>

> Feedback and tests on this series is very much appreciated.

>

> v5 --> v6

> - Addressed comments from Robin & Lorenzo.

>   : Moved iort_parse_rmr() to acpi_iort_init() from

>     iort_init_platform_devices().

>   : Removed use of struct iort_rmr_entry during the initial

>     parse. Using struct iommu_resv_region instead.

>   : Report RMR address alignment and overlap errors, but continue.

>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).

> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).

> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based

>   on Type of RMR region. Suggested by Jon N.

>

> Thanks,

> Shameer

> [0] https://developer.arm.com/documentation/den0049/latest/

> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7

> ------

> v4 --> v5

>  -Added a fw_data union to struct iommu_resv_region and removed

>   struct iommu_rmr (Based on comments from Joerg/Robin).

>  -Added iommu_put_rmrs() to release mem.

>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by

>   yet because of the above changes.

>

> v3 -->v4

> -Included the SMMUv2 SMR bypass install changes suggested by

>  Steve(patch #7)

> -As per Robin's comments, RMR reserve implementation is now

>  more generic  (patch #8) and dropped v3 patches 8 and 10.

> -Rebase to 5.13-rc1

>

> RFC v2 --> v3

>  -Dropped RFC tag as the ACPICA header changes are now ready to be

>   part of 5.13[0]. But this series still has a dependency on that patch.

>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for

>   PCIe).

>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and

>   discussion here[1].

>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)

> ------

>

> Jon Nettleton (1):

>   iommu/arm-smmu: Get associated RMR info and install bypass SMR

>

> Shameer Kolothum (8):

>   iommu: Introduce a union to struct iommu_resv_region

>   ACPI/IORT: Add support for RMR node parsing

>   iommu/dma: Introduce generic helper to retrieve RMR info

>   ACPI/IORT: Add a helper to retrieve RMR memory regions

>   iommu/arm-smmu-v3: Introduce strtab init helper

>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force

>     bypass

>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE

>   iommu/dma: Reserve any RMR regions associated with a dev

>

>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-

>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--

>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++

>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-

>  include/linux/acpi_iort.h                   |   7 +

>  include/linux/dma-iommu.h                   |  13 ++

>  include/linux/iommu.h                       |  11 ++

>  7 files changed, 393 insertions(+), 23 deletions(-)

>

> --

> 2.17.1

>

>

> _______________________________________________

> linux-arm-kernel mailing list

> linux-arm-kernel@lists.infradead.org

> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Shameerali Kolothum Thodi Aug. 5, 2021, 1:35 p.m. UTC | #2
> -----Original Message-----

> From: Ard Biesheuvel [mailto:ardb@kernel.org]

> Sent: 05 August 2021 14:23

> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>

> Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List

> <linux-acpi@vger.kernel.org>; Linux IOMMU

> <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;

> Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel

> <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon

> <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun

> (Hanjun Guo) <guohanjun@huawei.com>; Steven Price

> <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon

> Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;

> yangyicong <yangyicong@huawei.com>

> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node

> 

> On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum

> <shameerali.kolothum.thodi@huawei.com> wrote:

> >

> > Hi,

> >

> > The series adds support to IORT RMR nodes specified in IORT

> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe

> > memory ranges that are used by endpoints and require a unity

> > mapping in SMMU.

> >

> > We have faced issues with 3408iMR RAID controller cards which

> > fail to boot when SMMU is enabled. This is because these

> > controllers make use of host memory for various caching related

> > purposes and when SMMU is enabled the iMR firmware fails to

> > access these memory regions as there is no mapping for them.

> > IORT RMR provides a way for UEFI to describe and report these

> > memory regions so that the kernel can make a unity mapping for

> > these in SMMU.

> >

> 

> Does this mean we are ignoring the RMR memory ranges, and exposing the

> entire physical address space to devices using the stream IDs in

> question?


Nope. RMR node is used to describe the memory ranges used by end points
behind SMMU. And this information is used to create 1 : 1 mappings for those
ranges in SMMU. Anything outside those ranges will result in translation
fault(if there are no other dynamic DMA mappings).

Thanks,
Shameer

> 

> > Change History:

> >

> > v6 --> v7

> >

> > The only change from v6 is the fix pointed out by Steve to

> > the SMMUv2 SMR bypass install in patch #8.

> >

> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and

> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags

> > yet as the series still needs more review[1].

> >

> > Feedback and tests on this series is very much appreciated.

> >

> > v5 --> v6

> > - Addressed comments from Robin & Lorenzo.

> >   : Moved iort_parse_rmr() to acpi_iort_init() from

> >     iort_init_platform_devices().

> >   : Removed use of struct iort_rmr_entry during the initial

> >     parse. Using struct iommu_resv_region instead.

> >   : Report RMR address alignment and overlap errors, but continue.

> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).

> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).

> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based

> >   on Type of RMR region. Suggested by Jon N.

> >

> > Thanks,

> > Shameer

> > [0] https://developer.arm.com/documentation/den0049/latest/

> > [1]

> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth

> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7

> > ------

> > v4 --> v5

> >  -Added a fw_data union to struct iommu_resv_region and removed

> >   struct iommu_rmr (Based on comments from Joerg/Robin).

> >  -Added iommu_put_rmrs() to release mem.

> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by

> >   yet because of the above changes.

> >

> > v3 -->v4

> > -Included the SMMUv2 SMR bypass install changes suggested by

> >  Steve(patch #7)

> > -As per Robin's comments, RMR reserve implementation is now

> >  more generic  (patch #8) and dropped v3 patches 8 and 10.

> > -Rebase to 5.13-rc1

> >

> > RFC v2 --> v3

> >  -Dropped RFC tag as the ACPICA header changes are now ready to be

> >   part of 5.13[0]. But this series still has a dependency on that patch.

> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for

> >   PCIe).

> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and

> >   discussion here[1].

> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)

> > ------

> >

> > Jon Nettleton (1):

> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR

> >

> > Shameer Kolothum (8):

> >   iommu: Introduce a union to struct iommu_resv_region

> >   ACPI/IORT: Add support for RMR node parsing

> >   iommu/dma: Introduce generic helper to retrieve RMR info

> >   ACPI/IORT: Add a helper to retrieve RMR memory regions

> >   iommu/arm-smmu-v3: Introduce strtab init helper

> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force

> >     bypass

> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE

> >   iommu/dma: Reserve any RMR regions associated with a dev

> >

> >  drivers/acpi/arm64/iort.c                   | 172

> +++++++++++++++++++-

> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--

> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++

> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-

> >  include/linux/acpi_iort.h                   |   7 +

> >  include/linux/dma-iommu.h                   |  13 ++

> >  include/linux/iommu.h                       |  11 ++

> >  7 files changed, 393 insertions(+), 23 deletions(-)

> >

> > --

> > 2.17.1

> >

> >

> > _______________________________________________

> > linux-arm-kernel mailing list

> > linux-arm-kernel@lists.infradead.org

> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Ard Biesheuvel Aug. 5, 2021, 2:09 p.m. UTC | #3
On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>

>

>

> > -----Original Message-----

> > From: Ard Biesheuvel [mailto:ardb@kernel.org]

> > Sent: 05 August 2021 14:23

> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>

> > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List

> > <linux-acpi@vger.kernel.org>; Linux IOMMU

> > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;

> > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel

> > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon

> > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun

> > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price

> > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon

> > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;

> > yangyicong <yangyicong@huawei.com>

> > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node

> >

> > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum

> > <shameerali.kolothum.thodi@huawei.com> wrote:

> > >

> > > Hi,

> > >

> > > The series adds support to IORT RMR nodes specified in IORT

> > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe

> > > memory ranges that are used by endpoints and require a unity

> > > mapping in SMMU.

> > >

> > > We have faced issues with 3408iMR RAID controller cards which

> > > fail to boot when SMMU is enabled. This is because these

> > > controllers make use of host memory for various caching related

> > > purposes and when SMMU is enabled the iMR firmware fails to

> > > access these memory regions as there is no mapping for them.

> > > IORT RMR provides a way for UEFI to describe and report these

> > > memory regions so that the kernel can make a unity mapping for

> > > these in SMMU.

> > >

> >

> > Does this mean we are ignoring the RMR memory ranges, and exposing the

> > entire physical address space to devices using the stream IDs in

> > question?

>

> Nope. RMR node is used to describe the memory ranges used by end points

> behind SMMU. And this information is used to create 1 : 1 mappings for those

> ranges in SMMU. Anything outside those ranges will result in translation

> fault(if there are no other dynamic DMA mappings).

>


Excellent! It was not obvious to me from looking at the patches, so I
had to ask.

Thanks,
Ard.

>

> >

> > > Change History:

> > >

> > > v6 --> v7

> > >

> > > The only change from v6 is the fix pointed out by Steve to

> > > the SMMUv2 SMR bypass install in patch #8.

> > >

> > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and

> > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags

> > > yet as the series still needs more review[1].

> > >

> > > Feedback and tests on this series is very much appreciated.

> > >

> > > v5 --> v6

> > > - Addressed comments from Robin & Lorenzo.

> > >   : Moved iort_parse_rmr() to acpi_iort_init() from

> > >     iort_init_platform_devices().

> > >   : Removed use of struct iort_rmr_entry during the initial

> > >     parse. Using struct iommu_resv_region instead.

> > >   : Report RMR address alignment and overlap errors, but continue.

> > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).

> > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).

> > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based

> > >   on Type of RMR region. Suggested by Jon N.

> > >

> > > Thanks,

> > > Shameer

> > > [0] https://developer.arm.com/documentation/den0049/latest/

> > > [1]

> > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth

> > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7

> > > ------

> > > v4 --> v5

> > >  -Added a fw_data union to struct iommu_resv_region and removed

> > >   struct iommu_rmr (Based on comments from Joerg/Robin).

> > >  -Added iommu_put_rmrs() to release mem.

> > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by

> > >   yet because of the above changes.

> > >

> > > v3 -->v4

> > > -Included the SMMUv2 SMR bypass install changes suggested by

> > >  Steve(patch #7)

> > > -As per Robin's comments, RMR reserve implementation is now

> > >  more generic  (patch #8) and dropped v3 patches 8 and 10.

> > > -Rebase to 5.13-rc1

> > >

> > > RFC v2 --> v3

> > >  -Dropped RFC tag as the ACPICA header changes are now ready to be

> > >   part of 5.13[0]. But this series still has a dependency on that patch.

> > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for

> > >   PCIe).

> > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and

> > >   discussion here[1].

> > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)

> > > ------

> > >

> > > Jon Nettleton (1):

> > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR

> > >

> > > Shameer Kolothum (8):

> > >   iommu: Introduce a union to struct iommu_resv_region

> > >   ACPI/IORT: Add support for RMR node parsing

> > >   iommu/dma: Introduce generic helper to retrieve RMR info

> > >   ACPI/IORT: Add a helper to retrieve RMR memory regions

> > >   iommu/arm-smmu-v3: Introduce strtab init helper

> > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force

> > >     bypass

> > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE

> > >   iommu/dma: Reserve any RMR regions associated with a dev

> > >

> > >  drivers/acpi/arm64/iort.c                   | 172

> > +++++++++++++++++++-

> > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--

> > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++

> > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-

> > >  include/linux/acpi_iort.h                   |   7 +

> > >  include/linux/dma-iommu.h                   |  13 ++

> > >  include/linux/iommu.h                       |  11 ++

> > >  7 files changed, 393 insertions(+), 23 deletions(-)

> > >

> > > --

> > > 2.17.1

> > >

> > >

> > > _______________________________________________

> > > linux-arm-kernel mailing list

> > > linux-arm-kernel@lists.infradead.org

> > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Jon Nettleton Aug. 31, 2021, 5:06 a.m. UTC | #4
On Thu, Aug 5, 2021 at 4:09 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>

> On Thu, 5 Aug 2021 at 15:35, Shameerali Kolothum Thodi

> <shameerali.kolothum.thodi@huawei.com> wrote:

> >

> >

> >

> > > -----Original Message-----

> > > From: Ard Biesheuvel [mailto:ardb@kernel.org]

> > > Sent: 05 August 2021 14:23

> > > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>

> > > Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>; ACPI Devel Maling List

> > > <linux-acpi@vger.kernel.org>; Linux IOMMU

> > > <iommu@lists.linux-foundation.org>; Linuxarm <linuxarm@huawei.com>;

> > > Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; Joerg Roedel

> > > <joro@8bytes.org>; Robin Murphy <robin.murphy@arm.com>; Will Deacon

> > > <will@kernel.org>; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun

> > > (Hanjun Guo) <guohanjun@huawei.com>; Steven Price

> > > <steven.price@arm.com>; Sami Mujawar <Sami.Mujawar@arm.com>; Jon

> > > Nettleton <jon@solid-run.com>; Eric Auger <eric.auger@redhat.com>;

> > > yangyicong <yangyicong@huawei.com>

> > > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node

> > >

> > > On Thu, 5 Aug 2021 at 10:10, Shameer Kolothum

> > > <shameerali.kolothum.thodi@huawei.com> wrote:

> > > >

> > > > Hi,

> > > >

> > > > The series adds support to IORT RMR nodes specified in IORT

> > > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe

> > > > memory ranges that are used by endpoints and require a unity

> > > > mapping in SMMU.

> > > >

> > > > We have faced issues with 3408iMR RAID controller cards which

> > > > fail to boot when SMMU is enabled. This is because these

> > > > controllers make use of host memory for various caching related

> > > > purposes and when SMMU is enabled the iMR firmware fails to

> > > > access these memory regions as there is no mapping for them.

> > > > IORT RMR provides a way for UEFI to describe and report these

> > > > memory regions so that the kernel can make a unity mapping for

> > > > these in SMMU.

> > > >

> > >

> > > Does this mean we are ignoring the RMR memory ranges, and exposing the

> > > entire physical address space to devices using the stream IDs in

> > > question?

> >

> > Nope. RMR node is used to describe the memory ranges used by end points

> > behind SMMU. And this information is used to create 1 : 1 mappings for those

> > ranges in SMMU. Anything outside those ranges will result in translation

> > fault(if there are no other dynamic DMA mappings).

> >

>

> Excellent! It was not obvious to me from looking at the patches, so I

> had to ask.

>

> Thanks,

> Ard.

>

> >

> > >

> > > > Change History:

> > > >

> > > > v6 --> v7

> > > >

> > > > The only change from v6 is the fix pointed out by Steve to

> > > > the SMMUv2 SMR bypass install in patch #8.

> > > >

> > > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and

> > > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags

> > > > yet as the series still needs more review[1].

> > > >

> > > > Feedback and tests on this series is very much appreciated.

> > > >

> > > > v5 --> v6

> > > > - Addressed comments from Robin & Lorenzo.

> > > >   : Moved iort_parse_rmr() to acpi_iort_init() from

> > > >     iort_init_platform_devices().

> > > >   : Removed use of struct iort_rmr_entry during the initial

> > > >     parse. Using struct iommu_resv_region instead.

> > > >   : Report RMR address alignment and overlap errors, but continue.

> > > >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).

> > > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).

> > > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based

> > > >   on Type of RMR region. Suggested by Jon N.

> > > >

> > > > Thanks,

> > > > Shameer

> > > > [0] https://developer.arm.com/documentation/den0049/latest/

> > > > [1]

> > > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth

> > > um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7

> > > > ------

> > > > v4 --> v5

> > > >  -Added a fw_data union to struct iommu_resv_region and removed

> > > >   struct iommu_rmr (Based on comments from Joerg/Robin).

> > > >  -Added iommu_put_rmrs() to release mem.

> > > >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by

> > > >   yet because of the above changes.

> > > >

> > > > v3 -->v4

> > > > -Included the SMMUv2 SMR bypass install changes suggested by

> > > >  Steve(patch #7)

> > > > -As per Robin's comments, RMR reserve implementation is now

> > > >  more generic  (patch #8) and dropped v3 patches 8 and 10.

> > > > -Rebase to 5.13-rc1

> > > >

> > > > RFC v2 --> v3

> > > >  -Dropped RFC tag as the ACPICA header changes are now ready to be

> > > >   part of 5.13[0]. But this series still has a dependency on that patch.

> > > >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for

> > > >   PCIe).

> > > >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and

> > > >   discussion here[1].

> > > >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)

> > > > ------

> > > >

> > > > Jon Nettleton (1):

> > > >   iommu/arm-smmu: Get associated RMR info and install bypass SMR

> > > >

> > > > Shameer Kolothum (8):

> > > >   iommu: Introduce a union to struct iommu_resv_region

> > > >   ACPI/IORT: Add support for RMR node parsing

> > > >   iommu/dma: Introduce generic helper to retrieve RMR info

> > > >   ACPI/IORT: Add a helper to retrieve RMR memory regions

> > > >   iommu/arm-smmu-v3: Introduce strtab init helper

> > > >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force

> > > >     bypass

> > > >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE

> > > >   iommu/dma: Reserve any RMR regions associated with a dev

> > > >

> > > >  drivers/acpi/arm64/iort.c                   | 172

> > > +++++++++++++++++++-

> > > >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--

> > > >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++

> > > >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-

> > > >  include/linux/acpi_iort.h                   |   7 +

> > > >  include/linux/dma-iommu.h                   |  13 ++

> > > >  include/linux/iommu.h                       |  11 ++

> > > >  7 files changed, 393 insertions(+), 23 deletions(-)

> > > >

> > > > --

> > > > 2.17.1

> > > >

> > > >

> > > > _______________________________________________

> > > > linux-arm-kernel mailing list

> > > > linux-arm-kernel@lists.infradead.org

> > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


Ping,  Can we get some movement on this patchset?  The standard was
was ratified over a year ago, and there is real world hardware that is
using or needs this functionality.

Thanks,
-Jon
Eric Auger Sept. 30, 2021, 9:47 a.m. UTC | #5
Hi Shameer,

On 8/5/21 10:07 AM, Shameer Kolothum wrote:
> Hi,

>

> The series adds support to IORT RMR nodes specified in IORT

> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe

> memory ranges that are used by endpoints and require a unity

> mapping in SMMU.


I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage
use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]
(matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map MSI
physical doorbells.

That way when an assigned device protected by a vSMMUv3 implemented upon
nested stage issues an MSI transaction, let's say using IOVA=0x8000000,
we would get:
                    S1 (guest)           S2 (host)
0x8000000            0x8000000            Physical DB

This method was suggested by Jean-Philippe (added in CC) and it
simplifies the nested stage integration because we don't have to care
about nested stage MSI bindings.

However if I understand correctly we cannot define a range of SIDs using
the same RMR (due to the single mapping bit which must be set, Table 5
flags format). This is a spec restriction and not an issue with your series.

As VFIO devices can be hot-plugged we thus need to create as many RMR
nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have
5 pcie root ports as it is usual in VMs. Then this causes some trouble
at qemu level for instance, wrt migration. See [RFC]
hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.

Do you know if there is a plan to remove the single mapping limitation
in the spec?

Thanks

Eric
>

> We have faced issues with 3408iMR RAID controller cards which

> fail to boot when SMMU is enabled. This is because these

> controllers make use of host memory for various caching related

> purposes and when SMMU is enabled the iMR firmware fails to 

> access these memory regions as there is no mapping for them.

> IORT RMR provides a way for UEFI to describe and report these

> memory regions so that the kernel can make a unity mapping for

> these in SMMU.

>

> Change History: 

>

> v6 --> v7

>

> The only change from v6 is the fix pointed out by Steve to

> the SMMUv2 SMR bypass install in patch #8.

>

> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and

> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags

> yet as the series still needs more review[1].

>

> Feedback and tests on this series is very much appreciated.

>

> v5 --> v6

> - Addressed comments from Robin & Lorenzo.

>   : Moved iort_parse_rmr() to acpi_iort_init() from

>     iort_init_platform_devices().

>   : Removed use of struct iort_rmr_entry during the initial

>     parse. Using struct iommu_resv_region instead.

>   : Report RMR address alignment and overlap errors, but continue.

>   : Reworked arm_smmu_init_bypass_stes() (patch # 6).

> - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).

> - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based

>   on Type of RMR region. Suggested by Jon N.

>

> Thanks,

> Shameer

> [0] https://developer.arm.com/documentation/den0049/latest/

> [1] https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.kolothum.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7

> ------

> v4 --> v5

>  -Added a fw_data union to struct iommu_resv_region and removed

>   struct iommu_rmr (Based on comments from Joerg/Robin).

>  -Added iommu_put_rmrs() to release mem.

>  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by

>   yet because of the above changes.

>

> v3 -->v4

> -Included the SMMUv2 SMR bypass install changes suggested by

>  Steve(patch #7)

> -As per Robin's comments, RMR reserve implementation is now

>  more generic  (patch #8) and dropped v3 patches 8 and 10.

> -Rebase to 5.13-rc1

>

> RFC v2 --> v3

>  -Dropped RFC tag as the ACPICA header changes are now ready to be

>   part of 5.13[0]. But this series still has a dependency on that patch.

>  -Added IORT E.b related changes(node flags, _DSM function 5 checks for

>   PCIe).

>  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and

>   discussion here[1].

>  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)

> ------

>

> Jon Nettleton (1):

>   iommu/arm-smmu: Get associated RMR info and install bypass SMR

>

> Shameer Kolothum (8):

>   iommu: Introduce a union to struct iommu_resv_region

>   ACPI/IORT: Add support for RMR node parsing

>   iommu/dma: Introduce generic helper to retrieve RMR info

>   ACPI/IORT: Add a helper to retrieve RMR memory regions

>   iommu/arm-smmu-v3: Introduce strtab init helper

>   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force

>     bypass

>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE

>   iommu/dma: Reserve any RMR regions associated with a dev

>

>  drivers/acpi/arm64/iort.c                   | 172 +++++++++++++++++++-

>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--

>  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++

>  drivers/iommu/dma-iommu.c                   |  89 +++++++++-

>  include/linux/acpi_iort.h                   |   7 +

>  include/linux/dma-iommu.h                   |  13 ++

>  include/linux/iommu.h                       |  11 ++

>  7 files changed, 393 insertions(+), 23 deletions(-)

>
Shameerali Kolothum Thodi Sept. 30, 2021, 10:50 a.m. UTC | #6
Hi Eric,

> -----Original Message-----

> From: Eric Auger [mailto:eric.auger@redhat.com]

> Sent: 30 September 2021 10:48

> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;

> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;

> iommu@lists.linux-foundation.org; Jean-Philippe Brucker

> <Jean-Philippe.Brucker@arm.com>

> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;

> joro@8bytes.org; robin.murphy@arm.com; will@kernel.org; wanghuiqiang

> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)

> <guohanjun@huawei.com>; steven.price@arm.com; Sami.Mujawar@arm.com;

> jon@solid-run.com; yangyicong <yangyicong@huawei.com>

> Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node

> 

> Hi Shameer,

> 

> On 8/5/21 10:07 AM, Shameer Kolothum wrote:

> > Hi,

> >

> > The series adds support to IORT RMR nodes specified in IORT

> > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe

> > memory ranges that are used by endpoints and require a unity

> > mapping in SMMU.

> 

> I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage

> use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000]

> (matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map

> MSI

> physical doorbells.

> 

> That way when an assigned device protected by a vSMMUv3 implemented

> upon

> nested stage issues an MSI transaction, let's say using IOVA=0x8000000,

> we would get:

>                     S1 (guest)           S2 (host)

> 0x8000000            0x8000000            Physical DB

> 

> This method was suggested by Jean-Philippe (added in CC) and it

> simplifies the nested stage integration because we don't have to care

> about nested stage MSI bindings.

> 

> However if I understand correctly we cannot define a range of SIDs using

> the same RMR (due to the single mapping bit which must be set, Table 5

> flags format). This is a spec restriction and not an issue with your series.


Yes. The spec currently mandates single mapping bit to be set.

> 

> As VFIO devices can be hot-plugged we thus need to create as many RMR

> nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have

> 5 pcie root ports as it is usual in VMs. Then this causes some trouble

> at qemu level for instance, wrt migration. See [RFC]

> hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding.

> 

> Do you know if there is a plan to remove the single mapping limitation

> in the spec?


I would imagine so. In an earlier comment[1], Robin did mention about possible
relaxing of this in future spec revision.

Thanks,
Shameer
1. https://lore.kernel.org/linux-arm-kernel/15c7fac0-11a8-0cdb-aac3-b5d8feb8f066@arm.com/

> Thanks

> 

> Eric

> >

> > We have faced issues with 3408iMR RAID controller cards which

> > fail to boot when SMMU is enabled. This is because these

> > controllers make use of host memory for various caching related

> > purposes and when SMMU is enabled the iMR firmware fails to

> > access these memory regions as there is no mapping for them.

> > IORT RMR provides a way for UEFI to describe and report these

> > memory regions so that the kernel can make a unity mapping for

> > these in SMMU.

> >

> > Change History:

> >

> > v6 --> v7

> >

> > The only change from v6 is the fix pointed out by Steve to

> > the SMMUv2 SMR bypass install in patch #8.

> >

> > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and

> > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags

> > yet as the series still needs more review[1].

> >

> > Feedback and tests on this series is very much appreciated.

> >

> > v5 --> v6

> > - Addressed comments from Robin & Lorenzo.

> >   : Moved iort_parse_rmr() to acpi_iort_init() from

> >     iort_init_platform_devices().

> >   : Removed use of struct iort_rmr_entry during the initial

> >     parse. Using struct iommu_resv_region instead.

> >   : Report RMR address alignment and overlap errors, but continue.

> >   : Reworked arm_smmu_init_bypass_stes() (patch # 6).

> > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8).

> > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based

> >   on Type of RMR region. Suggested by Jon N.

> >

> > Thanks,

> > Shameer

> > [0] https://developer.arm.com/documentation/den0049/latest/

> > [1]

> https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth

> um.thodi@huawei.com/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7

> > ------

> > v4 --> v5

> >  -Added a fw_data union to struct iommu_resv_region and removed

> >   struct iommu_rmr (Based on comments from Joerg/Robin).

> >  -Added iommu_put_rmrs() to release mem.

> >  -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by

> >   yet because of the above changes.

> >

> > v3 -->v4

> > -Included the SMMUv2 SMR bypass install changes suggested by

> >  Steve(patch #7)

> > -As per Robin's comments, RMR reserve implementation is now

> >  more generic  (patch #8) and dropped v3 patches 8 and 10.

> > -Rebase to 5.13-rc1

> >

> > RFC v2 --> v3

> >  -Dropped RFC tag as the ACPICA header changes are now ready to be

> >   part of 5.13[0]. But this series still has a dependency on that patch.

> >  -Added IORT E.b related changes(node flags, _DSM function 5 checks for

> >   PCIe).

> >  -Changed RMR to stream id mapping from M:N to M:1 as per the spec and

> >   discussion here[1].

> >  -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)

> > ------

> >

> > Jon Nettleton (1):

> >   iommu/arm-smmu: Get associated RMR info and install bypass SMR

> >

> > Shameer Kolothum (8):

> >   iommu: Introduce a union to struct iommu_resv_region

> >   ACPI/IORT: Add support for RMR node parsing

> >   iommu/dma: Introduce generic helper to retrieve RMR info

> >   ACPI/IORT: Add a helper to retrieve RMR memory regions

> >   iommu/arm-smmu-v3: Introduce strtab init helper

> >   iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force

> >     bypass

> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE

> >   iommu/dma: Reserve any RMR regions associated with a dev

> >

> >  drivers/acpi/arm64/iort.c                   | 172

> +++++++++++++++++++-

> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  76 +++++++--

> >  drivers/iommu/arm/arm-smmu/arm-smmu.c       |  48 ++++++

> >  drivers/iommu/dma-iommu.c                   |  89 +++++++++-

> >  include/linux/acpi_iort.h                   |   7 +

> >  include/linux/dma-iommu.h                   |  13 ++

> >  include/linux/iommu.h                       |  11 ++

> >  7 files changed, 393 insertions(+), 23 deletions(-)

> >
Shameerali Kolothum Thodi Jan. 25, 2022, 1 p.m. UTC | #7
Hi Robin/Lorenzo,

> -----Original Message-----
> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
> Of Shameer Kolothum
> Sent: 05 August 2021 09:07
> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org
> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
> <wanghuiqiang@huawei.com>
> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> 
> Hi,
> 
> The series adds support to IORT RMR nodes specified in IORT
> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
> memory ranges that are used by endpoints and require a unity
> mapping in SMMU.
> 
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these
> controllers make use of host memory for various caching related
> purposes and when SMMU is enabled the iMR firmware fails to
> access these memory regions as there is no mapping for them.
> IORT RMR provides a way for UEFI to describe and report these
> memory regions so that the kernel can make a unity mapping for
> these in SMMU.
> 
> Change History:
> 
> v6 --> v7
> 
> The only change from v6 is the fix pointed out by Steve to
> the SMMUv2 SMR bypass install in patch #8.
> 
> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
> yet as the series still needs more review[1].
> 
> Feedback and tests on this series is very much appreciated.

Since we have an update to IORT spec(E.c) now[1] and includes additional
attributes/flags for the RMR node, I am planning to respin this series soon.

Going through the new spec, I have a few queries,

The memory range attributes can now be described as one of the following,

0x00: Device-nGnRnE memory
0x01: Device-nGnRE memory
0x02: Device-nGRE memory
0x03: Device-GRE memory
0x04: Normal Inner Non-cacheable Outer Non-cacheable
0x05: Normal Inner Write-back Outer Write-back Inner Shareable

I am not sure how this needs to be captured and used in the kernel. Is there
any intention of using these fine-grained attributes in the kernel now
or a generic mapping of the above to the struct iommu_rev_region prot field
is enough? i.e., something like,

{
    ....
    prot = IOMMU_READ | IOMMU_WRITE;

    if (rmr_attr == normal_mem) // 0x05
        prot |= IOMMU_CACHE;
    
    if (rmr_attr == device_mem) { //0x00 - 0x03
        prot |= IOMMU_MMIO;
        prot |= IOMMU_NOEXEC;
    }
    ....
}

Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
IOMMU_PRIV ?
  
Please let me know.

Thanks,
Shameer

[1] https://developer.arm.com/documentation/den0049/ec/?lang=en
Robin Murphy Jan. 25, 2022, 7:30 p.m. UTC | #8
On 2022-01-25 13:00, Shameerali Kolothum Thodi wrote:
> Hi Robin/Lorenzo,
> 
>> -----Original Message-----
>> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf
>> Of Shameer Kolothum
>> Sent: 05 August 2021 09:07
>> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
>> iommu@lists.linux-foundation.org
>> Cc: robin.murphy@arm.com; jon@solid-run.com; Linuxarm
>> <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; yangyicong <yangyicong@huawei.com>;
>> Sami.Mujawar@arm.com; will@kernel.org; wanghuiqiang
>> <wanghuiqiang@huawei.com>
>> Subject: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
>>
>> Hi,
>>
>> The series adds support to IORT RMR nodes specified in IORT
>> Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe
>> memory ranges that are used by endpoints and require a unity
>> mapping in SMMU.
>>
>> We have faced issues with 3408iMR RAID controller cards which
>> fail to boot when SMMU is enabled. This is because these
>> controllers make use of host memory for various caching related
>> purposes and when SMMU is enabled the iMR firmware fails to
>> access these memory regions as there is no mapping for them.
>> IORT RMR provides a way for UEFI to describe and report these
>> memory regions so that the kernel can make a unity mapping for
>> these in SMMU.
>>
>> Change History:
>>
>> v6 --> v7
>>
>> The only change from v6 is the fix pointed out by Steve to
>> the SMMUv2 SMR bypass install in patch #8.
>>
>> Thanks to the Tested-by tags by Laurentiu with SMMUv2 and
>> Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags
>> yet as the series still needs more review[1].
>>
>> Feedback and tests on this series is very much appreciated.
> 
> Since we have an update to IORT spec(E.c) now[1] and includes additional
> attributes/flags for the RMR node, I am planning to respin this series soon.
> 
> Going through the new spec, I have a few queries,
> 
> The memory range attributes can now be described as one of the following,
> 
> 0x00: Device-nGnRnE memory
> 0x01: Device-nGnRE memory
> 0x02: Device-nGRE memory
> 0x03: Device-GRE memory
> 0x04: Normal Inner Non-cacheable Outer Non-cacheable
> 0x05: Normal Inner Write-back Outer Write-back Inner Shareable
> 
> I am not sure how this needs to be captured and used in the kernel. Is there
> any intention of using these fine-grained attributes in the kernel now
> or a generic mapping of the above to the struct iommu_rev_region prot field
> is enough? i.e., something like,
> 
> {
>      ....
>      prot = IOMMU_READ | IOMMU_WRITE;
> 
>      if (rmr_attr == normal_mem) // 0x05
>          prot |= IOMMU_CACHE;
>      
>      if (rmr_attr == device_mem) { //0x00 - 0x03
>          prot |= IOMMU_MMIO;
>          prot |= IOMMU_NOEXEC;
>      }
>      ....
> }

Yup, pretty much that, except don't bother with IOMMU_NOEXEC. We can't 
reliably infer it - e.g. on an AXI-based interconnect AxCACHE and AxPROT 
are entirely orthogonal, so a Device-type read with the "Instruction 
access" hint is perfectly legal - and in the common IORT code we're not 
in a position to second-guess what any given RMR might represent for 
whatever agent is accessing it.

All we can reasonably do here is map the Device types to IOMMU_MMIO and 
Write-back to IOMMU_CACHE, and if anyone ever does want to insist that 
that's not sufficient, then they're welcome to send patches to make the 
IOMMU API more expressive :)

> Similarly for the 'flags' field, the new 'Access Privilege' is intended to set the
> IOMMU_PRIV ?

Yes, exactly!

Cheers,
Robin.

>    
> Please let me know.
> 
> Thanks,
> Shameer
> 
> [1] https://developer.arm.com/documentation/den0049/ec/?lang=en
>