| Message ID | 20231114135654.30475-1-quic_bibekkum@quicinc.com |
|---|---|
| Headers | show |
| Series | iommu/arm-smmu: introduction of ACTLR implementation for Qualcomm SoCs | expand |
On 11/14/2023 7:38 PM, Dmitry Baryshkov wrote: > On Tue, 14 Nov 2023 at 15:57, Bibek Kumar Patro > <quic_bibekkum@quicinc.com> wrote: >> >> Currently in Qualcomm SoCs the default prefetch is set to 1 which allows >> the TLB to fetch just the next page table. MMU-500 features ACTLR >> register which is implementation defined and is used for Qualcomm SoCs >> to have a prefetch setting of 1/3/7/15 enabling TLB to prefetch >> the next set of page tables accordingly allowing for faster translations. >> >> ACTLR value is unique for each SMR (Stream matching register) and stored >> in a pre-populated table. This value is set to the register during >> context bank initialisation. >> >> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> >> --- >> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 41 ++++++++++++++++++++++ >> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h | 2 ++ >> drivers/iommu/arm/arm-smmu/arm-smmu.c | 5 +-- >> drivers/iommu/arm/arm-smmu/arm-smmu.h | 5 +++ >> 4 files changed, 51 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >> index 549ae4dba3a6..578c662c7c30 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >> @@ -14,6 +14,17 @@ >> >> #define QCOM_DUMMY_VAL -1 >> >> +struct actlr_config { >> + const struct actlr_data *adata; >> + size_t size; > > Merge this into struct qcom_smmu_match_data. > Just saw your response on the other thread in v1 patch, let me try again once to accomodate into single structure as suggested >> +}; >> + >> +struct actlr_data { >> + u16 sid; >> + u16 mask; >> + u32 actlr; >> +}; >> + >> static struct qcom_smmu *to_qcom_smmu(struct arm_smmu_device *smmu) >> { >> return container_of(smmu, struct qcom_smmu, smmu); >> @@ -261,9 +272,36 @@ static const struct of_device_id qcom_smmu_client_of_match[] __maybe_unused = { >> { } >> }; >> >> +static void arm_smmu_set_actlr(struct arm_smmu_device *smmu, int idx, >> + const struct actlr_config *actlrcfg) >> +{ >> + struct arm_smmu_smr *smr = smmu->smrs; >> + int i; >> + u16 id; >> + u16 mask; >> + >> + for (i = 0; i < actlrcfg->size; ++i) { >> + id = actlrcfg->adata[i].sid; >> + mask = actlrcfg->adata[i].mask; >> + if (!smr_is_subset(*smr, id, mask)) >> + arm_smmu_cb_write(smmu, idx, ARM_SMMU_CB_ACTLR, >> + actlrcfg->adata[i].actlr); >> + } >> +} >> + >> static int qcom_smmu_init_context(struct arm_smmu_domain *smmu_domain, >> struct io_pgtable_cfg *pgtbl_cfg, struct device *dev) >> { >> + struct arm_smmu_device *smmu = smmu_domain->smmu; >> + struct qcom_smmu *qsmmu = to_qcom_smmu(smmu); >> + const struct actlr_config *actlrcfg; >> + int idx = smmu_domain->cfg.cbndx; >> + >> + if (qsmmu->actlrcfg) { >> + actlrcfg = qsmmu->actlrcfg; >> + arm_smmu_set_actlr(smmu, idx, actlrcfg); >> + } >> + >> smmu_domain->cfg.flush_walk_prefer_tlbiasid = true; >> >> return 0; >> @@ -467,6 +505,9 @@ static struct arm_smmu_device *qcom_smmu_create(struct arm_smmu_device *smmu, >> qsmmu->smmu.impl = impl; >> qsmmu->cfg = data->cfg; >> >> + if (data->actlrcfg && (data->actlrcfg->size)) >> + qsmmu->actlrcfg = data->actlrcfg; >> + >> return &qsmmu->smmu; >> } >> >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> index 593910567b88..4b6862715070 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> @@ -9,6 +9,7 @@ >> struct qcom_smmu { >> struct arm_smmu_device smmu; >> const struct qcom_smmu_config *cfg; >> + const struct actlr_config *actlrcfg; >> bool bypass_quirk; >> u8 bypass_cbndx; >> u32 stall_enabled; >> @@ -25,6 +26,7 @@ struct qcom_smmu_config { >> }; >> >> struct qcom_smmu_match_data { >> + const struct actlr_config *actlrcfg; >> const struct qcom_smmu_config *cfg; >> const struct arm_smmu_impl *impl; >> const struct arm_smmu_impl *adreno_impl; >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> index d6d1a2a55cc0..8e4faf015286 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> @@ -990,9 +990,10 @@ static int arm_smmu_find_sme(struct arm_smmu_device *smmu, u16 id, u16 mask) >> * expect simply identical entries for this case, but there's >> * no harm in accommodating the generalisation. >> */ >> - if ((mask & smrs[i].mask) == mask && >> - !((id ^ smrs[i].id) & ~smrs[i].mask)) >> + >> + if (smr_is_subset(smrs[i], id, mask)) >> return i; >> + >> /* >> * If the new entry has any other overlap with an existing one, >> * though, then there always exists at least one stream ID >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h b/drivers/iommu/arm/arm-smmu/arm-smmu.h >> index 703fd5817ec1..b1638bbc41d4 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h >> @@ -501,6 +501,11 @@ static inline void arm_smmu_writeq(struct arm_smmu_device *smmu, int page, >> writeq_relaxed(val, arm_smmu_page(smmu, page) + offset); >> } >> >> +static inline bool smr_is_subset(struct arm_smmu_smr smrs, u16 id, u16 mask) >> +{ >> + return (mask & smrs.mask) == mask && !((id ^ smrs.id) & ~smrs.mask); >> +} >> + >> #define ARM_SMMU_GR0 0 >> #define ARM_SMMU_GR1 1 >> #define ARM_SMMU_CB(s, n) ((s)->numpage + (n)) >> -- >> 2.17.1 >> > > > -- > With best wishes > Dmitry regards, Bibek
On Wed, 15 Nov 2023 at 11:45, Bibek Kumar Patro <quic_bibekkum@quicinc.com> wrote: > > On 11/14/2023 7:45 PM, Dmitry Baryshkov wrote: > > On Tue, 14 Nov 2023 at 15:57, Bibek Kumar Patro > > <quic_bibekkum@quicinc.com> wrote: > >> > >> Context caching is re-enabled in the prefetch buffer for Qualcomm SoCs > >> through SoC specific reset ops, which is disabled in the default MMU-500 > >> reset ops, but is expected for context banks using ACTLR register to > >> retain the prefetch value during reset and runtime suspend. > > > > Please refer to Documentation/process/submitting-patches.rst and > > rephrase this following the rules there. > > > > Noted, will go through the description once and rephrase it > in next version complying with rules. > > >> > >> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> > >> --- > >> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 25 ++++++++++++++++++---- > >> 1 file changed, 21 insertions(+), 4 deletions(-) > >> > >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c > >> index 0eaf6f2a2e49..fa867b1d9d16 100644 > >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c > >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c > >> @@ -478,11 +478,28 @@ static int qcom_smmu_def_domain_type(struct device *dev) > >> return match ? IOMMU_DOMAIN_IDENTITY : 0; > >> } > >> > >> +static int qcom_smmu500_reset(struct arm_smmu_device *smmu) > >> +{ > >> + int i; > >> + u32 reg; > >> + > >> + arm_mmu500_reset(smmu); > >> + > >> + /* Re-enable context caching after reset */ > >> + for (i = 0; i < smmu->num_context_banks; ++i) { > >> + reg = arm_smmu_cb_read(smmu, i, ARM_SMMU_CB_ACTLR); > >> + reg |= CPRE; > >> + arm_smmu_cb_write(smmu, i, ARM_SMMU_CB_ACTLR, reg); > >> + } > >> + > >> + return 0; > >> +} > >> + > >> static int qcom_sdm845_smmu500_reset(struct arm_smmu_device *smmu) > >> { > >> int ret; > >> > >> - arm_mmu500_reset(smmu); > >> + qcom_smmu500_reset(smmu); > > > > Is this applicable for sdm845? For all other platforms supported by > > qcom_smmu_500 implementation? > > > > In arm_mmu500_reset operation drivers/iommu/arm/arm-smmu/arm-smmu-impl.c > CPRE bit is reset for all SoC based on mmu500 platform, hence for all > Qualcomm SoCs including sm845 we are setting back the CPRE bit. The errata for the CoreLink MMU-500 requires CPRE to be disabled for all revisions before r2p2. Do we know whether these SoC used CoreLink MMU-500 and which version of it? > > >> > >> /* > >> * To address performance degradation in non-real time clients, > >> @@ -509,7 +526,7 @@ static const struct arm_smmu_impl qcom_smmu_500_impl = { > >> .init_context = qcom_smmu_init_context, > >> .cfg_probe = qcom_smmu_cfg_probe, > >> .def_domain_type = qcom_smmu_def_domain_type, > >> - .reset = arm_mmu500_reset, > >> + .reset = qcom_smmu500_reset, > >> .write_s2cr = qcom_smmu_write_s2cr, > >> .tlb_sync = qcom_smmu_tlb_sync, > >> }; > >> @@ -528,7 +545,7 @@ static const struct arm_smmu_impl sm8550_smmu_500_impl = { > >> .init_context = qcom_smmu_init_context, > >> .cfg_probe = qcom_smmu_cfg_probe, > >> .def_domain_type = qcom_smmu_def_domain_type, > >> - .reset = arm_mmu500_reset, > >> + .reset = qcom_smmu500_reset, > >> .write_s2cr = qcom_smmu_write_s2cr, > >> .tlb_sync = qcom_smmu_tlb_sync, > >> }; > >> @@ -544,7 +561,7 @@ static const struct arm_smmu_impl qcom_adreno_smmu_v2_impl = { > >> static const struct arm_smmu_impl qcom_adreno_smmu_500_impl = { > >> .init_context = qcom_adreno_smmu_init_context, > >> .def_domain_type = qcom_smmu_def_domain_type, > >> - .reset = arm_mmu500_reset, > >> + .reset = qcom_smmu500_reset, > >> .alloc_context_bank = qcom_adreno_smmu_alloc_context_bank, > >> .write_sctlr = qcom_adreno_smmu_write_sctlr, > >> .tlb_sync = qcom_smmu_tlb_sync, > >> -- > >> 2.17.1 > >> > > > > -- With best wishes Dmitry
On 11/14/2023 10:25 PM, Robin Murphy wrote: > On 14/11/2023 1:56 pm, Bibek Kumar Patro wrote: >> Currently in Qualcomm SoCs the default prefetch is set to 1 which allows >> the TLB to fetch just the next page table. MMU-500 features ACTLR >> register which is implementation defined and is used for Qualcomm SoCs >> to have a prefetch setting of 1/3/7/15 enabling TLB to prefetch >> the next set of page tables accordingly allowing for faster translations. >> >> ACTLR value is unique for each SMR (Stream matching register) and stored >> in a pre-populated table. This value is set to the register during >> context bank initialisation. >> >> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> >> --- >> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 41 ++++++++++++++++++++++ >> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h | 2 ++ >> drivers/iommu/arm/arm-smmu/arm-smmu.c | 5 +-- >> drivers/iommu/arm/arm-smmu/arm-smmu.h | 5 +++ >> 4 files changed, 51 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >> b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >> index 549ae4dba3a6..578c662c7c30 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >> @@ -14,6 +14,17 @@ >> >> #define QCOM_DUMMY_VAL -1 >> >> +struct actlr_config { >> + const struct actlr_data *adata; >> + size_t size; >> +}; >> + >> +struct actlr_data { >> + u16 sid; >> + u16 mask; > > Do we need to worry about masks? If you're already assuming that any SMR > will be programmed to match a superset of the data here, surely a single > unique ID per device would suffice? > If you refer to the arm_smmu_set_actlr below, mask would be needed as we would check for mask along with sid as well while assigning actlr configuration. Also with mask no of actlr entries can be reduces as with out mask we have to have entry for each sid. >> + u32 actlr; >> +}; >> + >> static struct qcom_smmu *to_qcom_smmu(struct arm_smmu_device *smmu) >> { >> return container_of(smmu, struct qcom_smmu, smmu); >> @@ -261,9 +272,36 @@ static const struct of_device_id >> qcom_smmu_client_of_match[] __maybe_unused = { >> { } >> }; >> >> +static void arm_smmu_set_actlr(struct arm_smmu_device *smmu, int idx, >> + const struct actlr_config *actlrcfg) >> +{ >> + struct arm_smmu_smr *smr = smmu->smrs; >> + int i; >> + u16 id; >> + u16 mask; >> + >> + for (i = 0; i < actlrcfg->size; ++i) { >> + id = actlrcfg->adata[i].sid; >> + mask = actlrcfg->adata[i].mask; >> + if (!smr_is_subset(*smr, id, mask)) > > How well have you tested this? ;) > Well, this logic has worked pretty good for us till now in our downstream implementation. :) (During testing as well this logic helped to better match the SMRs instead of manually mathcing the mask and SID which missed some SIDs) Also this is already being used to arm_smmu_find_sme hence packaged this logic in a wrapper to be used in other places as well(including ACTLR register setting case here.) >> + arm_smmu_cb_write(smmu, idx, ARM_SMMU_CB_ACTLR, >> + actlrcfg->adata[i].actlr); >> + } >> +} >> + >> static int qcom_smmu_init_context(struct arm_smmu_domain *smmu_domain, >> struct io_pgtable_cfg *pgtbl_cfg, struct device *dev) >> { >> + struct arm_smmu_device *smmu = smmu_domain->smmu; >> + struct qcom_smmu *qsmmu = to_qcom_smmu(smmu); >> + const struct actlr_config *actlrcfg; >> + int idx = smmu_domain->cfg.cbndx; >> + >> + if (qsmmu->actlrcfg) { >> + actlrcfg = qsmmu->actlrcfg; >> + arm_smmu_set_actlr(smmu, idx, actlrcfg); >> + } >> + >> smmu_domain->cfg.flush_walk_prefer_tlbiasid = true; >> >> return 0; >> @@ -467,6 +505,9 @@ static struct arm_smmu_device >> *qcom_smmu_create(struct arm_smmu_device *smmu, >> qsmmu->smmu.impl = impl; >> qsmmu->cfg = data->cfg; >> >> + if (data->actlrcfg && (data->actlrcfg->size)) >> + qsmmu->actlrcfg = data->actlrcfg; > > Do we really need to replicate multiple parts of the data, or would it > be sensible to just replace qsmmu->cfg with qsmmu->data and handle the > further dereferences in the places that want them? > >> + >> return &qsmmu->smmu; >> } >> >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> index 593910567b88..4b6862715070 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> @@ -9,6 +9,7 @@ >> struct qcom_smmu { >> struct arm_smmu_device smmu; >> const struct qcom_smmu_config *cfg; >> + const struct actlr_config *actlrcfg; >> bool bypass_quirk; >> u8 bypass_cbndx; >> u32 stall_enabled; >> @@ -25,6 +26,7 @@ struct qcom_smmu_config { >> }; >> >> struct qcom_smmu_match_data { >> + const struct actlr_config *actlrcfg; >> const struct qcom_smmu_config *cfg; >> const struct arm_smmu_impl *impl; >> const struct arm_smmu_impl *adreno_impl; >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c >> b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> index d6d1a2a55cc0..8e4faf015286 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> @@ -990,9 +990,10 @@ static int arm_smmu_find_sme(struct >> arm_smmu_device *smmu, u16 id, u16 mask) >> * expect simply identical entries for this case, but there's >> * no harm in accommodating the generalisation. >> */ >> - if ((mask & smrs[i].mask) == mask && >> - !((id ^ smrs[i].id) & ~smrs[i].mask)) >> + >> + if (smr_is_subset(smrs[i], id, mask)) >> return i; >> + >> /* >> * If the new entry has any other overlap with an existing one, >> * though, then there always exists at least one stream ID >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h >> b/drivers/iommu/arm/arm-smmu/arm-smmu.h >> index 703fd5817ec1..b1638bbc41d4 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h >> @@ -501,6 +501,11 @@ static inline void arm_smmu_writeq(struct >> arm_smmu_device *smmu, int page, >> writeq_relaxed(val, arm_smmu_page(smmu, page) + offset); >> } >> >> +static inline bool smr_is_subset(struct arm_smmu_smr smrs, u16 id, >> u16 mask) > > Hmm, that name reads as implying the opposite of what it actually tests, > not to mention that passing structs by value is a bit horrid as well :( > It might be okay to name it as subset_of_smr() though. You have any other naming suggestion in mind which could correctly describe the logic? Thanks & regards, Bibek > Thanks, > Robin. > >> +{ >> + return (mask & smrs.mask) == mask && !((id ^ smrs.id) & ~smrs.mask); >> +} >> + >> #define ARM_SMMU_GR0 0 >> #define ARM_SMMU_GR1 1 >> #define ARM_SMMU_CB(s, n) ((s)->numpage + (n)) >> -- >> 2.17.1 >>
>> @@ -467,6 +505,9 @@ static struct arm_smmu_device >> *qcom_smmu_create(struct arm_smmu_device *smmu, >> qsmmu->smmu.impl = impl; >> qsmmu->cfg = data->cfg; >> >> + if (data->actlrcfg && (data->actlrcfg->size)) >> + qsmmu->actlrcfg = data->actlrcfg; > > Do we really need to replicate multiple parts of the data, or would it > be sensible to just replace qsmmu->cfg with qsmmu->data and handle the > further dereferences in the places that want them? > Mm, could not understand this properly. :( Could you help explain more please? As per my understanding aren't data and qsmmu different structures. qcom_smmu is a superset of arm_smmu housing additonal properties and qcom_smmu_match_data is kind of a superset of arm_smmu_impl with additional specific implmentations, so both needs to be in place? Apologies if I understood your statement incorrectly. >> + >> return &qsmmu->smmu; >> } >> >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> index 593910567b88..4b6862715070 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >> @@ -9,6 +9,7 @@ >> struct qcom_smmu { >> struct arm_smmu_device smmu; >> const struct qcom_smmu_config *cfg; >> + const struct actlr_config *actlrcfg; >> bool bypass_quirk; >> u8 bypass_cbndx; >> u32 stall_enabled; >> @@ -25,6 +26,7 @@ struct qcom_smmu_config { >> }; >>
On 11/15/2023 4:33 PM, Dmitry Baryshkov wrote: > On Wed, 15 Nov 2023 at 11:45, Bibek Kumar Patro > <quic_bibekkum@quicinc.com> wrote: >> >> On 11/14/2023 7:45 PM, Dmitry Baryshkov wrote: >>> On Tue, 14 Nov 2023 at 15:57, Bibek Kumar Patro >>> <quic_bibekkum@quicinc.com> wrote: >>>> >>>> Context caching is re-enabled in the prefetch buffer for Qualcomm SoCs >>>> through SoC specific reset ops, which is disabled in the default MMU-500 >>>> reset ops, but is expected for context banks using ACTLR register to >>>> retain the prefetch value during reset and runtime suspend. >>> >>> Please refer to Documentation/process/submitting-patches.rst and >>> rephrase this following the rules there. >>> >> >> Noted, will go through the description once and rephrase it >> in next version complying with rules. >> >>>> >>>> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> >>>> --- >>>> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 25 ++++++++++++++++++---- >>>> 1 file changed, 21 insertions(+), 4 deletions(-) >>>> >>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>> index 0eaf6f2a2e49..fa867b1d9d16 100644 >>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>> @@ -478,11 +478,28 @@ static int qcom_smmu_def_domain_type(struct device *dev) >>>> return match ? IOMMU_DOMAIN_IDENTITY : 0; >>>> } >>>> >>>> +static int qcom_smmu500_reset(struct arm_smmu_device *smmu) >>>> +{ >>>> + int i; >>>> + u32 reg; >>>> + >>>> + arm_mmu500_reset(smmu); >>>> + >>>> + /* Re-enable context caching after reset */ >>>> + for (i = 0; i < smmu->num_context_banks; ++i) { >>>> + reg = arm_smmu_cb_read(smmu, i, ARM_SMMU_CB_ACTLR); >>>> + reg |= CPRE; >>>> + arm_smmu_cb_write(smmu, i, ARM_SMMU_CB_ACTLR, reg); >>>> + } >>>> + >>>> + return 0; >>>> +} >>>> + >>>> static int qcom_sdm845_smmu500_reset(struct arm_smmu_device *smmu) >>>> { >>>> int ret; >>>> >>>> - arm_mmu500_reset(smmu); >>>> + qcom_smmu500_reset(smmu); >>> >>> Is this applicable for sdm845? For all other platforms supported by >>> qcom_smmu_500 implementation? >>> >> >> In arm_mmu500_reset operation drivers/iommu/arm/arm-smmu/arm-smmu-impl.c >> CPRE bit is reset for all SoC based on mmu500 platform, hence for all >> Qualcomm SoCs including sm845 we are setting back the CPRE bit. > > The errata for the CoreLink MMU-500 requires CPRE to be disabled for > all revisions before r2p2. Do we know whether these SoC used CoreLink > MMU-500 and which version of it? > Just checked all these SoCs are using r2p4 revision. So CPRE needs to be enabled back here then? >> >>>> >>>> /* >>>> * To address performance degradation in non-real time clients, >>>> @@ -509,7 +526,7 @@ static const struct arm_smmu_impl qcom_smmu_500_impl = { >>>> .init_context = qcom_smmu_init_context, >>>> .cfg_probe = qcom_smmu_cfg_probe, >>>> .def_domain_type = qcom_smmu_def_domain_type, >>>> - .reset = arm_mmu500_reset, >>>> + .reset = qcom_smmu500_reset, >>>> .write_s2cr = qcom_smmu_write_s2cr, >>>> .tlb_sync = qcom_smmu_tlb_sync, >>>> }; >>>> @@ -528,7 +545,7 @@ static const struct arm_smmu_impl sm8550_smmu_500_impl = { >>>> .init_context = qcom_smmu_init_context, >>>> .cfg_probe = qcom_smmu_cfg_probe, >>>> .def_domain_type = qcom_smmu_def_domain_type, >>>> - .reset = arm_mmu500_reset, >>>> + .reset = qcom_smmu500_reset, >>>> .write_s2cr = qcom_smmu_write_s2cr, >>>> .tlb_sync = qcom_smmu_tlb_sync, >>>> }; >>>> @@ -544,7 +561,7 @@ static const struct arm_smmu_impl qcom_adreno_smmu_v2_impl = { >>>> static const struct arm_smmu_impl qcom_adreno_smmu_500_impl = { >>>> .init_context = qcom_adreno_smmu_init_context, >>>> .def_domain_type = qcom_smmu_def_domain_type, >>>> - .reset = arm_mmu500_reset, >>>> + .reset = qcom_smmu500_reset, >>>> .alloc_context_bank = qcom_adreno_smmu_alloc_context_bank, >>>> .write_sctlr = qcom_adreno_smmu_write_sctlr, >>>> .tlb_sync = qcom_smmu_tlb_sync, >>>> -- >>>> 2.17.1 >>>> >>> >>> > > > > -- > With best wishes > Dmitry
On Thu, 16 Nov 2023 at 14:45, Bibek Kumar Patro <quic_bibekkum@quicinc.com> wrote: > > > > On 11/15/2023 4:33 PM, Dmitry Baryshkov wrote: > > On Wed, 15 Nov 2023 at 11:45, Bibek Kumar Patro > > <quic_bibekkum@quicinc.com> wrote: > >> > >> On 11/14/2023 7:45 PM, Dmitry Baryshkov wrote: > >>> On Tue, 14 Nov 2023 at 15:57, Bibek Kumar Patro > >>> <quic_bibekkum@quicinc.com> wrote: > >>>> > >>>> Context caching is re-enabled in the prefetch buffer for Qualcomm SoCs > >>>> through SoC specific reset ops, which is disabled in the default MMU-500 > >>>> reset ops, but is expected for context banks using ACTLR register to > >>>> retain the prefetch value during reset and runtime suspend. > >>> > >>> Please refer to Documentation/process/submitting-patches.rst and > >>> rephrase this following the rules there. > >>> > >> > >> Noted, will go through the description once and rephrase it > >> in next version complying with rules. > >> > >>>> > >>>> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> > >>>> --- > >>>> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 25 ++++++++++++++++++---- > >>>> 1 file changed, 21 insertions(+), 4 deletions(-) > >>>> > >>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c > >>>> index 0eaf6f2a2e49..fa867b1d9d16 100644 > >>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c > >>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c > >>>> @@ -478,11 +478,28 @@ static int qcom_smmu_def_domain_type(struct device *dev) > >>>> return match ? IOMMU_DOMAIN_IDENTITY : 0; > >>>> } > >>>> > >>>> +static int qcom_smmu500_reset(struct arm_smmu_device *smmu) > >>>> +{ > >>>> + int i; > >>>> + u32 reg; > >>>> + > >>>> + arm_mmu500_reset(smmu); > >>>> + > >>>> + /* Re-enable context caching after reset */ > >>>> + for (i = 0; i < smmu->num_context_banks; ++i) { > >>>> + reg = arm_smmu_cb_read(smmu, i, ARM_SMMU_CB_ACTLR); > >>>> + reg |= CPRE; > >>>> + arm_smmu_cb_write(smmu, i, ARM_SMMU_CB_ACTLR, reg); > >>>> + } > >>>> + > >>>> + return 0; > >>>> +} > >>>> + > >>>> static int qcom_sdm845_smmu500_reset(struct arm_smmu_device *smmu) > >>>> { > >>>> int ret; > >>>> > >>>> - arm_mmu500_reset(smmu); > >>>> + qcom_smmu500_reset(smmu); > >>> > >>> Is this applicable for sdm845? For all other platforms supported by > >>> qcom_smmu_500 implementation? > >>> > >> > >> In arm_mmu500_reset operation drivers/iommu/arm/arm-smmu/arm-smmu-impl.c > >> CPRE bit is reset for all SoC based on mmu500 platform, hence for all > >> Qualcomm SoCs including sm845 we are setting back the CPRE bit. > > > > The errata for the CoreLink MMU-500 requires CPRE to be disabled for > > all revisions before r2p2. Do we know whether these SoC used CoreLink > > MMU-500 and which version of it? > > > > Just checked all these SoCs are using r2p4 revision. > So CPRE needs to be enabled back here then? can be enabled, yes. > > >> > >>>> > >>>> /* > >>>> * To address performance degradation in non-real time clients, > >>>> @@ -509,7 +526,7 @@ static const struct arm_smmu_impl qcom_smmu_500_impl = { > >>>> .init_context = qcom_smmu_init_context, > >>>> .cfg_probe = qcom_smmu_cfg_probe, > >>>> .def_domain_type = qcom_smmu_def_domain_type, > >>>> - .reset = arm_mmu500_reset, > >>>> + .reset = qcom_smmu500_reset, > >>>> .write_s2cr = qcom_smmu_write_s2cr, > >>>> .tlb_sync = qcom_smmu_tlb_sync, > >>>> }; > >>>> @@ -528,7 +545,7 @@ static const struct arm_smmu_impl sm8550_smmu_500_impl = { > >>>> .init_context = qcom_smmu_init_context, > >>>> .cfg_probe = qcom_smmu_cfg_probe, > >>>> .def_domain_type = qcom_smmu_def_domain_type, > >>>> - .reset = arm_mmu500_reset, > >>>> + .reset = qcom_smmu500_reset, > >>>> .write_s2cr = qcom_smmu_write_s2cr, > >>>> .tlb_sync = qcom_smmu_tlb_sync, > >>>> }; > >>>> @@ -544,7 +561,7 @@ static const struct arm_smmu_impl qcom_adreno_smmu_v2_impl = { > >>>> static const struct arm_smmu_impl qcom_adreno_smmu_500_impl = { > >>>> .init_context = qcom_adreno_smmu_init_context, > >>>> .def_domain_type = qcom_smmu_def_domain_type, > >>>> - .reset = arm_mmu500_reset, > >>>> + .reset = qcom_smmu500_reset, > >>>> .alloc_context_bank = qcom_adreno_smmu_alloc_context_bank, > >>>> .write_sctlr = qcom_adreno_smmu_write_sctlr, > >>>> .tlb_sync = qcom_smmu_tlb_sync, > >>>> -- > >>>> 2.17.1 > >>>> > >>> > >>> > > > > > > > > -- > > With best wishes > > Dmitry
On 16/11/2023 3:24 pm, Dmitry Baryshkov wrote: > On Thu, 16 Nov 2023 at 14:45, Bibek Kumar Patro > <quic_bibekkum@quicinc.com> wrote: >> >> >> >> On 11/15/2023 4:33 PM, Dmitry Baryshkov wrote: >>> On Wed, 15 Nov 2023 at 11:45, Bibek Kumar Patro >>> <quic_bibekkum@quicinc.com> wrote: >>>> >>>> On 11/14/2023 7:45 PM, Dmitry Baryshkov wrote: >>>>> On Tue, 14 Nov 2023 at 15:57, Bibek Kumar Patro >>>>> <quic_bibekkum@quicinc.com> wrote: >>>>>> >>>>>> Context caching is re-enabled in the prefetch buffer for Qualcomm SoCs >>>>>> through SoC specific reset ops, which is disabled in the default MMU-500 >>>>>> reset ops, but is expected for context banks using ACTLR register to >>>>>> retain the prefetch value during reset and runtime suspend. >>>>> >>>>> Please refer to Documentation/process/submitting-patches.rst and >>>>> rephrase this following the rules there. >>>>> >>>> >>>> Noted, will go through the description once and rephrase it >>>> in next version complying with rules. >>>> >>>>>> >>>>>> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> >>>>>> --- >>>>>> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 25 ++++++++++++++++++---- >>>>>> 1 file changed, 21 insertions(+), 4 deletions(-) >>>>>> >>>>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>>>> index 0eaf6f2a2e49..fa867b1d9d16 100644 >>>>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>>>> @@ -478,11 +478,28 @@ static int qcom_smmu_def_domain_type(struct device *dev) >>>>>> return match ? IOMMU_DOMAIN_IDENTITY : 0; >>>>>> } >>>>>> >>>>>> +static int qcom_smmu500_reset(struct arm_smmu_device *smmu) >>>>>> +{ >>>>>> + int i; >>>>>> + u32 reg; >>>>>> + >>>>>> + arm_mmu500_reset(smmu); >>>>>> + >>>>>> + /* Re-enable context caching after reset */ >>>>>> + for (i = 0; i < smmu->num_context_banks; ++i) { >>>>>> + reg = arm_smmu_cb_read(smmu, i, ARM_SMMU_CB_ACTLR); >>>>>> + reg |= CPRE; >>>>>> + arm_smmu_cb_write(smmu, i, ARM_SMMU_CB_ACTLR, reg); >>>>>> + } >>>>>> + >>>>>> + return 0; >>>>>> +} >>>>>> + >>>>>> static int qcom_sdm845_smmu500_reset(struct arm_smmu_device *smmu) >>>>>> { >>>>>> int ret; >>>>>> >>>>>> - arm_mmu500_reset(smmu); >>>>>> + qcom_smmu500_reset(smmu); >>>>> >>>>> Is this applicable for sdm845? For all other platforms supported by >>>>> qcom_smmu_500 implementation? >>>>> >>>> >>>> In arm_mmu500_reset operation drivers/iommu/arm/arm-smmu/arm-smmu-impl.c >>>> CPRE bit is reset for all SoC based on mmu500 platform, hence for all >>>> Qualcomm SoCs including sm845 we are setting back the CPRE bit. >>> >>> The errata for the CoreLink MMU-500 requires CPRE to be disabled for >>> all revisions before r2p2. Do we know whether these SoC used CoreLink >>> MMU-500 and which version of it? >>> >> >> Just checked all these SoCs are using r2p4 revision. >> So CPRE needs to be enabled back here then? > > can be enabled, yes. There are still open errata #562869 and #1047329 which might need this workaround. I guess one could argue that we're not (knowingly) using nested translation at the moment, and also probably not running this in situations which would end up using short-descriptor format, however stuff like pKVM and IOMMUFD could potentially change those assumptions in future, so they still feel a bit sketchy to me. Thanks, Robin. > >> >>>> >>>>>> >>>>>> /* >>>>>> * To address performance degradation in non-real time clients, >>>>>> @@ -509,7 +526,7 @@ static const struct arm_smmu_impl qcom_smmu_500_impl = { >>>>>> .init_context = qcom_smmu_init_context, >>>>>> .cfg_probe = qcom_smmu_cfg_probe, >>>>>> .def_domain_type = qcom_smmu_def_domain_type, >>>>>> - .reset = arm_mmu500_reset, >>>>>> + .reset = qcom_smmu500_reset, >>>>>> .write_s2cr = qcom_smmu_write_s2cr, >>>>>> .tlb_sync = qcom_smmu_tlb_sync, >>>>>> }; >>>>>> @@ -528,7 +545,7 @@ static const struct arm_smmu_impl sm8550_smmu_500_impl = { >>>>>> .init_context = qcom_smmu_init_context, >>>>>> .cfg_probe = qcom_smmu_cfg_probe, >>>>>> .def_domain_type = qcom_smmu_def_domain_type, >>>>>> - .reset = arm_mmu500_reset, >>>>>> + .reset = qcom_smmu500_reset, >>>>>> .write_s2cr = qcom_smmu_write_s2cr, >>>>>> .tlb_sync = qcom_smmu_tlb_sync, >>>>>> }; >>>>>> @@ -544,7 +561,7 @@ static const struct arm_smmu_impl qcom_adreno_smmu_v2_impl = { >>>>>> static const struct arm_smmu_impl qcom_adreno_smmu_500_impl = { >>>>>> .init_context = qcom_adreno_smmu_init_context, >>>>>> .def_domain_type = qcom_smmu_def_domain_type, >>>>>> - .reset = arm_mmu500_reset, >>>>>> + .reset = qcom_smmu500_reset, >>>>>> .alloc_context_bank = qcom_adreno_smmu_alloc_context_bank, >>>>>> .write_sctlr = qcom_adreno_smmu_write_sctlr, >>>>>> .tlb_sync = qcom_smmu_tlb_sync, >>>>>> -- >>>>>> 2.17.1 >>>>>> >>>>> >>>>> >>> >>> >>> >>> -- >>> With best wishes >>> Dmitry > > >
On 11/15/2023 8:23 PM, Robin Murphy wrote: > On 2023-11-15 1:54 pm, Bibek Kumar Patro wrote: >> >>>> @@ -467,6 +505,9 @@ static struct arm_smmu_device >>>> *qcom_smmu_create(struct arm_smmu_device *smmu, >>>> qsmmu->smmu.impl = impl; >>>> qsmmu->cfg = data->cfg; >>>> >>>> + if (data->actlrcfg && (data->actlrcfg->size)) >>>> + qsmmu->actlrcfg = data->actlrcfg; >>> >>> Do we really need to replicate multiple parts of the data, or would >>> it be sensible to just replace qsmmu->cfg with qsmmu->data and handle >>> the further dereferences in the places that want them? >>> >> >> Mm, could not understand this properly. :( Could you help explain more >> please? >> As per my understanding aren't data and qsmmu different structures. >> qcom_smmu is a superset of arm_smmu housing additonal properties >> and qcom_smmu_match_data is kind of a superset of arm_smmu_impl with >> additional specific implmentations, so both needs to be in place? >> Apologies if I understood your statement incorrectly. > > My point is that the data is static and constant, so there's really no > point storing multiple pointers into different bits of it. So rather than: > > qsmmu->cfg = data->cfg; > qssmu->actlrcfg = data->actlrcfg; > ... > do_something(qsmmu->cfg); > ... > do_other_thing(qsmmu->actlrcfg); > > we can just store the one pointer and have: > > qsmmu->data = data; > ... > do_something(qsmmu->data->cfg); > ... > do_other_thing(qsmmu->data->actlrcfg); > > Thanks, > Robin. > I see, this looks like probably we need a separate patch altogether for this cleanup, as "cfg" is used in other fault handling places as well as i can see and is introduced as a part of different patch. Should we scope this work for a separate patch if it's okay? Thanks, Bibek >>>> + >>>> return &qsmmu->smmu; >>>> } >>>> >>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> index 593910567b88..4b6862715070 100644 >>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> @@ -9,6 +9,7 @@ >>>> struct qcom_smmu { >>>> struct arm_smmu_device smmu; >>>> const struct qcom_smmu_config *cfg; >>>> + const struct actlr_config *actlrcfg; >>>> bool bypass_quirk; >>>> u8 bypass_cbndx; >>>> u32 stall_enabled; >>>> @@ -25,6 +26,7 @@ struct qcom_smmu_config { >>>> }; >>>>
On 11/15/2023 10:13 PM, Konrad Dybcio wrote: > > > On 11/14/23 14:56, Bibek Kumar Patro wrote: >> Context caching is re-enabled in the prefetch buffer for Qualcomm SoCs >> through SoC specific reset ops, which is disabled in the default MMU-500 >> reset ops, but is expected for context banks using ACTLR register to >> retain the prefetch value during reset and runtime suspend. >> >> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> >> --- > And I assume that goes for all SMMU500 implementations? > Right, for all SMMU500 implementation for Qualcomm SoCs. Hence implemented this enablement with Qualcomm specific reset operation. > Looking at the 8550 ACTRL array from patch 2, CPRE is not enabled > at all times.. Is that because of performance, or some other > technical reason? > > Will this regress platforms without ACTRL tables? > It should not regress, If you check my recent reply on Dimitry's response, the Corelink revision is r2p4 and it can be enabled. On the Robin's mentioned errata workarounds, let me check once. Thanks & regards, Bibek > Konrad
On 11/16/2023 10:34 PM, Robin Murphy wrote: > On 16/11/2023 3:24 pm, Dmitry Baryshkov wrote: >> On Thu, 16 Nov 2023 at 14:45, Bibek Kumar Patro >> <quic_bibekkum@quicinc.com> wrote: >>> >>> >>> >>> On 11/15/2023 4:33 PM, Dmitry Baryshkov wrote: >>>> On Wed, 15 Nov 2023 at 11:45, Bibek Kumar Patro >>>> <quic_bibekkum@quicinc.com> wrote: >>>>> >>>>> On 11/14/2023 7:45 PM, Dmitry Baryshkov wrote: >>>>>> On Tue, 14 Nov 2023 at 15:57, Bibek Kumar Patro >>>>>> <quic_bibekkum@quicinc.com> wrote: >>>>>>> >>>>>>> Context caching is re-enabled in the prefetch buffer for Qualcomm >>>>>>> SoCs >>>>>>> through SoC specific reset ops, which is disabled in the default >>>>>>> MMU-500 >>>>>>> reset ops, but is expected for context banks using ACTLR register to >>>>>>> retain the prefetch value during reset and runtime suspend. >>>>>> >>>>>> Please refer to Documentation/process/submitting-patches.rst and >>>>>> rephrase this following the rules there. >>>>>> >>>>> >>>>> Noted, will go through the description once and rephrase it >>>>> in next version complying with rules. >>>>> >>>>>>> >>>>>>> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> >>>>>>> --- >>>>>>> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 25 >>>>>>> ++++++++++++++++++---- >>>>>>> 1 file changed, 21 insertions(+), 4 deletions(-) >>>>>>> >>>>>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>>>>> b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>>>>> index 0eaf6f2a2e49..fa867b1d9d16 100644 >>>>>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>>>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>>>>> @@ -478,11 +478,28 @@ static int qcom_smmu_def_domain_type(struct >>>>>>> device *dev) >>>>>>> return match ? IOMMU_DOMAIN_IDENTITY : 0; >>>>>>> } >>>>>>> >>>>>>> +static int qcom_smmu500_reset(struct arm_smmu_device *smmu) >>>>>>> +{ >>>>>>> + int i; >>>>>>> + u32 reg; >>>>>>> + >>>>>>> + arm_mmu500_reset(smmu); >>>>>>> + >>>>>>> + /* Re-enable context caching after reset */ >>>>>>> + for (i = 0; i < smmu->num_context_banks; ++i) { >>>>>>> + reg = arm_smmu_cb_read(smmu, i, ARM_SMMU_CB_ACTLR); >>>>>>> + reg |= CPRE; >>>>>>> + arm_smmu_cb_write(smmu, i, ARM_SMMU_CB_ACTLR, reg); >>>>>>> + } >>>>>>> + >>>>>>> + return 0; >>>>>>> +} >>>>>>> + >>>>>>> static int qcom_sdm845_smmu500_reset(struct arm_smmu_device >>>>>>> *smmu) >>>>>>> { >>>>>>> int ret; >>>>>>> >>>>>>> - arm_mmu500_reset(smmu); >>>>>>> + qcom_smmu500_reset(smmu); >>>>>> >>>>>> Is this applicable for sdm845? For all other platforms supported by >>>>>> qcom_smmu_500 implementation? >>>>>> >>>>> >>>>> In arm_mmu500_reset operation >>>>> drivers/iommu/arm/arm-smmu/arm-smmu-impl.c >>>>> CPRE bit is reset for all SoC based on mmu500 platform, hence for all >>>>> Qualcomm SoCs including sm845 we are setting back the CPRE bit. >>>> >>>> The errata for the CoreLink MMU-500 requires CPRE to be disabled for >>>> all revisions before r2p2. Do we know whether these SoC used CoreLink >>>> MMU-500 and which version of it? >>>> >>> >>> Just checked all these SoCs are using r2p4 revision. >>> So CPRE needs to be enabled back here then? >> >> can be enabled, yes. > > There are still open errata #562869 and #1047329 which might need this > workaround. I guess one could argue that we're not (knowingly) using > nested translation at the moment, and also probably not running this in > situations which would end up using short-descriptor format, however > stuff like pKVM and IOMMUFD could potentially change those assumptions > in future, so they still feel a bit sketchy to me. > Could you help provide some details on these two errata (#562869 and #1047329).Both of these erratum are there for r2p4 revisions as well? Thanks & regards, Bibek > Thanks, > Robin. > >> >>> >>>>> >>>>>>> >>>>>>> /* >>>>>>> * To address performance degradation in non-real time >>>>>>> clients, >>>>>>> @@ -509,7 +526,7 @@ static const struct arm_smmu_impl >>>>>>> qcom_smmu_500_impl = { >>>>>>> .init_context = qcom_smmu_init_context, >>>>>>> .cfg_probe = qcom_smmu_cfg_probe, >>>>>>> .def_domain_type = qcom_smmu_def_domain_type, >>>>>>> - .reset = arm_mmu500_reset, >>>>>>> + .reset = qcom_smmu500_reset, >>>>>>> .write_s2cr = qcom_smmu_write_s2cr, >>>>>>> .tlb_sync = qcom_smmu_tlb_sync, >>>>>>> }; >>>>>>> @@ -528,7 +545,7 @@ static const struct arm_smmu_impl >>>>>>> sm8550_smmu_500_impl = { >>>>>>> .init_context = qcom_smmu_init_context, >>>>>>> .cfg_probe = qcom_smmu_cfg_probe, >>>>>>> .def_domain_type = qcom_smmu_def_domain_type, >>>>>>> - .reset = arm_mmu500_reset, >>>>>>> + .reset = qcom_smmu500_reset, >>>>>>> .write_s2cr = qcom_smmu_write_s2cr, >>>>>>> .tlb_sync = qcom_smmu_tlb_sync, >>>>>>> }; >>>>>>> @@ -544,7 +561,7 @@ static const struct arm_smmu_impl >>>>>>> qcom_adreno_smmu_v2_impl = { >>>>>>> static const struct arm_smmu_impl qcom_adreno_smmu_500_impl = { >>>>>>> .init_context = qcom_adreno_smmu_init_context, >>>>>>> .def_domain_type = qcom_smmu_def_domain_type, >>>>>>> - .reset = arm_mmu500_reset, >>>>>>> + .reset = qcom_smmu500_reset, >>>>>>> .alloc_context_bank = >>>>>>> qcom_adreno_smmu_alloc_context_bank, >>>>>>> .write_sctlr = qcom_adreno_smmu_write_sctlr, >>>>>>> .tlb_sync = qcom_smmu_tlb_sync, >>>>>>> -- >>>>>>> 2.17.1 >>>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >>>> -- >>>> With best wishes >>>> Dmitry >> >> >>
On 11/15/2023 8:34 PM, Robin Murphy wrote: > On 2023-11-15 12:32 pm, Bibek Kumar Patro wrote: >> >> >> On 11/14/2023 10:25 PM, Robin Murphy wrote: >>> On 14/11/2023 1:56 pm, Bibek Kumar Patro wrote: >>>> Currently in Qualcomm SoCs the default prefetch is set to 1 which >>>> allows >>>> the TLB to fetch just the next page table. MMU-500 features ACTLR >>>> register which is implementation defined and is used for Qualcomm SoCs >>>> to have a prefetch setting of 1/3/7/15 enabling TLB to prefetch >>>> the next set of page tables accordingly allowing for faster >>>> translations. >>>> >>>> ACTLR value is unique for each SMR (Stream matching register) and >>>> stored >>>> in a pre-populated table. This value is set to the register during >>>> context bank initialisation. >>>> >>>> Signed-off-by: Bibek Kumar Patro <quic_bibekkum@quicinc.com> >>>> --- >>>> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 41 >>>> ++++++++++++++++++++++ >>>> drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h | 2 ++ >>>> drivers/iommu/arm/arm-smmu/arm-smmu.c | 5 +-- >>>> drivers/iommu/arm/arm-smmu/arm-smmu.h | 5 +++ >>>> 4 files changed, 51 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>> b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>> index 549ae4dba3a6..578c662c7c30 100644 >>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c >>>> @@ -14,6 +14,17 @@ >>>> >>>> #define QCOM_DUMMY_VAL -1 >>>> >>>> +struct actlr_config { >>>> + const struct actlr_data *adata; >>>> + size_t size; >>>> +}; >>>> + >>>> +struct actlr_data { >>>> + u16 sid; >>>> + u16 mask; >>> >>> Do we need to worry about masks? If you're already assuming that any >>> SMR will be programmed to match a superset of the data here, surely a >>> single unique ID per device would suffice? >>> Thanks for this comment. I see now, only assuming smrs might not be able to correctly set the ACTLR values for all the context banks without the index. it should be smr = smmu->smrs[idx] instead of smr = smmu->smrs or it will be considering the single SMR for all the associated devices. >> >> If you refer to the arm_smmu_set_actlr below, mask would be needed as >> we would check for mask along with sid as well while assigning actlr >> configuration. Also with mask no of actlr entries can be reduces as >> with out mask we have to have entry for each sid. > > But why? As far as I can tell, the design in patch #2 is to have a > single data entry for each distinct device, since you're expecting to > see a single SMR programmed to match at least the full id/mask range of > the entry. However any SMR which matches that full range will by > definition also match any smaller subset of that range as well, so we > can uniquely identify any device in this context from just any *one* of > its IDs, thus we could store less data and have simpler matching logic. > I was going though your earlier reply once again and noticed we would need to mention the context bank index against the smr as well to take the right ACTLR value, as I replied to your earlier response above. >>>> + u32 actlr; >>>> +}; >>>> + >>>> static struct qcom_smmu *to_qcom_smmu(struct arm_smmu_device *smmu) >>>> { >>>> return container_of(smmu, struct qcom_smmu, smmu); >>>> @@ -261,9 +272,36 @@ static const struct of_device_id >>>> qcom_smmu_client_of_match[] __maybe_unused = { >>>> { } >>>> }; >>>> >>>> +static void arm_smmu_set_actlr(struct arm_smmu_device *smmu, int idx, >>>> + const struct actlr_config *actlrcfg) >>>> +{ >>>> + struct arm_smmu_smr *smr = smmu->smrs; It would be smr = smmu->smrs[idx] instead of smr = smmu->smrs to get sid of individual device attached to corresponding context bank. >>>> + int i; >>>> + u16 id; >>>> + u16 mask; >>>> + >>>> + for (i = 0; i < actlrcfg->size; ++i) { >>>> + id = actlrcfg->adata[i].sid; >>>> + mask = actlrcfg->adata[i].mask; >>>> + if (!smr_is_subset(*smr, id, mask)) >>> >>> How well have you tested this? ;) >>> >> >> Well, this logic has worked pretty good for us till now in our >> downstream implementation. :) (During testing as well this logic >> helped to better match the SMRs instead of manually mathcing the mask >> and SID which missed some SIDs) >> Also this is already being used to arm_smmu_find_sme hence packaged this >> logic in a wrapper to be used in other places as well(including ACTLR >> register setting case here.) > > 1) Look at what you're matching the id and mask values *against* > Got this now. Will take care of this in next version. > 2) Look at the polarity of the condition > > 3) Consider the old saying "two wrongs don't make a right" > Noted. >>>> + arm_smmu_cb_write(smmu, idx, ARM_SMMU_CB_ACTLR, >>>> + actlrcfg->adata[i].actlr); >>>> + } >>>> +} >>>> + >>>> static int qcom_smmu_init_context(struct arm_smmu_domain >>>> *smmu_domain, >>>> struct io_pgtable_cfg *pgtbl_cfg, struct device *dev) >>>> { >>>> + struct arm_smmu_device *smmu = smmu_domain->smmu; >>>> + struct qcom_smmu *qsmmu = to_qcom_smmu(smmu); >>>> + const struct actlr_config *actlrcfg; >>>> + int idx = smmu_domain->cfg.cbndx; >>>> + >>>> + if (qsmmu->actlrcfg) { >>>> + actlrcfg = qsmmu->actlrcfg; >>>> + arm_smmu_set_actlr(smmu, idx, actlrcfg); >>>> + } >>>> + >>>> smmu_domain->cfg.flush_walk_prefer_tlbiasid = true; >>>> >>>> return 0; >>>> @@ -467,6 +505,9 @@ static struct arm_smmu_device >>>> *qcom_smmu_create(struct arm_smmu_device *smmu, >>>> qsmmu->smmu.impl = impl; >>>> qsmmu->cfg = data->cfg; >>>> >>>> + if (data->actlrcfg && (data->actlrcfg->size)) >>>> + qsmmu->actlrcfg = data->actlrcfg; >>> >>> Do we really need to replicate multiple parts of the data, or would >>> it be sensible to just replace qsmmu->cfg with qsmmu->data and handle >>> the further dereferences in the places that want them? >>> >>>> + >>>> return &qsmmu->smmu; >>>> } >>>> >>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> index 593910567b88..4b6862715070 100644 >>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu-qcom.h >>>> @@ -9,6 +9,7 @@ >>>> struct qcom_smmu { >>>> struct arm_smmu_device smmu; >>>> const struct qcom_smmu_config *cfg; >>>> + const struct actlr_config *actlrcfg; >>>> bool bypass_quirk; >>>> u8 bypass_cbndx; >>>> u32 stall_enabled; >>>> @@ -25,6 +26,7 @@ struct qcom_smmu_config { >>>> }; >>>> >>>> struct qcom_smmu_match_data { >>>> + const struct actlr_config *actlrcfg; >>>> const struct qcom_smmu_config *cfg; >>>> const struct arm_smmu_impl *impl; >>>> const struct arm_smmu_impl *adreno_impl; >>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c >>>> b/drivers/iommu/arm/arm-smmu/arm-smmu.c >>>> index d6d1a2a55cc0..8e4faf015286 100644 >>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c >>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c >>>> @@ -990,9 +990,10 @@ static int arm_smmu_find_sme(struct >>>> arm_smmu_device *smmu, u16 id, u16 mask) >>>> * expect simply identical entries for this case, but there's >>>> * no harm in accommodating the generalisation. >>>> */ >>>> - if ((mask & smrs[i].mask) == mask && >>>> - !((id ^ smrs[i].id) & ~smrs[i].mask)) >>>> + >>>> + if (smr_is_subset(smrs[i], id, mask)) >>>> return i; >>>> + >>>> /* >>>> * If the new entry has any other overlap with an existing >>>> one, >>>> * though, then there always exists at least one stream ID >>>> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h >>>> b/drivers/iommu/arm/arm-smmu/arm-smmu.h >>>> index 703fd5817ec1..b1638bbc41d4 100644 >>>> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h >>>> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h >>>> @@ -501,6 +501,11 @@ static inline void arm_smmu_writeq(struct >>>> arm_smmu_device *smmu, int page, >>>> writeq_relaxed(val, arm_smmu_page(smmu, page) + offset); >>>> } >>>> >>>> +static inline bool smr_is_subset(struct arm_smmu_smr smrs, u16 id, >>>> u16 mask) >>> >>> Hmm, that name reads as implying the opposite of what it actually >>> tests, not to mention that passing structs by value is a bit horrid >>> as well :( >>> >> >> It might be okay to name it as subset_of_smr() though. You have any >> other naming suggestion in mind which could correctly describe the >> logic? > > As above I think the ideal answer is to avoid the reason for factoring > it out at all, by using a simpler matching process to begin with. > As per the approach mentioned above to use the context bank index, this same approach should be okay to go forward with. Thanks Robin for you above comments, I have noted the inputs and will take care of this in next version. Thanks & regards, Bibek > Thanks, > Robin.