| Message ID | 20201028140847.1018-1-lukasz.luba@arm.com |
|---|---|
| Headers | show |
| Series | Add sustainable OPP concept | expand |
On 28-10-20, 14:08, Lukasz Luba wrote: > Hi all, > > This patch set introduces a concept of sustainable OPP, which then can be used > by kernel frameworks or governors for estimating system sustainable system > state. This kind of estimation is done e.g. in thermal governor Intelligent > Power Allocation (IPA), which calculates sustainable power of the whole system > and then derives some coefficients for internal algorithm. > > The patch set introduces a new DT bindings 'opp-sustainable', with parsing > code. It also adds a function (in patch 3/4) which allows device drivers to set > directly the sustainable OPP. This is helpful when the device drivers populate > the OPP table by themself (example in patch 4/4). > Can we please have some more information about this ? What does the sustainable OPP mean ? How will platform guys know or learn about this ? How we are going to use it finally ? What does it have to do with temperature of the SoC or the thermal affects, etc. -- viresh
On 29-10-20, 13:10, Viresh Kumar wrote: > On 28-10-20, 14:08, Lukasz Luba wrote: > > Hi all, > > > > This patch set introduces a concept of sustainable OPP, which then can be used > > by kernel frameworks or governors for estimating system sustainable system > > state. This kind of estimation is done e.g. in thermal governor Intelligent > > Power Allocation (IPA), which calculates sustainable power of the whole system > > and then derives some coefficients for internal algorithm. > > > > The patch set introduces a new DT bindings 'opp-sustainable', with parsing > > code. It also adds a function (in patch 3/4) which allows device drivers to set > > directly the sustainable OPP. This is helpful when the device drivers populate > > the OPP table by themself (example in patch 4/4). > > > > Can we please have some more information about this ? What does the > sustainable OPP mean ? How will platform guys know or learn about this > ? How we are going to use it finally ? What does it have to do with > temperature of the SoC or the thermal affects, etc. And that we need a real user of this first if it is ever going to be merged.
On 10/29/20 7:53 AM, Viresh Kumar wrote: > On 29-10-20, 13:10, Viresh Kumar wrote: >> On 28-10-20, 14:08, Lukasz Luba wrote: >>> Hi all, >>> >>> This patch set introduces a concept of sustainable OPP, which then can be used >>> by kernel frameworks or governors for estimating system sustainable system >>> state. This kind of estimation is done e.g. in thermal governor Intelligent >>> Power Allocation (IPA), which calculates sustainable power of the whole system >>> and then derives some coefficients for internal algorithm. >>> >>> The patch set introduces a new DT bindings 'opp-sustainable', with parsing >>> code. It also adds a function (in patch 3/4) which allows device drivers to set >>> directly the sustainable OPP. This is helpful when the device drivers populate >>> the OPP table by themself (example in patch 4/4). >>> >> >> Can we please have some more information about this ? What does the >> sustainable OPP mean ? How will platform guys know or learn about this >> ? How we are going to use it finally ? What does it have to do with >> temperature of the SoC or the thermal affects, etc. There were discussions about Energy Model (EM), scale of values (mW or abstract scale) and relation to EAS and IPA. You can find quite long discussion below v2 [1] (there is also v3 send after agreement [2]). We have in thermal DT binding: 'sustainable-power' expressed in mW, which is used by IPA, but it would not support bogoWatts. The sustainable power is used for estimation of internal coefficients (also for power budget), which I am trying to change to work with 'abstract scale' [3][4]. This would allow to estimate sustainable power of the system based on CPUs, GPU opp-sustainable points, where we don't have 'sustainable-power' or devices using bogoWatts. > > And that we need a real user of this first if it is ever going to be > merged. > IPA would be the first user of this in combination with scmi-cpufreq.c, which can feed 'abstract scale' in to EM. Currently IPA takes lowest allowed OPPs into account for this estimation which is not optimal. This marked OPPs would make estimation a lot better. Regards, Lukasz [1] https://lore.kernel.org/lkml/20201002114426.31277-1-lukasz.luba@arm.com/ [2] https://lore.kernel.org/lkml/20201019140601.3047-1-lukasz.luba@arm.com/ [3] https://lore.kernel.org/linux-pm/5f682bbb-b250-49e6-dbb7-aea522a58595@arm.com/ [4] https://lore.kernel.org/lkml/20201009135850.14727-1-lukasz.luba@arm.com/
On 10:04-20201029, Lukasz Luba wrote: > > > On 10/28/20 9:47 PM, Nishanth Menon wrote: > > On 14:08-20201028, Lukasz Luba wrote: > > > Add opp-sustainable as an additional property in the OPP node to describe > > > the sustainable performance level of the device. This will help to > > > estimate the sustainable performance of the whole system. > > > > > > Signed-off-by: Lukasz Luba <lukasz.luba@arm.com> > > > --- > > > Documentation/devicetree/bindings/opp/opp.txt | 4 ++++ > > > 1 file changed, 4 insertions(+) > > > > > > diff --git a/Documentation/devicetree/bindings/opp/opp.txt b/Documentation/devicetree/bindings/opp/opp.txt > > > index 9847dfeeffcb..cd01028de305 100644 > > > --- a/Documentation/devicetree/bindings/opp/opp.txt > > > +++ b/Documentation/devicetree/bindings/opp/opp.txt > > > @@ -154,6 +154,10 @@ Optional properties: > > > - opp-suspend: Marks the OPP to be used during device suspend. If multiple OPPs > > > in the table have this, the OPP with highest opp-hz will be used. > > > +- opp-sustainable: Marks the OPP as sustainable. This property can be used for > > > + estimating sustainable performance of the whole system. If multiple OPPs in > > > + the table have this, the OPP with highest opp-hz will be used. > > > > > > By "sustainable", do you mean sustainable across Process, Voltage and > > Temperature corners upto the max rated operational Power-ON hours > > without IDLE state being achieved on the processor? > > Yes, in case of CPU: running 100% without idle at that particular OPP. > Running above that OPP would lead to cross control temperature. We need to tighten the definitions a lot more here and add that to the binding. What we are stating, if I am not misunderstanding is an OPP that is guaranteed by SoC vendor that across Process Voltage and Temperature corners - aka across the entire production spectrum for the part number, *all* devices will operate at this OPP for the mandated power-on-hours rating without hitting IDLE. Example: So -40C to 125C, across the process (hot/cold/nominal), 100s of thousands/millions of units can operate upto 125,0000 power-on-hours while running a tight deadloop OR maybe high processing function or even cpuburn[1]? Can you give me one SoC vendor and part that guarantees this? I am wondering if this is all theoretical... There are tons of parameters that come into play for "reliability" "sustainability" etc. Those are tricky terminology that typically makes legal folks pretty happy to debate for decades.. just my 2 cents. > > > > > OR do you mean to leave it up to interpretation? > > I can tell how I would use them. There is thermal governor IPA, which > needs sustainable power either form DT or uses internal algorithm to > estimate it based on lowest allowed freq OPPs. Then it estimated > internal coefficients based on that value, which is not optimal > for lowest OPPs. When some higher OPP could be marked as sustainable, > it would lead to better estimation and better power budget split. Seeing your series, I got an idea about how you plan on using it, I just think we need to be more precise in our definition.. [1] https://patrickmn.com/projects/cpuburn/
On 10/29/20 12:59 PM, Nishanth Menon wrote: > On 10:04-20201029, Lukasz Luba wrote: >> >> >> On 10/28/20 9:47 PM, Nishanth Menon wrote: >>> On 14:08-20201028, Lukasz Luba wrote: >>>> Add opp-sustainable as an additional property in the OPP node to describe >>>> the sustainable performance level of the device. This will help to >>>> estimate the sustainable performance of the whole system. >>>> >>>> Signed-off-by: Lukasz Luba <lukasz.luba@arm.com> >>>> --- >>>> Documentation/devicetree/bindings/opp/opp.txt | 4 ++++ >>>> 1 file changed, 4 insertions(+) >>>> >>>> diff --git a/Documentation/devicetree/bindings/opp/opp.txt b/Documentation/devicetree/bindings/opp/opp.txt >>>> index 9847dfeeffcb..cd01028de305 100644 >>>> --- a/Documentation/devicetree/bindings/opp/opp.txt >>>> +++ b/Documentation/devicetree/bindings/opp/opp.txt >>>> @@ -154,6 +154,10 @@ Optional properties: >>>> - opp-suspend: Marks the OPP to be used during device suspend. If multiple OPPs >>>> in the table have this, the OPP with highest opp-hz will be used. >>>> +- opp-sustainable: Marks the OPP as sustainable. This property can be used for >>>> + estimating sustainable performance of the whole system. If multiple OPPs in >>>> + the table have this, the OPP with highest opp-hz will be used. >>> >>> >>> By "sustainable", do you mean sustainable across Process, Voltage and >>> Temperature corners upto the max rated operational Power-ON hours >>> without IDLE state being achieved on the processor? >> >> Yes, in case of CPU: running 100% without idle at that particular OPP. >> Running above that OPP would lead to cross control temperature. > > We need to tighten the definitions a lot more here and add that to the > binding. What we are stating, if I am not misunderstanding is an OPP > that is guaranteed by SoC vendor that across Process Voltage and > Temperature corners - aka across the entire production spectrum > for the part number, *all* devices will operate at this OPP for the > mandated power-on-hours rating without hitting IDLE. > > Example: So -40C to 125C, across the process (hot/cold/nominal), 100s of > thousands/millions of units can operate upto 125,0000 power-on-hours > while running a tight deadloop OR maybe high processing function or even > cpuburn[1]? I think I know what you mean. But this would lead to redefining a lot more that just this optional field. This wide range -40C to 125C is for automotive chips, then what about opp-suspend, when the device cannot even reach that OPP under some stress test e.g. outside temp ~100-110C... Or opp-turbo, shell all the OPPs have multidimensional table to reflect the temperature dependency for all affected optional fields? > > > Can you give me one SoC vendor and part that guarantees this? I am > wondering if this is all theoretical... There are tons of parameters > that come into play for "reliability" "sustainability" etc. Those are > tricky terminology that typically makes legal folks pretty happy to > debate for decades.. Yes, but the outside temperature is probably most important for this use case. > > just my 2 cents. >> >>> >>> OR do you mean to leave it up to interpretation? >> >> I can tell how I would use them. There is thermal governor IPA, which >> needs sustainable power either form DT or uses internal algorithm to >> estimate it based on lowest allowed freq OPPs. Then it estimated >> internal coefficients based on that value, which is not optimal >> for lowest OPPs. When some higher OPP could be marked as sustainable, >> it would lead to better estimation and better power budget split. > > Seeing your series, I got an idea about how you plan on using it, I > just think we need to be more precise in our definition.. Thank you for having a look on that and understanding the motivation behind this series. How about adding a description that this sustainable OPP is considered for normal room temp (20-25C)? BTW, in the Arm SCMI spec definition of that value (used in patch 4/4), there is no specific temperature for it, just: 'This is the maximum performance level that the platform can sustain under normal conditions. In exceptional circumstances, such as thermal runaway, the platform might not be be able to guarantee this level.' I can put this whole description into the DT binding, if you like. > > [1] https://patrickmn.com/projects/cpuburn/ >
On 13:33-20201029, Lukasz Luba wrote: > > > On 10/29/20 12:59 PM, Nishanth Menon wrote: > > On 10:04-20201029, Lukasz Luba wrote: > > > > > > > > > On 10/28/20 9:47 PM, Nishanth Menon wrote: > > > > On 14:08-20201028, Lukasz Luba wrote: > > > > > Add opp-sustainable as an additional property in the OPP node to describe > > > > > the sustainable performance level of the device. This will help to > > > > > estimate the sustainable performance of the whole system. > > > > > > > > > > Signed-off-by: Lukasz Luba <lukasz.luba@arm.com> > > > > > --- > > > > > Documentation/devicetree/bindings/opp/opp.txt | 4 ++++ > > > > > 1 file changed, 4 insertions(+) > > > > > > > > > > diff --git a/Documentation/devicetree/bindings/opp/opp.txt b/Documentation/devicetree/bindings/opp/opp.txt > > > > > index 9847dfeeffcb..cd01028de305 100644 > > > > > --- a/Documentation/devicetree/bindings/opp/opp.txt > > > > > +++ b/Documentation/devicetree/bindings/opp/opp.txt > > > > > @@ -154,6 +154,10 @@ Optional properties: > > > > > - opp-suspend: Marks the OPP to be used during device suspend. If multiple OPPs > > > > > in the table have this, the OPP with highest opp-hz will be used. > > > > > +- opp-sustainable: Marks the OPP as sustainable. This property can be used for > > > > > + estimating sustainable performance of the whole system. If multiple OPPs in > > > > > + the table have this, the OPP with highest opp-hz will be used. > > > > > > > > > > > > By "sustainable", do you mean sustainable across Process, Voltage and > > > > Temperature corners upto the max rated operational Power-ON hours > > > > without IDLE state being achieved on the processor? > > > > > > Yes, in case of CPU: running 100% without idle at that particular OPP. > > > Running above that OPP would lead to cross control temperature. > > > > We need to tighten the definitions a lot more here and add that to the > > binding. What we are stating, if I am not misunderstanding is an OPP > > that is guaranteed by SoC vendor that across Process Voltage and > > Temperature corners - aka across the entire production spectrum > > for the part number, *all* devices will operate at this OPP for the > > mandated power-on-hours rating without hitting IDLE. > > > > Example: So -40C to 125C, across the process (hot/cold/nominal), 100s of > > thousands/millions of units can operate upto 125,0000 power-on-hours > > while running a tight deadloop OR maybe high processing function or even > > cpuburn[1]? > > I think I know what you mean. But this would lead to redefining a lot > more that just this optional field. This wide range -40C to 125C is for > automotive chips, then what about opp-suspend, when the device cannot > even reach that OPP under some stress test e.g. outside temp > ~100-110C... > Or opp-turbo, shell all the OPPs have multidimensional table to reflect > the temperature dependency for all affected optional fields? yes, and down the rabbit hole we will go :) > > > > > > > Can you give me one SoC vendor and part that guarantees this? I am > > wondering if this is all theoretical... There are tons of parameters > > that come into play for "reliability" "sustainability" etc. Those are > > tricky terminology that typically makes legal folks pretty happy to > > debate for decades.. > > Yes, but the outside temperature is probably most important for this use > case. > > > > > just my 2 cents. > > > > > > > > > > > OR do you mean to leave it up to interpretation? > > > > > > I can tell how I would use them. There is thermal governor IPA, which > > > needs sustainable power either form DT or uses internal algorithm to > > > estimate it based on lowest allowed freq OPPs. Then it estimated > > > internal coefficients based on that value, which is not optimal > > > for lowest OPPs. When some higher OPP could be marked as sustainable, > > > it would lead to better estimation and better power budget split. > > > > Seeing your series, I got an idea about how you plan on using it, I > > just think we need to be more precise in our definition.. > > Thank you for having a look on that and understanding the motivation > behind this series. > > How about adding a description that this sustainable OPP is considered > for normal room temp (20-25C)? You could.. but then, practically as we go into smaller process nodes, the 20-25C reliability is just theoretical. I mean, we Texans in summer or Finns in winter would probably define "normal room temperature" as something different in practise (ISO not withstanding ;) ).. Challenge of reliability has always been on the edge of the PVT ranges. To ignore that OR to have a scheme that does not scale to describe that, IMHO is a lacking definition. My entire point is, if we can avoid getting into rabbit hole definitions, we probably should.. IMHO.. keep things as simple as possible. > > BTW, in the Arm SCMI spec definition of that value (used in patch 4/4), You mean [1] Table 11 Performance Domain Levels with Special Significance > there is no specific temperature for it, just: > 'This is the maximum performance level that the platform can > sustain under normal conditions. In exceptional circumstances, > such as thermal runaway, the platform might not be be able to > guarantee this level.' > Hehe.. Vincent and SCMI teams have been having fun there :)... But, I think the definition has little practical significance for the very reasons I made above IMHO, and with full respect to SCMI team(defining SCMI is not an easy task, I admit) - it is at best a theoretical, "works at the engineer's cube definition", as typical "nominal operation conditions" escape clause tend to be, OR at the worst ignoring to define the parameters that constitute what would bound things in a closed box precisely (example: does'nt mention process, so just nominal OR considers all process corners - what does omission of that factor really mean?). > I can put this whole description into the DT binding, if you like. Will leave it to Viresh and others to comment and guide, the terminology got my attention, since I almost got bit by a similar usage.. my 2 cents: I dont think that suffices unfortunately. what it lacks are the parameters of what that terminology really means, One actual production part that demonstrates this will probably help guide the discussion, I guess.. /me goes back to OPP hibernation > [1] https://developer.arm.com/documentation/den0056/b
On 29-10-20, 09:56, Lukasz Luba wrote: > There were discussions about Energy Model (EM), scale of values (mW or > abstract scale) and relation to EAS and IPA. You can find quite long > discussion below v2 [1] (there is also v3 send after agreement [2]). > We have in thermal DT binding: 'sustainable-power' expressed in mW, > which is used by IPA, but it would not support bogoWatts. Why so ? (I am sorry, can't dig into such long threads without knowing which message I am looking for :( ). Lets assume if that same property can be used for bogoWatts, will that be sufficient for you ? Or you will still need this patch set ? > The sustainable power is used for estimation of internal coefficients > (also for power budget), which I am trying to change to work with > 'abstract scale' [3][4]. > > This would allow to estimate sustainable power of the system based on > CPUs, GPU opp-sustainable points, where we don't have > 'sustainable-power' or devices using bogoWatts. Then maybe we should ahve sustainable-power in those cases too instead of adding a meaningless (IMHO) binding. Honestly speaking, as Nishanth said, there is nothing like a sustainable OPP in reality. Moreover, the DT needs to describe the hardware as it is (and in some cases the behavior of the firmware). And what you are trying to add here is none of them and so it should not go in DT as such. There are too many factors which play a part here, ambient temperature is one of the biggest ones, and the software needs to find the sustainable OPP by itself based on the current situation. So I don't really see a good reason why such a property should be added here. Coming to properties like suspend-opp, it made sense for some of the platforms as the last configured frequency of the CPU plays a part in deciding the power consumed by the SoC even when the system is suspended. And finding an optimal OPP (normally the lowest) there would make sense and so was that property added. -- viresh
On 10/30/20 8:29 AM, Viresh Kumar wrote: > On 29-10-20, 09:56, Lukasz Luba wrote: >> There were discussions about Energy Model (EM), scale of values (mW or >> abstract scale) and relation to EAS and IPA. You can find quite long >> discussion below v2 [1] (there is also v3 send after agreement [2]). >> We have in thermal DT binding: 'sustainable-power' expressed in mW, >> which is used by IPA, but it would not support bogoWatts. > > Why so ? (I am sorry, can't dig into such long threads without knowing > which message I am looking for :( ). Lets assume if that same property > can be used for bogoWatts, will that be sufficient for you ? Or you > will still need this patch set ? I had a patch for that, but I know Rob's opinion on this one [1] (which is below in that thread). > >> The sustainable power is used for estimation of internal coefficients >> (also for power budget), which I am trying to change to work with >> 'abstract scale' [3][4]. >> >> This would allow to estimate sustainable power of the system based on >> CPUs, GPU opp-sustainable points, where we don't have >> 'sustainable-power' or devices using bogoWatts. > > Then maybe we should ahve sustainable-power in those cases too instead > of adding a meaningless (IMHO) binding. How about dropping the DT binding, but just adding this new field into dev_pm_opp? There will be no DT parsing code, just the get/set functions, which will be used in SCMI patch 4/4 and in IPA? That would not require to change any DT bindings. > > Honestly speaking, as Nishanth said, there is nothing like a > sustainable OPP in reality. Moreover, the DT needs to describe the > hardware as it is (and in some cases the behavior of the firmware). > And what you are trying to add here is none of them and so it should > not go in DT as such. There are too many factors which play a part > here, ambient temperature is one of the biggest ones, and the software > needs to find the sustainable OPP by itself based on the current > situation. > > So I don't really see a good reason why such a property should be > added here. I see. Just for your information SCMI supports 'Sustained Performance' expressed in kHz. > > Coming to properties like suspend-opp, it made sense for some of the > platforms as the last configured frequency of the CPU plays a part in > deciding the power consumed by the SoC even when the system is > suspended. And finding an optimal OPP (normally the lowest) there > would make sense and so was that property added. > I also found that suspend-opp (83f8ca45afbf041e312909). I hope you wouldn't mind if I add this new field into dev_pm_opp (no DT support, just FW). [1] https://lore.kernel.org/lkml/20201002114426.31277-4-lukasz.luba@arm.com/
On 10/30/20 9:52 AM, Viresh Kumar wrote: > On 30-10-20, 09:19, Lukasz Luba wrote: >> How about dropping the DT binding, but just adding this new field into >> dev_pm_opp? There will be no DT parsing code, just the get/set >> functions, which will be used in SCMI patch 4/4 and in IPA? >> That would not require to change any DT bindings. > >> I see. Just for your information SCMI supports 'Sustained Performance' >> expressed in kHz. > > Even that doesn't sound great (but then I don't have any background of > why that was added there). The problem is not about how do we get this > data into the kernel (from DT or firmware), but why is it even > required. I really feel that software can find the sustainable OPP by > itself (which can keep changing). IPA tries to do that, even dynamically when e.g. GPU is supper busy in 3D games (~2000W) or almost idle showing 2D home screen. It tries to find highest 'sustainable' frequencies for the devices, at that various workloads and temp. But it needs some coefficients to start, which have big impact on the algorithm. It could slow down IPA a lot, when those coefficients are calculated based on lowest OPPs. > > About moving it into the OPP core, I am open to getting something > added there if it is really useful and if the OPP core is the best > suited place to keep such data. Though I am not sure of that for this > field right now. > > Is it ever going to be used by anyone else apart from IPA ? If not, > what about adding a helper in IPA to set sustainable-freq for a device > ? My backup plan was to add a flag into EM em_perf_state, extend SCMI perf exposing the 'sustained_freq_khz' to scmi-cpufreq, which would set that field after registering EM. IPA depends on EM, so should be OK. > > So only SCMI based platforms will be able to use this stuff ? That's I don't know who would also use it in future. I just presented you current user of this, as you asked. > very limited, isn't it ? I think we should still try to make it better > for everyone by making the software smarter. It has so much data, the > OPPs, the power it will consume (based on microvolt property?), the > heat we produce from that (from thermal framework), etc. Perhaps > building this information continuously at runtime based on when and > how we hit the trip points ? So we know which is the right frequency > where we can refrain from hitting the trip points. IPA works in this way. > > But may be I am asking too much :( > When you asked for user of this, I gave you instantly. This is one is more difficult. I am still not there with IPA tests in LISA. I have some out-of-tree kernel driver for testing, which also need polishing before can be used with LISA. Then proper workloads with results processing. EM for devfreq cooling devices. Then decent 'hot' board running preferably mainline kernel. What you requested is on my list, but it needs more work, which won't be ready over night. Regards, Lukasz
On 30-10-20, 10:56, Lukasz Luba wrote: > IPA tries to do that, even dynamically when e.g. GPU is supper busy > in 3D games (~2000W) or almost idle showing 2D home screen. > It tries to find highest 'sustainable' frequencies for the devices, > at that various workloads and temp. But it needs some coefficients to > start, which have big impact on the algorithm. It could slow down IPA a > lot, when those coefficients are calculated based on lowest OPPs. I see. So when you say it slows down IPA, what does that really mean ? IPA isn't performing that accurately during the initial period of booting (any time estimate here) ? Does it work fine after a time duration? Or will it suffer for ever ? And maybe you shouldn't start with the lowest OPPs while you calculate these coefficients dynamically ? Maybe start from the middle ? As the sustainable OPP would be something there only or maybe a bit higher only. But yeah, I don't have any idea about how those coefficients are calculated so this idea can be simply ignored as well :) > My backup plan was to add a flag into EM em_perf_state, extend SCMI perf > exposing the 'sustained_freq_khz' to scmi-cpufreq, which would set that > field after registering EM. IPA depends on EM, so should be OK. I think at this point (considering the limited number of users (only IPA) and providers (only SCMI)), it would be better that way only instead of updating the OPP framework. Of course we can revisit that if we ever feel that we need a better placeholder for it. > > So only SCMI based platforms will be able to use this stuff ? That's > > very limited, isn't it ? I think we should still try to make it better > > for everyone by making the software smarter. It has so much data, the > > OPPs, the power it will consume (based on microvolt property?), the > > heat we produce from that (from thermal framework), etc. Perhaps > > building this information continuously at runtime based on when and > > how we hit the trip points ? So we know which is the right frequency > > where we can refrain from hitting the trip points. > > IPA works in this way. Nice, that's what I thought as well but then got a bit confused with your patchset. > > But may be I am asking too much :( > > > > When you asked for user of this, I gave you instantly. This is one is > more difficult. I am still not there with IPA tests in LISA. I have some > out-of-tree kernel driver for testing, which also need polishing before > can be used with LISA. Then proper workloads with results processing. > EM for devfreq cooling devices. Then decent 'hot' board running > preferably mainline kernel. > What you requested is on my list, but it needs more work, which > won't be ready over night. I can understand what you are trying to do here. And this surely requires a lot of effort. -- viresh
On 10/30/20 11:17 AM, Viresh Kumar wrote: > On 30-10-20, 10:56, Lukasz Luba wrote: >> IPA tries to do that, even dynamically when e.g. GPU is supper busy >> in 3D games (~2000W) or almost idle showing 2D home screen. >> It tries to find highest 'sustainable' frequencies for the devices, >> at that various workloads and temp. But it needs some coefficients to >> start, which have big impact on the algorithm. It could slow down IPA a >> lot, when those coefficients are calculated based on lowest OPPs. > > I see. So when you say it slows down IPA, what does that really mean ? > IPA isn't performing that accurately during the initial period of > booting (any time estimate here) ? Does it work fine after a time > duration? Or will it suffer for ever ? The coefficients would stay 'forever', which determine the temp rising slope, until someone change them via sysfs (the: k_po, k_pu, k_i, sustainable_power). > > And maybe you shouldn't start with the lowest OPPs while you calculate > these coefficients dynamically ? Maybe start from the middle ? As the > sustainable OPP would be something there only or maybe a bit higher > only. But yeah, I don't have any idea about how those coefficients are > calculated so this idea can be simply ignored as well :) > >> My backup plan was to add a flag into EM em_perf_state, extend SCMI perf >> exposing the 'sustained_freq_khz' to scmi-cpufreq, which would set that >> field after registering EM. IPA depends on EM, so should be OK. > > I think at this point (considering the limited number of users (only > IPA) and providers (only SCMI)), it would be better that way only > instead of updating the OPP framework. Of course we can revisit that > if we ever feel that we need a better placeholder for it. OK, sounds good. > >>> So only SCMI based platforms will be able to use this stuff ? That's >>> very limited, isn't it ? I think we should still try to make it better >>> for everyone by making the software smarter. It has so much data, the >>> OPPs, the power it will consume (based on microvolt property?), the >>> heat we produce from that (from thermal framework), etc. Perhaps >>> building this information continuously at runtime based on when and >>> how we hit the trip points ? So we know which is the right frequency >>> where we can refrain from hitting the trip points. >> >> IPA works in this way. > > Nice, that's what I thought as well but then got a bit confused with > your patchset. > >>> But may be I am asking too much :( >>> >> >> When you asked for user of this, I gave you instantly. This is one is >> more difficult. I am still not there with IPA tests in LISA. I have some >> out-of-tree kernel driver for testing, which also need polishing before >> can be used with LISA. Then proper workloads with results processing. >> EM for devfreq cooling devices. Then decent 'hot' board running >> preferably mainline kernel. >> What you requested is on my list, but it needs more work, which >> won't be ready over night. > > I can understand what you are trying to do here. And this surely > requires a lot of effort. > Thank you Viresh for your opinion. I will take the EM approach, please ignore this patch set. Regards, Lukasz
Hi Quentin, On 10/30/20 11:47 AM, Quentin Perret wrote: > Hi Lukasz, > > On Wednesday 28 Oct 2020 at 14:08:45 (+0000), Lukasz Luba wrote: >> +unsigned long dev_pm_opp_get_sustainable_opp_freq(struct device *dev) >> +{ >> + struct opp_table *opp_table; >> + unsigned long freq = 0; >> + >> + opp_table = _find_opp_table(dev); >> + if (IS_ERR(opp_table)) >> + return 0; >> + >> + if (opp_table->sustainable_opp && opp_table->sustainable_opp->available) >> + freq = dev_pm_opp_get_freq(opp_table->sustainable_opp); >> + >> + dev_pm_opp_put_opp_table(opp_table); >> + >> + return freq; >> +} >> +EXPORT_SYMBOL_GPL(dev_pm_opp_get_sustainable_opp_freq); > > I'm guessing this is what IPA will use to find out what the sustainable > frequency is right? Yes, you are right. > > Is PM_OPP the right place for that? It feels odd IPA will get the EM > from one place, which includes the performance state, and the sustained > OPP from another. Should we move that to PM_EM instead? True, it might looks strange, but the OPP framework is available when we are adding the OPPs in scmi perf layer. The EM is available after we register the device, so at the end of scmi-cpufreq init. It would require a new scmi perf api function e.g. get_sustained_freq(), and a set/get function for EM, which is doable. I've discussed this approach to Viresh and he likes it better. I am happy that you are also suggesting the EM approach. I will send different patches for EM and SCMI to make that happen. Should I re-based them on top of the patch adding this milliwatts filed in EM [1]? Or do the opposite, changing the dependency order? Regards, Lukasz [1] https://lkml.org/lkml/2020/10/19/392 > > Thanks, > Quentin >
On Wed, Oct 28, 2020 at 02:08:44PM +0000, Lukasz Luba wrote: > Add opp-sustainable as an additional property in the OPP node to describe > the sustainable performance level of the device. This will help to > estimate the sustainable performance of the whole system. > > Signed-off-by: Lukasz Luba <lukasz.luba@arm.com> > --- > Documentation/devicetree/bindings/opp/opp.txt | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/Documentation/devicetree/bindings/opp/opp.txt b/Documentation/devicetree/bindings/opp/opp.txt > index 9847dfeeffcb..cd01028de305 100644 > --- a/Documentation/devicetree/bindings/opp/opp.txt > +++ b/Documentation/devicetree/bindings/opp/opp.txt > @@ -154,6 +154,10 @@ Optional properties: > - opp-suspend: Marks the OPP to be used during device suspend. If multiple OPPs > in the table have this, the OPP with highest opp-hz will be used. > > +- opp-sustainable: Marks the OPP as sustainable. This property can be used for > + estimating sustainable performance of the whole system. If multiple OPPs in > + the table have this, the OPP with highest opp-hz will be used. > + Isn't this just the inverse of the turbo? or boost? flag we already have? Couldn't this be learned? I ran at this frequency and then overheated. That could be dependent on ambient temperatures or dust build up on fans/heatsink. Rob
Hi Rob, On 10/30/20 7:34 PM, Rob Herring wrote: > On Wed, Oct 28, 2020 at 02:08:44PM +0000, Lukasz Luba wrote: >> Add opp-sustainable as an additional property in the OPP node to describe >> the sustainable performance level of the device. This will help to >> estimate the sustainable performance of the whole system. >> >> Signed-off-by: Lukasz Luba <lukasz.luba@arm.com> >> --- >> Documentation/devicetree/bindings/opp/opp.txt | 4 ++++ >> 1 file changed, 4 insertions(+) >> >> diff --git a/Documentation/devicetree/bindings/opp/opp.txt b/Documentation/devicetree/bindings/opp/opp.txt >> index 9847dfeeffcb..cd01028de305 100644 >> --- a/Documentation/devicetree/bindings/opp/opp.txt >> +++ b/Documentation/devicetree/bindings/opp/opp.txt >> @@ -154,6 +154,10 @@ Optional properties: >> - opp-suspend: Marks the OPP to be used during device suspend. If multiple OPPs >> in the table have this, the OPP with highest opp-hz will be used. >> >> +- opp-sustainable: Marks the OPP as sustainable. This property can be used for >> + estimating sustainable performance of the whole system. If multiple OPPs in >> + the table have this, the OPP with highest opp-hz will be used. >> + > > Isn't this just the inverse of the turbo? or boost? flag we already > have? True, it could be possible to use those flags. Then a function which returns one OPP below, would be 'sustainable'. I've already suggested to skip binding and only have get/set functions for OPP framework to mark 'sustainable' level, but Viresh is against. We have discussed this new opp sustainable with Viresh and since it would be only used by IPA but the definition is a bit blurry, we decided to drop this series. Maybe in future when there will be another user, we will come back to this. I am going to use Energy Model and mark the em_perf_state as sustained. > > Couldn't this be learned? I ran at this frequency and then overheated. > That could be dependent on ambient temperatures or dust build up on > fans/heatsink. Yes it could be learned. IPA tries to do that for different workloads, but it needs some starting conditions (coefficients and sustainable power) to converge in short time. Sustainable power is currently estimated based on lowest OPP. With that new 'sustainable' flag, it would be much easier for IPA to converge. Regards, Lukasz > > Rob >