mbox series

[v2,0/4] Apple SoC cpufreq driver

Message ID 20220504075153.185208-1-marcan@marcan.st
Headers show
Series Apple SoC cpufreq driver | expand

Message

Hector Martin May 4, 2022, 7:51 a.m. UTC
Hi folks,

Here's a second take on the cpufreq driver for Apple SoCs. This is a
complete rewrite using a stand-alone cpufreq driver instead of using the
cpufreq-dt infrastructure.

Since v1 we ran some experiments on the memory controller performance
switching and it turns out it doesn't make a huge difference, so it
makes sense to punt that feature to the future (perhaps once a proper
memory controller driver exists for other reasons, e.g. for error
handling).

One advantage of having a standalone cpufreq driver is that we can
support fast switching. This also means any future interaction with
the memory controller will probably use some bespoke mechanism instead
of the genpd infrastructure, so we can keep the fast path without
allowing sleeps/etc.

The driver is based on scpi-cpufreq.c, with some bits (e.g. the
apple,freq-domain stuff) inspired by how cpufreq-qcom-hw does it.
I'm not sure if that particular property should be described
in a binding, since it goes in the cpu nodes (qcom doesn't have it
anywhere...).

Changes since v1:
- Complete rewrite
- Reports current frequency to userspace properly (incl. if different
  from requested due to hardware constraints)
- Supports fast switching
- MCC latency control stuff no longer included, punted for later
- Supports exposing higher states as turbo states

Hector Martin (4):
  MAINTAINERS: Add entries for Apple SoC cpufreq driver
  dt-bindings: cpufreq: apple,soc-cpufreq: Add binding for Apple SoC
    cpufreq
  cpufreq: apple-soc: Add new driver to control Apple SoC CPU P-states
  arm64: dts: apple: Add CPU topology & cpufreq nodes for t8103

 .../bindings/cpufreq/apple,soc-cpufreq.yaml   | 121 +++++++
 MAINTAINERS                                   |   2 +
 arch/arm64/boot/dts/apple/t8103.dtsi          | 203 ++++++++++-
 drivers/cpufreq/Kconfig.arm                   |   9 +
 drivers/cpufreq/Makefile                      |   1 +
 drivers/cpufreq/apple-soc-cpufreq.c           | 330 ++++++++++++++++++
 drivers/cpufreq/cpufreq-dt-platdev.c          |   2 +
 7 files changed, 658 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
 create mode 100644 drivers/cpufreq/apple-soc-cpufreq.c

Comments

Krzysztof Kozlowski May 5, 2022, 8:42 a.m. UTC | #1
On 04/05/2022 16:52, Hector Martin wrote:
> On 04/05/2022 19.17, Viresh Kumar wrote:
>> On 04-05-22, 16:51, Hector Martin wrote:
>>> Splitting this commit, as usual, to facilitate merges via the SoC tree.
>>>
>>> Signed-off-by: Hector Martin <marcan@marcan.st>
>>> ---
>>>  MAINTAINERS | 2 ++
>>>  1 file changed, 2 insertions(+)
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index edc96cdb85e8..39bfa478fe55 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -1835,6 +1835,7 @@ T:	git https://github.com/AsahiLinux/linux.git
>>>  F:	Documentation/devicetree/bindings/arm/apple.yaml
>>>  F:	Documentation/devicetree/bindings/arm/apple/*
>>>  F:	Documentation/devicetree/bindings/clock/apple,nco.yaml
>>> +F:	Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
>>>  F:	Documentation/devicetree/bindings/i2c/apple,i2c.yaml
>>>  F:	Documentation/devicetree/bindings/interrupt-controller/apple,*
>>>  F:	Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml
>>> @@ -1844,6 +1845,7 @@ F:	Documentation/devicetree/bindings/power/apple*
>>>  F:	Documentation/devicetree/bindings/watchdog/apple,wdt.yaml
>>>  F:	arch/arm64/boot/dts/apple/
>>>  F:	drivers/clk/clk-apple-nco.c
>>> +F:	drivers/cpufreq/apple-soc-cpufreq.c
>>>  F:	drivers/i2c/busses/i2c-pasemi-core.c
>>>  F:	drivers/i2c/busses/i2c-pasemi-platform.c
>>>  F:	drivers/irqchip/irq-apple-aic.c
>>
>> This should be the last patch instead, or should at least be added
>> after the files are merged first. If someone checks out at this
>> commit, the files won't be available but still linked here.
> 
> Isn't that backwards? 

No, because we have tools for checking valid paths (in some places), so
when using that tool, the history is not bisectable.

> If someone touches the files, we want them to be
> able to get_maintainer.pl, so the MAINTAINERS entries should come first.
> It doesn't really cause any issues if there are entries that point at
> files that don't exist yet, right?

It hurts any current or future tools checking for valid paths.

> 
> Though this is mostly a moot point because the purpose of splitting this
> out is so we can merge this one patch through the SoC tree, at which
> point the ordering isn't guaranteed (unless the whole series goes
> through SoC). 

Just add each path change to respective commit adding that file. It
should not be a separate commit, at first place.

Separate commits are for adding entire Maintainers entry.

Best regards,
Krzysztof
Krzysztof Kozlowski May 5, 2022, 8:43 a.m. UTC | #2
On 04/05/2022 09:51, Hector Martin wrote:
> This binding represents the cpufreq/DVFS hardware present in Apple SoCs.
> The hardware has an independent controller per CPU cluster, but we
> represent them as a single cpufreq node since there can only be one
> systemwide cpufreq device (and since in the future, interactions with
> memory controller performance states will also involve cooperation
> between multiple frequency domains).
> 
> Signed-off-by: Hector Martin <marcan@marcan.st>
> ---
>  .../bindings/cpufreq/apple,soc-cpufreq.yaml   | 121 ++++++++++++++++++
>  1 file changed, 121 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
> 
> diff --git a/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml b/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
> new file mode 100644
> index 000000000000..f398c1bd5de5
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
> @@ -0,0 +1,121 @@
> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/cpufreq/apple,soc-cpufreq.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Apple SoC cpufreq device
> +
> +maintainers:
> +  - Hector Martin <marcan@marcan.st>
> +
> +description: |
> +  Apple SoCs (e.g. M1) have a per-cpu-cluster DVFS controller that is part of
> +  the cluster management register block. This binding uses the standard
> +  operating-points-v2 table to define the CPU performance states, with the
> +  opp-level property specifying the hardware p-state index for that level.
> +
> +properties:
> +  compatible:
> +    items:
> +      - enum:
> +          - apple,t8103-soc-cpufreq
> +          - apple,t6000-soc-cpufreq
> +      - const: apple,soc-cpufreq
> +
> +  reg:
> +    minItems: 1
> +    maxItems: 6

Is the number of clusters fixed for t8103 and t6000? Are these
compatibles strictly related to some specific M1 SoC? If yes, then you
should have constraints in allOf:if:then.


Best regards,
Krzysztof
Viresh Kumar May 5, 2022, 8:44 a.m. UTC | #3
On 05-05-22, 10:42, Krzysztof Kozlowski wrote:
> On 04/05/2022 16:52, Hector Martin wrote:
> > On 04/05/2022 19.17, Viresh Kumar wrote:
> >> This should be the last patch instead, or should at least be added
> >> after the files are merged first. If someone checks out at this
> >> commit, the files won't be available but still linked here.
> > 
> > Isn't that backwards? 
> 
> No, because we have tools for checking valid paths (in some places), so
> when using that tool, the history is not bisectable.
> 
> > If someone touches the files, we want them to be
> > able to get_maintainer.pl, so the MAINTAINERS entries should come first.
> > It doesn't really cause any issues if there are entries that point at
> > files that don't exist yet, right?
> 
> It hurts any current or future tools checking for valid paths.
> 
> > 
> > Though this is mostly a moot point because the purpose of splitting this
> > out is so we can merge this one patch through the SoC tree, at which
> > point the ordering isn't guaranteed (unless the whole series goes
> > through SoC). 
> 
> Just add each path change to respective commit adding that file. It
> should not be a separate commit, at first place.
> 
> Separate commits are for adding entire Maintainers entry.

And there is no need for this patch to go via SoC tree, we can handle
minor conflicts later on if required.
Hector Martin May 5, 2022, 11:06 a.m. UTC | #4
On 05/05/2022 17.43, Krzysztof Kozlowski wrote:
> On 04/05/2022 09:51, Hector Martin wrote:
>> This binding represents the cpufreq/DVFS hardware present in Apple SoCs.
>> The hardware has an independent controller per CPU cluster, but we
>> represent them as a single cpufreq node since there can only be one
>> systemwide cpufreq device (and since in the future, interactions with
>> memory controller performance states will also involve cooperation
>> between multiple frequency domains).
>>
>> Signed-off-by: Hector Martin <marcan@marcan.st>
>> ---
>>  .../bindings/cpufreq/apple,soc-cpufreq.yaml   | 121 ++++++++++++++++++
>>  1 file changed, 121 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
>>
>> diff --git a/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml b/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
>> new file mode 100644
>> index 000000000000..f398c1bd5de5
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
>> @@ -0,0 +1,121 @@
>> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/cpufreq/apple,soc-cpufreq.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Apple SoC cpufreq device
>> +
>> +maintainers:
>> +  - Hector Martin <marcan@marcan.st>
>> +
>> +description: |
>> +  Apple SoCs (e.g. M1) have a per-cpu-cluster DVFS controller that is part of
>> +  the cluster management register block. This binding uses the standard
>> +  operating-points-v2 table to define the CPU performance states, with the
>> +  opp-level property specifying the hardware p-state index for that level.
>> +
>> +properties:
>> +  compatible:
>> +    items:
>> +      - enum:
>> +          - apple,t8103-soc-cpufreq
>> +          - apple,t6000-soc-cpufreq
>> +      - const: apple,soc-cpufreq
>> +
>> +  reg:
>> +    minItems: 1
>> +    maxItems: 6
> 
> Is the number of clusters fixed for t8103 and t6000? Are these
> compatibles strictly related to some specific M1 SoC? If yes, then you
> should have constraints in allOf:if:then.

No, t6000 includes t6002 which is a 2-die version and has 6 clusters,
t6001 and t6000 have 3. t8103 always has 2, but it's conceivable that
compatible could be used for other chips within the same generation with
a different number.

The general idea for these compats is as a fallback in case we need to
quirk something for individual SoCs or start using registers which
changed for some reason, but right now they are not used. But I think
given how closely related t6000/t6001/t6002 are (t6002 is literally two
t6001 dies, and t6000 is almost identical to t6001 with a chunk
missing), I think spelling those out as 3 separate compatibles is
overkill. In general we treat those 3 SoCs as identical in terms of
compatibles, but the one-cpufreq-node limitation means t6002 does have
twice the clusters.

It's also conceivable that Apple could start releasing CPUs with entire
clusters fused off, so then our bootloader would have to start mutating
the reg entry to represent that.

I can add constraints to express some of this if you really want me to,
but I'm not sure it's that useful. The way I see it, the compatibles
really *mean* "this soc has t8103 clusters" or "this soc has t6000
clusters" and the number of clusters is arbitrary and the driver will
never care about that. Honestly I'd rather have separate cpufreq nodes
for each cluster, but nobody else does it like that and there's only one
driver instance, so that gets complicated...
Manivannan Sadhasivam May 8, 2022, 7:16 a.m. UTC | #5
On Wed, May 04, 2022 at 09:30:26PM +0530, Manivannan Sadhasivam wrote:
> Hi Viresh,
> 
> On Wed, May 04, 2022 at 03:57:45PM +0530, Viresh Kumar wrote:
> > On 04-05-22, 16:51, Hector Martin wrote:
> > > Hi folks,
> > > 
> > > Here's a second take on the cpufreq driver for Apple SoCs. This is a
> > > complete rewrite using a stand-alone cpufreq driver instead of using the
> > > cpufreq-dt infrastructure.
> > > 
> > > Since v1 we ran some experiments on the memory controller performance
> > > switching and it turns out it doesn't make a huge difference, so it
> > > makes sense to punt that feature to the future (perhaps once a proper
> > > memory controller driver exists for other reasons, e.g. for error
> > > handling).
> > > 
> > > One advantage of having a standalone cpufreq driver is that we can
> > > support fast switching. This also means any future interaction with
> > > the memory controller will probably use some bespoke mechanism instead
> > > of the genpd infrastructure, so we can keep the fast path without
> > > allowing sleeps/etc.
> > > 
> > > The driver is based on scpi-cpufreq.c, with some bits (e.g. the
> > > apple,freq-domain stuff) inspired by how cpufreq-qcom-hw does it.
> > > I'm not sure if that particular property should be described
> > > in a binding, since it goes in the cpu nodes (qcom doesn't have it
> > > anywhere...).
> > 
> > Hi Mani,
> > 
> > I can see that Rob asked you to add this somewhere, maybe in arm/cpu
> > stuff, but I don't think you ever sent a patch with that. What
> > happened ?
> > 
> > https://lore.kernel.org/lkml/20201013171800.GA3716411@bogus/
> > 
> 
> Oops. Looks like that one slipped through the cracks. I did add it to my todo
> list for qcom-cpufreq but missed it completely.
> 
> I will look into it.
> 

So I did send a patch for adding the qcom specific property [1], but Rob asked
for a common one and that's where it got lost.

But in the last revision [2], you've asked for converting the qcom cpufreq
driver to support the generic performance domains instead. For maintaining
compatibility with the old dts files, we need to support the qcom specific
property as well.

I will send a series for that.

Thanks,
Mani

[1] https://patchwork.kernel.org/project/linux-pm/patch/20201020153944.18047-1-manivannan.sadhasivam@linaro.org/#23718065
[2] https://patchwork.kernel.org/project/linux-pm/patch/20210701105730.322718-4-angelogioacchino.delregno@somainline.org/ 
> Thanks,
> Mani
> 
> > -- 
> > viresh
Rob Herring (Arm) May 16, 2022, 10:49 p.m. UTC | #6
On Wed, May 04, 2022 at 04:51:51PM +0900, Hector Martin wrote:
> This binding represents the cpufreq/DVFS hardware present in Apple SoCs.
> The hardware has an independent controller per CPU cluster, but we
> represent them as a single cpufreq node since there can only be one
> systemwide cpufreq device (and since in the future, interactions with
> memory controller performance states will also involve cooperation
> between multiple frequency domains).
> 
> Signed-off-by: Hector Martin <marcan@marcan.st>
> ---
>  .../bindings/cpufreq/apple,soc-cpufreq.yaml   | 121 ++++++++++++++++++
>  1 file changed, 121 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
> 
> diff --git a/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml b/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
> new file mode 100644
> index 000000000000..f398c1bd5de5
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/cpufreq/apple,soc-cpufreq.yaml
> @@ -0,0 +1,121 @@
> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/cpufreq/apple,soc-cpufreq.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Apple SoC cpufreq device
> +
> +maintainers:
> +  - Hector Martin <marcan@marcan.st>
> +
> +description: |
> +  Apple SoCs (e.g. M1) have a per-cpu-cluster DVFS controller that is part of
> +  the cluster management register block. This binding uses the standard
> +  operating-points-v2 table to define the CPU performance states, with the
> +  opp-level property specifying the hardware p-state index for that level.
> +
> +properties:
> +  compatible:
> +    items:
> +      - enum:
> +          - apple,t8103-soc-cpufreq
> +          - apple,t6000-soc-cpufreq
> +      - const: apple,soc-cpufreq
> +
> +  reg:
> +    minItems: 1
> +    maxItems: 6
> +    description: One register region per CPU cluster DVFS controller
> +
> +  reg-names:
> +    minItems: 1
> +    items:
> +      - const: cluster0
> +      - const: cluster1
> +      - const: cluster2
> +      - const: cluster3
> +      - const: cluster4
> +      - const: cluster5
> +
> +  '#freq-domain-cells':
> +    const: 1

Copied QCom it seems. Use 'performance-domains' which is the common 
binding.

> +
> +required:
> +  - compatible
> +  - reg
> +  - reg-names
> +  - '#freq-domain-cells'
> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    // This example shows a single CPU per domain and 2 domains,
> +    // with two p-states per domain.
> +    // Shipping hardware has 2-4 CPUs per domain and 2-6 domains.
> +    cpus {
> +      #address-cells = <2>;
> +      #size-cells = <0>;
> +
> +      cpu@0 {
> +        compatible = "apple,icestorm";
> +        device_type = "cpu";
> +        reg = <0x0 0x0>;
> +        operating-points-v2 = <&ecluster_opp>;
> +        apple,freq-domain = <&cpufreq_hw 0>;
> +      };
> +
> +      cpu@10100 {
> +        compatible = "apple,firestorm";
> +        device_type = "cpu";
> +        reg = <0x0 0x10100>;
> +        operating-points-v2 = <&pcluster_opp>;
> +        apple,freq-domain = <&cpufreq_hw 1>;
> +      };
> +    };
> +
> +    ecluster_opp: opp-table-0 {
> +      compatible = "operating-points-v2";
> +      opp-shared;
> +
> +      opp01 {
> +        opp-hz = /bits/ 64 <600000000>;
> +        opp-level = <1>;
> +        clock-latency-ns = <7500>;
> +      };
> +      opp02 {
> +        opp-hz = /bits/ 64 <972000000>;
> +        opp-level = <2>;
> +        clock-latency-ns = <22000>;
> +      };
> +    };
> +
> +    pcluster_opp: opp-table-1 {
> +      compatible = "operating-points-v2";
> +      opp-shared;
> +
> +      opp01 {
> +        opp-hz = /bits/ 64 <600000000>;
> +        opp-level = <1>;
> +        clock-latency-ns = <8000>;
> +      };
> +      opp02 {
> +        opp-hz = /bits/ 64 <828000000>;
> +        opp-level = <2>;
> +        clock-latency-ns = <19000>;
> +      };
> +    };
> +
> +    soc {
> +      #address-cells = <2>;
> +      #size-cells = <2>;
> +
> +      cpufreq_hw: cpufreq@210e20000 {
> +        compatible = "apple,t8103-soc-cpufreq", "apple,soc-cpufreq";
> +        reg = <0x2 0x10e20000 0 0x1000>,
> +              <0x2 0x11e20000 0 0x1000>;
> +        reg-names = "cluster0", "cluster1";
> +        #freq-domain-cells = <1>;
> +      };
> +    };
> -- 
> 2.35.1
> 
>