mbox series

[net-next,0/2] dt-bindings: define property describing supported ethernet PHY modes

Message ID 20210324103556.11338-1-kabel@kernel.org
Headers show
Series dt-bindings: define property describing supported ethernet PHY modes | expand

Message

Marek Behún March 24, 2021, 10:35 a.m. UTC
Hello,

the Marvell Alaska PHYs (88X3310, 88E2110) can, depending on their
configuration, change the PHY mode to the MAC, depending on the
copper/fiber speed.

The 88X3310, for example, can be configured (via MACTYPE register)
so that it communicates with the MAC in sgmii for 10/100/1000mbps,
2500base-x for 2500mbps, 5gbase-r for 5gbps and either 10gbase-r,
xaui or rxaui for 10gbps. Or the PHY may communicate with the MAC
in usxgmii, or one of the 10gbase-r, rxaui or xaui modes with rate
matching.

So for the 10gbps mode we have options 10gbase-r, xaui, rxaui and
usxgmii. The MAC can support some of these modes, and if it does more
than one, we need to know which one to use. Not all of these modes
must necessarily be supported by the board (xaui required wiring for
4 SerDes lanes, for example, and 10gbase-r requires wiring capable
of transmitting at 10.3125 GBd).

The MACTYPE is upon HW reset configured by strapping pins - so the
board should have a correct mode configured after HW reset.

One problem with this is that some boards configure the MACTYPE to
a rate matching mode, which, according to the errata, is broken in
some situations (it does not work for 10/100/1000, for example).

Another problem is that if lower modes are supported, we should
maybe use them in order to save power.

But for this we need to know which phy-modes are supported on the
board.

This series adds documentation for a new ethernet PHY property,
called `supported-mac-connection-types`.

When this property is present for a PHY node, only the phy-modes
listed in this property should be considered to be functional on
the board.

The second patch adds binding for this property.

The first patch does some YAML magic in ethernet-controller.yaml
in order to be able to reuse the PHY modes enum, so that the same
list does not have to be defined twice.

Marek

Marek Behún (2):
  dt-bindings: ethernet-controller: create a type for PHY interface
    modes
  dt-bindings: ethernet-phy: define `supported-mac-connection-types`
    property

 .../bindings/net/ethernet-controller.yaml     | 89 ++++++++++---------
 .../devicetree/bindings/net/ethernet-phy.yaml | 18 ++++
 2 files changed, 66 insertions(+), 41 deletions(-)

Comments

Rob Herring March 24, 2021, 8:07 p.m. UTC | #1
On Wed, Mar 24, 2021 at 11:35:55AM +0100, Marek Behún wrote:
> In order to be able to define a property describing an array of PHY
> interface modes, we need to change the current scalar
> `phy-connection-type`, which lists the possible PHY interface modes, to
> an array of length 1 (otherwise we would need to define the same list at
> two different places).
> 
> Moreover Rob Herring says that we cannot reuse the values of a property;
> we need to $ref a type.
> 
> Move the definition of possible PHY interface modes from the
> `phy-connection-type` property to an array type definition
> `phy-connection-type-array`, and simply reference this type in the
> original property.

Why not just extend phy-connection-type to support more than 1 entry?
Marek Behún March 24, 2021, 8:59 p.m. UTC | #2
On Wed, 24 Mar 2021 14:07:06 -0600
Rob Herring <robh@kernel.org> wrote:

> On Wed, Mar 24, 2021 at 11:35:55AM +0100, Marek Behún wrote:
> > In order to be able to define a property describing an array of PHY
> > interface modes, we need to change the current scalar
> > `phy-connection-type`, which lists the possible PHY interface modes, to
> > an array of length 1 (otherwise we would need to define the same list at
> > two different places).
> > 
> > Moreover Rob Herring says that we cannot reuse the values of a property;
> > we need to $ref a type.
> > 
> > Move the definition of possible PHY interface modes from the
> > `phy-connection-type` property to an array type definition
> > `phy-connection-type-array`, and simply reference this type in the
> > original property.  
> 
> Why not just extend phy-connection-type to support more than 1 entry?

Hmm, that would be even better, although it would complicate the
Russell's marvell10g patches a little if we want the code to be
backward compatible with older device trees.

I will look into this.

Marek
Marek Behún March 24, 2021, 11 p.m. UTC | #3
On Wed, 24 Mar 2021 14:19:28 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

> > Another problem is that if lower modes are supported, we should
> > maybe use them in order to save power.  
> 
> That is an interesting proposal but if you want it to be truly valuable,
> does not that mean that an user ought to be able to switch between any
> of the supported PHY <=> MAC interfaces at runtime, and then within
> those interfaces to the speeds that yield the best power savings?

If the code determines that there are multiple working configurations,
it theoretically could allow the user to switch between them.

My idea was that this should be done by kernel, though.

But power saving is not the main problem I am trying to solve.
What I am trying to solve is that if a board does not support all modes
supported by the MAC and PHY, because they are not wired or something,
we need to know about that so that we can select the correct mode for
PHYs that change this mode at runtime.

> > 
> > But for this we need to know which phy-modes are supported on the
> > board.
> > 
> > This series adds documentation for a new ethernet PHY property,
> > called `supported-mac-connection-types`.  
> 
> That naming does not quite make sense to me, if we want to describe the
> MAC supported connection types, then those would naturally be within the
> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
> capable, then we should be dropping "mac" from the property name not to
> create confusion.

I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
I guess I put "mac" there so that if in the future we wanted to specify
supported modes for the fiber side, we could add
`supported-fiber-connection-types`.

But otherwise it does not matter where this property is. Rob Herring
says that maybe we don't need a new property at all. We can reuse
phy-mode property of the MAC, and enumerate all supported modes there.

> 
> > 
> > When this property is present for a PHY node, only the phy-modes
> > listed in this property should be considered to be functional on
> > the board.  
> 
> Can you post the code that is going to utilize these properties so we
> have a clearer picture of how and what you need to solve?

I am still working on this, so the repo may change, but look at
https://git.kernel.org/pub/scm/linux/kernel/git/kabel/linux.git/log/?h=phy-supported-interfaces
at the last three patches.
Florian Fainelli March 24, 2021, 11:16 p.m. UTC | #4
On 3/24/2021 4:00 PM, Marek Behún wrote:
> On Wed, 24 Mar 2021 14:19:28 -0700
> Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
>>> Another problem is that if lower modes are supported, we should
>>> maybe use them in order to save power.  
>>
>> That is an interesting proposal but if you want it to be truly valuable,
>> does not that mean that an user ought to be able to switch between any
>> of the supported PHY <=> MAC interfaces at runtime, and then within
>> those interfaces to the speeds that yield the best power savings?
> 
> If the code determines that there are multiple working configurations,
> it theoretically could allow the user to switch between them.
> 
> My idea was that this should be done by kernel, though.
> 
> But power saving is not the main problem I am trying to solve.
> What I am trying to solve is that if a board does not support all modes
> supported by the MAC and PHY, because they are not wired or something,
> we need to know about that so that we can select the correct mode for
> PHYs that change this mode at runtime.

OK so the runtime part comes from plugging in various SFP modules into a
cage but other than that, for a "fixed" link such as a SFF or a soldered
down PHY, do we agree that there would be no runtime changing of the
'phy-mode'?

What I am trying to understand is why this needs to be added to the
Device Tree as opposed to a bitmask within the PHY driver that indicates
the various interface mode capabilities which, looking at the code you
shared below, is how you make decisions ultimately.

> 
>>>
>>> But for this we need to know which phy-modes are supported on the
>>> board.
>>>
>>> This series adds documentation for a new ethernet PHY property,
>>> called `supported-mac-connection-types`.  
>>
>> That naming does not quite make sense to me, if we want to describe the
>> MAC supported connection types, then those would naturally be within the
>> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
>> capable, then we should be dropping "mac" from the property name not to
>> create confusion.
> 
> I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
> host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
> I guess I put "mac" there so that if in the future we wanted to specify
> supported modes for the fiber side, we could add
> `supported-fiber-connection-types`.

You would traditionally find the words "line side" (copper, optical,
etc.) and "MAC side" being used in datasheets, maybe you can use a
similar naming here?

> 
> But otherwise it does not matter where this property is. Rob Herring
> says that maybe we don't need a new property at all. We can reuse
> phy-mode property of the MAC, and enumerate all supported modes there.
> 
>>
>>>
>>> When this property is present for a PHY node, only the phy-modes
>>> listed in this property should be considered to be functional on
>>> the board.  
>>
>> Can you post the code that is going to utilize these properties so we
>> have a clearer picture of how and what you need to solve?
> 
> I am still working on this, so the repo may change, but look at
> https://git.kernel.org/pub/scm/linux/kernel/git/kabel/linux.git/log/?h=phy-supported-interfaces
> at the last three patches.
>
Marek Behún March 24, 2021, 11:45 p.m. UTC | #5
On Wed, 24 Mar 2021 16:16:41 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

> On 3/24/2021 4:00 PM, Marek Behún wrote:
> > On Wed, 24 Mar 2021 14:19:28 -0700
> > Florian Fainelli <f.fainelli@gmail.com> wrote:
> >   
> >>> Another problem is that if lower modes are supported, we should
> >>> maybe use them in order to save power.    
> >>
> >> That is an interesting proposal but if you want it to be truly valuable,
> >> does not that mean that an user ought to be able to switch between any
> >> of the supported PHY <=> MAC interfaces at runtime, and then within
> >> those interfaces to the speeds that yield the best power savings?  
> > 
> > If the code determines that there are multiple working configurations,
> > it theoretically could allow the user to switch between them.
> > 
> > My idea was that this should be done by kernel, though.
> > 
> > But power saving is not the main problem I am trying to solve.
> > What I am trying to solve is that if a board does not support all modes
> > supported by the MAC and PHY, because they are not wired or something,
> > we need to know about that so that we can select the correct mode for
> > PHYs that change this mode at runtime.  
> 
> OK so the runtime part comes from plugging in various SFP modules into a
> cage but other than that, for a "fixed" link such as a SFF or a soldered
> down PHY, do we agree that there would be no runtime changing of the
> 'phy-mode'?

No, we do not. The PHY can be configured (by strapping pins or by
sw) to change phy-mode depending on the autonegotiated copper speed.

So if you plug in an ethernet cable where on the otherside is only 1g
capable device, the PHY will change mode to sgmii. But if you then plug
a 5g capable device, the PHY will change mode to 5gbase-r.

This happens if the PHY is configured into one of these changing
configurations. It can also be configured to USXGMII, or 10GBASER with
rate matching.

Not many MACs in kernel support USXGMII currently.

And if you use rate matching mode, and the copper side is
linked in lower speed (2.5g for example), and the MAC will start
sending too many packets, the internal buffer in the PHY is only 16 KB,
so it will fill up quickly. So you need pause frames support. But this
is broken for speeds <= 1g, according to erratum.

So you really want to change modes. The rate matching mode is
basically useless.

> 
> What I am trying to understand is why this needs to be added to the
> Device Tree as opposed to a bitmask within the PHY driver that indicates
> the various interface mode capabilities which, looking at the code you
> shared below, is how you make decisions ultimately.

Because someone can create a board with a SOC where MAC is capable of
all of the following modes: 10gbase-r, xaui, rxaui, 5gbase-r,
2.5gbase-x, sgmii.

And use Marvell 88X3310 PHY to translate to copper.

But only wire the PHY to the MAC with one SerDes lane. So for 10g,
10gbase-r mode must be used, xaui and rxaui cannot.
Or wire the PHY to the MAC with 2 SerDes lanes, but both lanes capable
only of 6 GHz freq. So for 10g, rxaui must be used.

And then make the mistake of wiring the strapping pins to the
rate-matching mode, which is useless.

So we need to know which modes are supported if we want to change the
configuration to a working one.

> >   
> >>>
> >>> But for this we need to know which phy-modes are supported on the
> >>> board.
> >>>
> >>> This series adds documentation for a new ethernet PHY property,
> >>> called `supported-mac-connection-types`.    
> >>
> >> That naming does not quite make sense to me, if we want to describe the
> >> MAC supported connection types, then those would naturally be within the
> >> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
> >> capable, then we should be dropping "mac" from the property name not to
> >> create confusion.  
> > 
> > I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
> > host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
> > I guess I put "mac" there so that if in the future we wanted to specify
> > supported modes for the fiber side, we could add
> > `supported-fiber-connection-types`.  
> 
> You would traditionally find the words "line side" (copper, optical,
> etc.) and "MAC side" being used in datasheets, maybe you can use a
> similar naming here?

So
  supported-connection-types-mac-side
  supported-connection-types-line-side
or maybe media-side?

I am still exploring whether this could be simply defined in the
ethernet controllers `phy-mode` property, as Rob Herring says. It would
be simpler...

Marek
Florian Fainelli March 25, 2021, 12:11 a.m. UTC | #6
On 3/24/2021 4:45 PM, Marek Behún wrote:
> On Wed, 24 Mar 2021 16:16:41 -0700
> Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
>> On 3/24/2021 4:00 PM, Marek Behún wrote:
>>> On Wed, 24 Mar 2021 14:19:28 -0700
>>> Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>   
>>>>> Another problem is that if lower modes are supported, we should
>>>>> maybe use them in order to save power.    
>>>>
>>>> That is an interesting proposal but if you want it to be truly valuable,
>>>> does not that mean that an user ought to be able to switch between any
>>>> of the supported PHY <=> MAC interfaces at runtime, and then within
>>>> those interfaces to the speeds that yield the best power savings?  
>>>
>>> If the code determines that there are multiple working configurations,
>>> it theoretically could allow the user to switch between them.
>>>
>>> My idea was that this should be done by kernel, though.
>>>
>>> But power saving is not the main problem I am trying to solve.
>>> What I am trying to solve is that if a board does not support all modes
>>> supported by the MAC and PHY, because they are not wired or something,
>>> we need to know about that so that we can select the correct mode for
>>> PHYs that change this mode at runtime.  
>>
>> OK so the runtime part comes from plugging in various SFP modules into a
>> cage but other than that, for a "fixed" link such as a SFF or a soldered
>> down PHY, do we agree that there would be no runtime changing of the
>> 'phy-mode'?
> 
> No, we do not. The PHY can be configured (by strapping pins or by
> sw) to change phy-mode depending on the autonegotiated copper speed.
> 
> So if you plug in an ethernet cable where on the otherside is only 1g
> capable device, the PHY will change mode to sgmii. But if you then plug
> a 5g capable device, the PHY will change mode to 5gbase-r.
> 
> This happens if the PHY is configured into one of these changing
> configurations. It can also be configured to USXGMII, or 10GBASER with
> rate matching.
> 
> Not many MACs in kernel support USXGMII currently.
> 
> And if you use rate matching mode, and the copper side is
> linked in lower speed (2.5g for example), and the MAC will start
> sending too many packets, the internal buffer in the PHY is only 16 KB,
> so it will fill up quickly. So you need pause frames support. But this
> is broken for speeds <= 1g, according to erratum.
> 
> So you really want to change modes. The rate matching mode is
> basically useless.

OK, so whenever there is a link change you are presumably reading the
mode in which the PHY has been reconfigured to, asking the MAC to
configured itself appropriately based on that, and if there is no
intersection, error out?

> 
>>
>> What I am trying to understand is why this needs to be added to the
>> Device Tree as opposed to a bitmask within the PHY driver that indicates
>> the various interface mode capabilities which, looking at the code you
>> shared below, is how you make decisions ultimately.
> 
> Because someone can create a board with a SOC where MAC is capable of
> all of the following modes: 10gbase-r, xaui, rxaui, 5gbase-r,
> 2.5gbase-x, sgmii.
> 
> And use Marvell 88X3310 PHY to translate to copper.
> 
> But only wire the PHY to the MAC with one SerDes lane. So for 10g,
> 10gbase-r mode must be used, xaui and rxaui cannot.
> Or wire the PHY to the MAC with 2 SerDes lanes, but both lanes capable
> only of 6 GHz freq. So for 10g, rxaui must be used.
> 
> And then make the mistake of wiring the strapping pins to the
> rate-matching mode, which is useless.
> 
> So we need to know which modes are supported if we want to change the
> configuration to a working one.

OK, so you need to know the PCB limitations which would be coming via
Device Tree and what mode the PHY has been configured into at the time
you attach/connect to the PHY which you could read from the device itself.

> 
>>>   
>>>>>
>>>>> But for this we need to know which phy-modes are supported on the
>>>>> board.
>>>>>
>>>>> This series adds documentation for a new ethernet PHY property,
>>>>> called `supported-mac-connection-types`.    
>>>>
>>>> That naming does not quite make sense to me, if we want to describe the
>>>> MAC supported connection types, then those would naturally be within the
>>>> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
>>>> capable, then we should be dropping "mac" from the property name not to
>>>> create confusion.  
>>>
>>> I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
>>> host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
>>> I guess I put "mac" there so that if in the future we wanted to specify
>>> supported modes for the fiber side, we could add
>>> `supported-fiber-connection-types`.  
>>
>> You would traditionally find the words "line side" (copper, optical,
>> etc.) and "MAC side" being used in datasheets, maybe you can use a
>> similar naming here?
> 
> So
>   supported-connection-types-mac-side
>   supported-connection-types-line-side
> or maybe media-side?

Yes, that sounds a bit better and more descriptive.

> 
> I am still exploring whether this could be simply defined in the
> ethernet controllers `phy-mode` property, as Rob Herring says. It would
> be simpler...

OK.
Russell King (Oracle) March 25, 2021, 12:28 a.m. UTC | #7
On Thu, Mar 25, 2021 at 12:45:25AM +0100, Marek Behún wrote:
> On Wed, 24 Mar 2021 16:16:41 -0700
> Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
> > On 3/24/2021 4:00 PM, Marek Behún wrote:
> > > On Wed, 24 Mar 2021 14:19:28 -0700
> > > Florian Fainelli <f.fainelli@gmail.com> wrote:
> > >   
> > >>> Another problem is that if lower modes are supported, we should
> > >>> maybe use them in order to save power.    
> > >>
> > >> That is an interesting proposal but if you want it to be truly valuable,
> > >> does not that mean that an user ought to be able to switch between any
> > >> of the supported PHY <=> MAC interfaces at runtime, and then within
> > >> those interfaces to the speeds that yield the best power savings?  
> > > 
> > > If the code determines that there are multiple working configurations,
> > > it theoretically could allow the user to switch between them.
> > > 
> > > My idea was that this should be done by kernel, though.
> > > 
> > > But power saving is not the main problem I am trying to solve.
> > > What I am trying to solve is that if a board does not support all modes
> > > supported by the MAC and PHY, because they are not wired or something,
> > > we need to know about that so that we can select the correct mode for
> > > PHYs that change this mode at runtime.  
> > 
> > OK so the runtime part comes from plugging in various SFP modules into a
> > cage but other than that, for a "fixed" link such as a SFF or a soldered
> > down PHY, do we agree that there would be no runtime changing of the
> > 'phy-mode'?
> 
> No, we do not. The PHY can be configured (by strapping pins or by
> sw) to change phy-mode depending on the autonegotiated copper speed.
> 
> So if you plug in an ethernet cable where on the otherside is only 1g
> capable device, the PHY will change mode to sgmii. But if you then plug
> a 5g capable device, the PHY will change mode to 5gbase-r.
> 
> This happens if the PHY is configured into one of these changing
> configurations. It can also be configured to USXGMII, or 10GBASER with
> rate matching.
> 
> Not many MACs in kernel support USXGMII currently.
> 
> And if you use rate matching mode, and the copper side is
> linked in lower speed (2.5g for example), and the MAC will start
> sending too many packets, the internal buffer in the PHY is only 16 KB,
> so it will fill up quickly. So you need pause frames support. But this
> is broken for speeds <= 1g, according to erratum.

Also, the sending of pause frames is only supported for 88x3310P
devices, not the 88x3310. The plain 88x3310 requires the MAC to
rate-limit in this mode.
Marek Behún March 25, 2021, 12:43 a.m. UTC | #8
On Wed, 24 Mar 2021 17:11:25 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

> On 3/24/2021 4:45 PM, Marek Behún wrote:
> > On Wed, 24 Mar 2021 16:16:41 -0700
> > Florian Fainelli <f.fainelli@gmail.com> wrote:
> >   
> >> On 3/24/2021 4:00 PM, Marek Behún wrote:  
> >>> On Wed, 24 Mar 2021 14:19:28 -0700
> >>> Florian Fainelli <f.fainelli@gmail.com> wrote:
> >>>     
> >>>>> Another problem is that if lower modes are supported, we should
> >>>>> maybe use them in order to save power.      
> >>>>
> >>>> That is an interesting proposal but if you want it to be truly valuable,
> >>>> does not that mean that an user ought to be able to switch between any
> >>>> of the supported PHY <=> MAC interfaces at runtime, and then within
> >>>> those interfaces to the speeds that yield the best power savings?    
> >>>
> >>> If the code determines that there are multiple working configurations,
> >>> it theoretically could allow the user to switch between them.
> >>>
> >>> My idea was that this should be done by kernel, though.
> >>>
> >>> But power saving is not the main problem I am trying to solve.
> >>> What I am trying to solve is that if a board does not support all modes
> >>> supported by the MAC and PHY, because they are not wired or something,
> >>> we need to know about that so that we can select the correct mode for
> >>> PHYs that change this mode at runtime.    
> >>
> >> OK so the runtime part comes from plugging in various SFP modules into a
> >> cage but other than that, for a "fixed" link such as a SFF or a soldered
> >> down PHY, do we agree that there would be no runtime changing of the
> >> 'phy-mode'?  
> > 
> > No, we do not. The PHY can be configured (by strapping pins or by
> > sw) to change phy-mode depending on the autonegotiated copper speed.
> > 
> > So if you plug in an ethernet cable where on the otherside is only 1g
> > capable device, the PHY will change mode to sgmii. But if you then plug
> > a 5g capable device, the PHY will change mode to 5gbase-r.
> > 
> > This happens if the PHY is configured into one of these changing
> > configurations. It can also be configured to USXGMII, or 10GBASER with
> > rate matching.
> > 
> > Not many MACs in kernel support USXGMII currently.
> > 
> > And if you use rate matching mode, and the copper side is
> > linked in lower speed (2.5g for example), and the MAC will start
> > sending too many packets, the internal buffer in the PHY is only 16 KB,
> > so it will fill up quickly. So you need pause frames support. But this
> > is broken for speeds <= 1g, according to erratum.
> > 
> > So you really want to change modes. The rate matching mode is
> > basically useless.  
> 
> OK, so whenever there is a link change you are presumably reading the
> mode in which the PHY has been reconfigured to, asking the MAC to
> configured itself appropriately based on that, and if there is no
> intersection, error out?

No. At initialization I tell the PHY to change between
  10gbase-r / 5gbase-r / 2500base-x / sgmii
according to the copper side. The PHY will do this alone on change on
copper side. I don't need to do this.

(This already works with current version of marvell10g driver - but
 kernel is not configuring this, it has to be configure via strapping
 pins.)

But I can tell the PHY at initialization to change instead between
  xaui / 5gbase-r / 2500base-x / sgmii
Again the PHY will do this on its own whenever speed on the copper side
changes.

But I need to know which of this settings I should use.

Marek