| Message ID | 20210112083931.1662874-1-idosch@idosch.org |
|---|---|
| Headers | show |
| Series | mlxsw: Expose number of physical ports | expand |
On Tue, 12 Jan 2021 10:39:30 +0200 Ido Schimmel wrote: > From: Danielle Ratson <danieller@nvidia.com> > > The switch ASIC has a limited capacity of physical ('flavour physical' > in devlink terminology) ports that it can support. While each system is > brought up with a different number of ports, this number can be > increased via splitting up to the ASIC's limit. > > Expose physical ports as a devlink resource so that user space will have > visibility to the maximum number of ports that can be supported and the > current occupancy. Any thoughts on making this a "generic" resource? The limitation on number of logical ports is pretty common for switches.
On Tue, Jan 12, 2021 at 08:21:22PM -0800, Jakub Kicinski wrote: > On Tue, 12 Jan 2021 10:39:30 +0200 Ido Schimmel wrote: > > From: Danielle Ratson <danieller@nvidia.com> > > > > The switch ASIC has a limited capacity of physical ('flavour physical' > > in devlink terminology) ports that it can support. While each system is > > brought up with a different number of ports, this number can be > > increased via splitting up to the ASIC's limit. > > > > Expose physical ports as a devlink resource so that user space will have > > visibility to the maximum number of ports that can be supported and the > > current occupancy. > > Any thoughts on making this a "generic" resource? It might be possible to allow drivers to pass the maximum number of physical ports to devlink during their initialization. Devlink can then use it as an indication to register the resource itself instead of the driver. It can report the current occupancy without driver intervention since the list of ports is maintained in devlink. There might be an issue with the resource identifier which is a 64-bit number passed from drivers. I think we can partition this to identifiers allocated by devlink / drivers. Danielle / Jiri?
Wed, Jan 13, 2021 at 09:32:41AM CET, idosch@idosch.org wrote: >On Tue, Jan 12, 2021 at 08:21:22PM -0800, Jakub Kicinski wrote: >> On Tue, 12 Jan 2021 10:39:30 +0200 Ido Schimmel wrote: >> > From: Danielle Ratson <danieller@nvidia.com> >> > >> > The switch ASIC has a limited capacity of physical ('flavour physical' >> > in devlink terminology) ports that it can support. While each system is >> > brought up with a different number of ports, this number can be >> > increased via splitting up to the ASIC's limit. >> > >> > Expose physical ports as a devlink resource so that user space will have >> > visibility to the maximum number of ports that can be supported and the >> > current occupancy. >> >> Any thoughts on making this a "generic" resource? > >It might be possible to allow drivers to pass the maximum number of >physical ports to devlink during their initialization. Devlink can then >use it as an indication to register the resource itself instead of the >driver. It can report the current occupancy without driver intervention >since the list of ports is maintained in devlink. > >There might be an issue with the resource identifier which is a 64-bit >number passed from drivers. I think we can partition this to identifiers >allocated by devlink / drivers. > >Danielle / Jiri? There is no concept of "generic resource". And I think it is a good reason for it, as the resource is something which is always quite hw-specific. Port number migth be one exception. Can you think of anything else? If not, I would vote for not having "generic resource" just for this one case.
On Wed, Jan 13, 2021 at 02:39:02PM +0100, Jiri Pirko wrote: > Wed, Jan 13, 2021 at 09:32:41AM CET, idosch@idosch.org wrote: > >On Tue, Jan 12, 2021 at 08:21:22PM -0800, Jakub Kicinski wrote: > >> On Tue, 12 Jan 2021 10:39:30 +0200 Ido Schimmel wrote: > >> > From: Danielle Ratson <danieller@nvidia.com> > >> > > >> > The switch ASIC has a limited capacity of physical ('flavour physical' > >> > in devlink terminology) ports that it can support. While each system is > >> > brought up with a different number of ports, this number can be > >> > increased via splitting up to the ASIC's limit. > >> > > >> > Expose physical ports as a devlink resource so that user space will have > >> > visibility to the maximum number of ports that can be supported and the > >> > current occupancy. > >> > >> Any thoughts on making this a "generic" resource? > > > >It might be possible to allow drivers to pass the maximum number of > >physical ports to devlink during their initialization. Devlink can then > >use it as an indication to register the resource itself instead of the > >driver. It can report the current occupancy without driver intervention > >since the list of ports is maintained in devlink. > > > >There might be an issue with the resource identifier which is a 64-bit > >number passed from drivers. I think we can partition this to identifiers > >allocated by devlink / drivers. > > > >Danielle / Jiri? > > There is no concept of "generic resource". And I think it is a good > reason for it, as the resource is something which is always quite > hw-specific. Port number migth be one exception. Can you think of > anything else? If not, I would vote for not having "generic resource" > just for this one case. I think Jakub's point is that he does not want drivers to expose the same resource to user space under different names. Question is how to try to guarantee it. One option is what I suggested above, but it might be an overkill. Another option is better documentation. To add a section of "generic" resources in devlink-resource documentation [1] and modify the kernel-doc comment above devlink_resource_register() to point to it. [1] https://www.kernel.org/doc/html/latest/networking/devlink/devlink-resource.html
On Wed, 13 Jan 2021 16:26:56 +0200 Ido Schimmel wrote: > On Wed, Jan 13, 2021 at 02:39:02PM +0100, Jiri Pirko wrote: > > Wed, Jan 13, 2021 at 09:32:41AM CET, idosch@idosch.org wrote: > > >On Tue, Jan 12, 2021 at 08:21:22PM -0800, Jakub Kicinski wrote: > > >> On Tue, 12 Jan 2021 10:39:30 +0200 Ido Schimmel wrote: > > >> > From: Danielle Ratson <danieller@nvidia.com> > > >> > > > >> > The switch ASIC has a limited capacity of physical ('flavour physical' > > >> > in devlink terminology) ports that it can support. While each system is > > >> > brought up with a different number of ports, this number can be > > >> > increased via splitting up to the ASIC's limit. > > >> > > > >> > Expose physical ports as a devlink resource so that user space will have > > >> > visibility to the maximum number of ports that can be supported and the > > >> > current occupancy. > > >> > > >> Any thoughts on making this a "generic" resource? > > > > > >It might be possible to allow drivers to pass the maximum number of > > >physical ports to devlink during their initialization. Devlink can then > > >use it as an indication to register the resource itself instead of the > > >driver. It can report the current occupancy without driver intervention > > >since the list of ports is maintained in devlink. > > > > > >There might be an issue with the resource identifier which is a 64-bit > > >number passed from drivers. I think we can partition this to identifiers > > >allocated by devlink / drivers. > > > > > >Danielle / Jiri? > > > > There is no concept of "generic resource". And I think it is a good > > reason for it, as the resource is something which is always quite > > hw-specific. Port number migth be one exception. Can you think of > > anything else? If not, I would vote for not having "generic resource" > > just for this one case. > > I think Jakub's point is that he does not want drivers to expose the > same resource to user space under different names. Exactly. > Question is how to > try to guarantee it. One option is what I suggested above, but it might > be an overkill. Another option is better documentation. To add a section > of "generic" resources in devlink-resource documentation [1] and modify > the kernel-doc comment above devlink_resource_register() to point to it. > > [1] https://www.kernel.org/doc/html/latest/networking/devlink/devlink-resource.html Yup, an entry in documentation and a common define in net/devlink.h is fine by me. We can always move around the kernel internals, like what registers the resource, later.
From: Ido Schimmel <idosch@nvidia.com> The switch ASIC has a limited capacity of physical ports that it can support. While each system is brought up with a different number of ports, this number can be increased via splitting up to the ASIC's limit. Expose physical ports as a devlink resource so that user space will have visibility into the maximum number of ports that can be supported and the current occupancy. With this resource it is possible, for example, to write generic (i.e., not platform dependent) tests for port splitting. Patch #1 adds the new resource and patch #2 adds a selftest. Danielle Ratson (2): mlxsw: Register physical ports as a devlink resource selftests: mlxsw: Add a scale test for physical ports drivers/net/ethernet/mellanox/mlxsw/core.c | 77 ++++++++++++++++--- drivers/net/ethernet/mellanox/mlxsw/core.h | 5 ++ .../net/ethernet/mellanox/mlxsw/spectrum.h | 2 +- .../selftests/drivers/net/mlxsw/port_scale.sh | 64 +++++++++++++++ .../net/mlxsw/spectrum-2/port_scale.sh | 16 ++++ .../net/mlxsw/spectrum-2/resource_scale.sh | 2 +- .../drivers/net/mlxsw/spectrum/port_scale.sh | 16 ++++ .../net/mlxsw/spectrum/resource_scale.sh | 2 +- 8 files changed, 171 insertions(+), 13 deletions(-) create mode 100644 tools/testing/selftests/drivers/net/mlxsw/port_scale.sh create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh