Message ID | 162295949351.1109360.10329014558746500142.stgit@dwillia2-desk3.amr.corp.intel.com |
---|---|
Headers | show |
Series | CXL port and decoder enumeration | expand |
On Sun, Jun 6, 2021 at 8:05 AM Dan Williams <dan.j.williams@intel.com> wrote: > > The recently released CXL specification change (ECN) for the CXL Fixed > Memory Window Structure (CFMWS) extension to the CXL Early Discovery > Table (CEDT) enables a large amount of functionality. It defines the > root of a CXL memory topology and is needed for all OS flows for CXL > provisioning CXL memory expanders. For ease of merging and tree > management add the new ACPI definition locally (drivers/cxl/acpi.h) in > such a way that they will not collide with the eventual arrival of the > definitions through the ACPICA project to their final location > (drivers/acpi/actbl1.h). I've just applied the ACPICA series including this change which can be made available as a forward-only branch in my tree, if that helps. > The definitions in drivers/cxl/acpi.h can be dropped post -rc1. > > Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net> > Co-developed-by: Alison Schofield <alison.schofield@intel.com> > Co-developed-by: Erik Kaneda <erik.kaneda@intel.com> > Signed-off-by: Alison Schofield <alison.schofield@intel.com> > Signed-off-by: Erik Kaneda <erik.kaneda@intel.com> > Signed-off-by: Dan Williams <dan.j.williams@intel.com> > --- > drivers/cxl/acpi.h | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 48 insertions(+) > create mode 100644 drivers/cxl/acpi.h > > diff --git a/drivers/cxl/acpi.h b/drivers/cxl/acpi.h > new file mode 100644 > index 000000000000..1482c19e7227 > --- /dev/null > +++ b/drivers/cxl/acpi.h > @@ -0,0 +1,48 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* Copyright(c) 2021 Intel Corporation. */ > +#ifndef __CXL_ACPI_H__ > +#define __CXL_ACPI_H__ > + > +#ifndef ACPI_CEDT_CHBS_VERSION_CXL20 > +/* > + * NOTE: These definitions are temporary and to be deleted in v5.14-rc1 > + * when the identical definitions become available from > + * include/acpi/actbl1.h. > + */ > + > +#define ACPI_CEDT_TYPE_CFMWS 1 > +#define ACPI_CEDT_TYPE_RESERVED 2 > + > +#define ACPI_CEDT_CHBS_VERSION_CXL11 (0) > +#define ACPI_CEDT_CHBS_VERSION_CXL20 (1) > + > +#define ACPI_CEDT_CHBS_LENGTH_CXL11 (0x2000) > +#define ACPI_CEDT_CHBS_LENGTH_CXL20 (0x10000) > + > +struct acpi_cedt_cfmws { > + struct acpi_cedt_header header; > + u32 reserved1; > + u64 base_hpa; > + u64 window_size; > + u8 interleave_ways; > + u8 interleave_arithmetic; > + u16 reserved2; > + u32 granularity; > + u16 restrictions; > + u16 qtg_id; > + u32 interleave_targets[]; > +}; > + > +/* Values for Interleave Arithmetic field above */ > + > +#define ACPI_CEDT_CFMWS_ARITHMETIC_MODULO (0) > + > +/* Values for Restrictions field above */ > + > +#define ACPI_CEDT_CFMWS_RESTRICT_TYPE2 (1) > +#define ACPI_CEDT_CFMWS_RESTRICT_TYPE3 (1 << 1) > +#define ACPI_CEDT_CFMWS_RESTRICT_VOLATILE (1 << 2) > +#define ACPI_CEDT_CFMWS_RESTRICT_PMEM (1 << 3) > +#define ACPI_CEDT_CFMWS_RESTRICT_FIXED (1 << 4) > +#endif /* ACPI_CEDT_CHBS_VERSION_CXL20 */ > +#endif /* __CXL_ACPI_H__ */ >
On Mon, Jun 7, 2021 at 5:26 AM Rafael J. Wysocki <rafael@kernel.org> wrote: > > On Sun, Jun 6, 2021 at 8:05 AM Dan Williams <dan.j.williams@intel.com> wrote: > > > > The recently released CXL specification change (ECN) for the CXL Fixed > > Memory Window Structure (CFMWS) extension to the CXL Early Discovery > > Table (CEDT) enables a large amount of functionality. It defines the > > root of a CXL memory topology and is needed for all OS flows for CXL > > provisioning CXL memory expanders. For ease of merging and tree > > management add the new ACPI definition locally (drivers/cxl/acpi.h) in > > such a way that they will not collide with the eventual arrival of the > > definitions through the ACPICA project to their final location > > (drivers/acpi/actbl1.h). > > I've just applied the ACPICA series including this change which can be > made available as a forward-only branch in my tree, if that helps. Yes, please, that would be my preference. When I created this patch the concern was that a stable branch was possibly weeks away.
On Sat, 5 Jun 2021 23:05:22 -0700 Dan Williams <dan.j.williams@intel.com> wrote: > While the resources enumerated by the CEDT.CFMWS identify a cxl_port > with host bridges as downstream ports, host bridges themselves are > upstream ports that decode to downstream ports represented by PCIe Root > Ports. Walk the PCIe Root Ports connected to a CXL Host Bridge, > identified by the ACPI0016 _HID, and add each one as a cxl_dport of the > host bridge cxl_port. > > For now, component registers are not enumerated, only the first order > uport / dport relationships. > > Signed-off-by: Dan Williams <dan.j.williams@intel.com> LGTM Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> > --- > drivers/cxl/acpi.c | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++- > 1 file changed, 95 insertions(+), 2 deletions(-) > > diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c > index 0ae7464b603d..ec324bf063b8 100644 > --- a/drivers/cxl/acpi.c > +++ b/drivers/cxl/acpi.c > @@ -8,6 +8,48 @@ > #include <linux/pci.h> > #include "cxl.h" > > +struct cxl_walk_context { > + struct device *dev; > + struct pci_bus *root; > + struct cxl_port *port; > + int error; > + int count; > +}; > + > +static int match_add_root_ports(struct pci_dev *pdev, void *data) > +{ > + struct cxl_walk_context *ctx = data; > + struct pci_bus *root_bus = ctx->root; > + struct cxl_port *port = ctx->port; > + int type = pci_pcie_type(pdev); > + struct device *dev = ctx->dev; > + u32 lnkcap, port_num; > + int rc; > + > + if (pdev->bus != root_bus) > + return 0; > + if (!pci_is_pcie(pdev)) > + return 0; > + if (type != PCI_EXP_TYPE_ROOT_PORT) > + return 0; > + if (pci_read_config_dword(pdev, pci_pcie_cap(pdev) + PCI_EXP_LNKCAP, > + &lnkcap) != PCIBIOS_SUCCESSFUL) > + return 0; > + > + /* TODO walk DVSEC to find component register base */ > + port_num = FIELD_GET(PCI_EXP_LNKCAP_PN, lnkcap); > + rc = cxl_add_dport(port, &pdev->dev, port_num, CXL_RESOURCE_NONE); > + if (rc) { > + ctx->error = rc; > + return rc; > + } > + ctx->count++; > + > + dev_dbg(dev, "add dport%d: %s\n", port_num, dev_name(&pdev->dev)); > + > + return 0; > +} > + > static struct acpi_device *to_cxl_host_bridge(struct device *dev) > { > struct acpi_device *adev = to_acpi_device(dev); > @@ -17,6 +59,44 @@ static struct acpi_device *to_cxl_host_bridge(struct device *dev) > return NULL; > } > > +/* > + * A host bridge is a dport to a CFMWS decode and it is a uport to the > + * dport (PCIe Root Ports) in the host bridge. > + */ > +static int add_host_bridge_uport(struct device *match, void *arg) > +{ > + struct acpi_device *bridge = to_cxl_host_bridge(match); > + struct cxl_port *root_port = arg; > + struct device *host = root_port->dev.parent; > + struct acpi_pci_root *pci_root; > + struct cxl_walk_context ctx; > + struct cxl_port *port; > + > + if (!bridge) > + return 0; > + > + pci_root = acpi_pci_find_root(bridge->handle); > + if (!pci_root) > + return -ENXIO; > + > + /* TODO: fold in CEDT.CHBS retrieval */ > + port = devm_cxl_add_port(host, match, CXL_RESOURCE_NONE, root_port); > + if (IS_ERR(port)) > + return PTR_ERR(port); > + dev_dbg(host, "%s: add: %s\n", dev_name(match), dev_name(&port->dev)); > + > + ctx = (struct cxl_walk_context){ > + .dev = host, > + .root = pci_root->bus, > + .port = port, > + }; > + pci_walk_bus(pci_root->bus, match_add_root_ports, &ctx); > + > + if (ctx.count == 0) > + return -ENODEV; > + return ctx.error; > +} > + > static int add_host_bridge_dport(struct device *match, void *arg) > { > int rc; > @@ -49,6 +129,7 @@ static int add_host_bridge_dport(struct device *match, void *arg) > > static int cxl_acpi_probe(struct platform_device *pdev) > { > + int rc; > struct cxl_port *root_port; > struct device *host = &pdev->dev; > struct acpi_device *adev = ACPI_COMPANION(host); > @@ -58,8 +139,20 @@ static int cxl_acpi_probe(struct platform_device *pdev) > return PTR_ERR(root_port); > dev_dbg(host, "add: %s\n", dev_name(&root_port->dev)); > > - return bus_for_each_dev(adev->dev.bus, NULL, root_port, > - add_host_bridge_dport); > + rc = bus_for_each_dev(adev->dev.bus, NULL, root_port, > + add_host_bridge_dport); > + if (rc) > + return rc; > + > + /* > + * Root level scanned with host-bridge as dports, now scan host-bridges > + * for their role as CXL uports to their CXL-capable PCIe Root Ports. > + */ > + rc = bus_for_each_dev(adev->dev.bus, NULL, root_port, > + add_host_bridge_uport); > + if (rc) > + dev_err(host, "failed to scan host bridges\n"); > + return 0; > } > > static const struct acpi_device_id cxl_acpi_ids[] = { >
On Mon, Jun 7, 2021 at 10:03 AM Dan Williams <dan.j.williams@intel.com> wrote: > > On Mon, Jun 7, 2021 at 5:26 AM Rafael J. Wysocki <rafael@kernel.org> wrote: > > > > On Sun, Jun 6, 2021 at 8:05 AM Dan Williams <dan.j.williams@intel.com> wrote: > > > > > > The recently released CXL specification change (ECN) for the CXL Fixed > > > Memory Window Structure (CFMWS) extension to the CXL Early Discovery > > > Table (CEDT) enables a large amount of functionality. It defines the > > > root of a CXL memory topology and is needed for all OS flows for CXL > > > provisioning CXL memory expanders. For ease of merging and tree > > > management add the new ACPI definition locally (drivers/cxl/acpi.h) in > > > such a way that they will not collide with the eventual arrival of the > > > definitions through the ACPICA project to their final location > > > (drivers/acpi/actbl1.h). > > > > I've just applied the ACPICA series including this change which can be > > made available as a forward-only branch in my tree, if that helps. > > Yes, please, that would be my preference. When I created this patch > the concern was that a stable branch was possibly weeks away. Rafael, I see "4a2c1dcfaf59 ACPICA: Add the CFMWS structure definition to the CEDT table" in your tree, I can safely assume that commit will not rebase at this point? I'll likely rewind your acpica branch to that point and merge there to avoid carrying any unrelated follow-on commits.
On Tue, Jun 8, 2021 at 8:13 PM Dan Williams <dan.j.williams@intel.com> wrote: > > On Mon, Jun 7, 2021 at 10:03 AM Dan Williams <dan.j.williams@intel.com> wrote: > > > > On Mon, Jun 7, 2021 at 5:26 AM Rafael J. Wysocki <rafael@kernel.org> wrote: > > > > > > On Sun, Jun 6, 2021 at 8:05 AM Dan Williams <dan.j.williams@intel.com> wrote: > > > > > > > > The recently released CXL specification change (ECN) for the CXL Fixed > > > > Memory Window Structure (CFMWS) extension to the CXL Early Discovery > > > > Table (CEDT) enables a large amount of functionality. It defines the > > > > root of a CXL memory topology and is needed for all OS flows for CXL > > > > provisioning CXL memory expanders. For ease of merging and tree > > > > management add the new ACPI definition locally (drivers/cxl/acpi.h) in > > > > such a way that they will not collide with the eventual arrival of the > > > > definitions through the ACPICA project to their final location > > > > (drivers/acpi/actbl1.h). > > > > > > I've just applied the ACPICA series including this change which can be > > > made available as a forward-only branch in my tree, if that helps. > > > > Yes, please, that would be my preference. When I created this patch > > the concern was that a stable branch was possibly weeks away. > > Rafael, I see "4a2c1dcfaf59 ACPICA: Add the CFMWS structure definition > to the CEDT table" in your tree, I can safely assume that commit will > not rebase at this point? Yes, please. > I'll likely rewind your acpica branch to > that point and merge there to avoid carrying any unrelated follow-on > commits. Sure.