| Message ID | 20210515124055.22225-1-sergio.paracuellos@gmail.com |
|---|---|
| Headers | show |
| Series | MIPS: ralink: pci: driver for Pcie controller in MT7621 SoCs | expand |
On Sat, May 15, 2021 at 02:40:51PM +0200, Sergio Paracuellos wrote: > MediaTek MT7621 PCIe subsys supports single Root complex (RC) > with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link. > Topology is as follows: > > MT7621 PCIe HOST Topology > > .-------. > | | > | CPU | > | | > '-------' > | > | > | > v > .------------------. > .-----------| HOST/PCI Bridge |------------. > | '------------------' | Type1 > BUS0 | | | Access > v v v On Bus0 > .-------------. .-------------. .-------------. > | VIRTUAL P2P | | VIRTUAL P2P | | VIRTUAL P2P | > | BUS0 | | BUS0 | | BUS0 | > | DEV0 | | DEV1 | | DEV2 | > '-------------' '-------------' '-------------' > Type0 | Type0 | Type0 | > Access BUS1 | Access BUS2| Access BUS3| > On Bus1 v On Bus2 v On Bus3 v > .----------. .----------. .----------. > | Device 0 | | Device 0 | | Device 0 | > | Func 0 | | Func 0 | | Func 0 | > '----------' '----------' '----------' > > This driver has been very long time in staging and I have been cleaning > it from its first versions where there was code kaos and PCI_LEGACY support. > Original code came probably from openWRT based on mediatek's SDK code. There > is no documentation at all about the mt7621 PCI subsystem. > I have been cleaning it targeting mt7621 SoC which is the one I use in > my GNUBee PC1 board and HiLink HLK-MT7621A evaluation board. > > Now I think is clean enough to be moved into 'arch/mips/pci'. > > This driver also uses already mainlined pci phy driver located in > 'drivers/phy/ralink/phy-mt7621-pci.c'. There are two instances of > the phy being the first one dual ported for pci0 and pci1, and the > second one not dual ported dedicated to pci2. Because of writing twice > some phy registers of the dual-ported one sometimes become in not > confident boot cycles we have to take care of this when device link > is checked here in controller driver. We power on the dual ported-phy > if there is something connected in pcie0 or pcie1. In the same manner > we have to properly disable it only if nothing is connected in of both > pcie0 and pci1 slots. > > Another thing that must be mentioned is that this driver uses IO > in physical address 0x001e160000. IO_SPACE_LIMIT for MIPS is 0xffff > so some generic PCI functions (like of_pci_range_to_resource) won't > work and the resource ranges part for IO is set manually. > > I had already sent binding documentation to be reviewed but I am > include also here with the driver itself and this cover letter > to make easy review process. > > Best regards, > Sergio Paracuellos > > Sergio Paracuellos (4): > dt-bindings: mt7621-pci: PCIe binding documentation for MT7621 SoCs > MIPS: pci: Add driver for MT7621 PCIe controller > staging: mt7621-pci: remove driver from staging Generally it's better if the move can be done in one commit instead of an add followed by a remove. I see there are a bunch of MIPS PCI controller drivers in arch/mips/pci/, so I see the argument for putting this one there as well. But most of the similar drivers are in drivers/pci/controller/, where I think it's easier to keep them up to date with changes in the PCI core. Have you considered putting this one there? > MAINTAINERS: add myself as maintainer of the MT7621 PCI controller > driver > > .../bindings/pci/mediatek,mt7621-pci.yaml | 149 ++++++++++++++++++ > MAINTAINERS | 6 + > arch/mips/pci/Makefile | 1 + > .../mt7621-pci => arch/mips/pci}/pci-mt7621.c | 0 > arch/mips/ralink/Kconfig | 9 +- > drivers/staging/Kconfig | 2 - > drivers/staging/Makefile | 1 - > drivers/staging/mt7621-pci/Kconfig | 8 - > drivers/staging/mt7621-pci/Makefile | 2 - > drivers/staging/mt7621-pci/TODO | 4 - > .../mt7621-pci/mediatek,mt7621-pci.txt | 104 ------------ > 11 files changed, 164 insertions(+), 122 deletions(-) > create mode 100644 Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > rename {drivers/staging/mt7621-pci => arch/mips/pci}/pci-mt7621.c (100%) > delete mode 100644 drivers/staging/mt7621-pci/Kconfig > delete mode 100644 drivers/staging/mt7621-pci/Makefile > delete mode 100644 drivers/staging/mt7621-pci/TODO > delete mode 100644 drivers/staging/mt7621-pci/mediatek,mt7621-pci.txt > > -- > 2.25.1
Hi Bjorn, On Wed, May 19, 2021 at 10:36 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > On Sat, May 15, 2021 at 02:40:51PM +0200, Sergio Paracuellos wrote: > > MediaTek MT7621 PCIe subsys supports single Root complex (RC) > > with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link. > > Topology is as follows: > > > > MT7621 PCIe HOST Topology > > > > .-------. > > | | > > | CPU | > > | | > > '-------' > > | > > | > > | > > v > > .------------------. > > .-----------| HOST/PCI Bridge |------------. > > | '------------------' | Type1 > > BUS0 | | | Access > > v v v On Bus0 > > .-------------. .-------------. .-------------. > > | VIRTUAL P2P | | VIRTUAL P2P | | VIRTUAL P2P | > > | BUS0 | | BUS0 | | BUS0 | > > | DEV0 | | DEV1 | | DEV2 | > > '-------------' '-------------' '-------------' > > Type0 | Type0 | Type0 | > > Access BUS1 | Access BUS2| Access BUS3| > > On Bus1 v On Bus2 v On Bus3 v > > .----------. .----------. .----------. > > | Device 0 | | Device 0 | | Device 0 | > > | Func 0 | | Func 0 | | Func 0 | > > '----------' '----------' '----------' > > > > This driver has been very long time in staging and I have been cleaning > > it from its first versions where there was code kaos and PCI_LEGACY support. > > Original code came probably from openWRT based on mediatek's SDK code. There > > is no documentation at all about the mt7621 PCI subsystem. > > I have been cleaning it targeting mt7621 SoC which is the one I use in > > my GNUBee PC1 board and HiLink HLK-MT7621A evaluation board. > > > > Now I think is clean enough to be moved into 'arch/mips/pci'. > > > > This driver also uses already mainlined pci phy driver located in > > 'drivers/phy/ralink/phy-mt7621-pci.c'. There are two instances of > > the phy being the first one dual ported for pci0 and pci1, and the > > second one not dual ported dedicated to pci2. Because of writing twice > > some phy registers of the dual-ported one sometimes become in not > > confident boot cycles we have to take care of this when device link > > is checked here in controller driver. We power on the dual ported-phy > > if there is something connected in pcie0 or pcie1. In the same manner > > we have to properly disable it only if nothing is connected in of both > > pcie0 and pci1 slots. > > > > Another thing that must be mentioned is that this driver uses IO > > in physical address 0x001e160000. IO_SPACE_LIMIT for MIPS is 0xffff > > so some generic PCI functions (like of_pci_range_to_resource) won't > > work and the resource ranges part for IO is set manually. > > > > I had already sent binding documentation to be reviewed but I am > > include also here with the driver itself and this cover letter > > to make easy review process. > > > > Best regards, > > Sergio Paracuellos > > > > Sergio Paracuellos (4): > > dt-bindings: mt7621-pci: PCIe binding documentation for MT7621 SoCs > > MIPS: pci: Add driver for MT7621 PCIe controller > > staging: mt7621-pci: remove driver from staging > > Generally it's better if the move can be done in one commit instead of > an add followed by a remove. I have no problem at all to just move the driver instead of add and remove. But it is easier to review initially in this way. No other reason but this one. So, if after review is ok, we can just move this in the way you are pointing out here. > > I see there are a bunch of MIPS PCI controller drivers in > arch/mips/pci/, so I see the argument for putting this one there as > well. > > But most of the similar drivers are in drivers/pci/controller/, where > I think it's easier to keep them up to date with changes in the PCI > core. Have you considered putting this one there? Most pci drivers in 'arch/mips/pci' are still using PCI_LEGACY stuff. In contrast mt7621-pci is using current pci generic apis but even most of the code is generic enough, there is one remaining thing which depends on mips architecture which is the iocu region configuration which must be done in the driver itself. This is the only reason to move this driver into 'arch/mips/pci' instead of 'drivers/pci/controller/'. So... I am all ears to listen to suggestions for the proper place for this driver. Thomas, do you have any thoughts on this? Thanks, Sergio Paracuellos > > > MAINTAINERS: add myself as maintainer of the MT7621 PCI controller > > driver > > > > .../bindings/pci/mediatek,mt7621-pci.yaml | 149 ++++++++++++++++++ > > MAINTAINERS | 6 + > > arch/mips/pci/Makefile | 1 + > > .../mt7621-pci => arch/mips/pci}/pci-mt7621.c | 0 > > arch/mips/ralink/Kconfig | 9 +- > > drivers/staging/Kconfig | 2 - > > drivers/staging/Makefile | 1 - > > drivers/staging/mt7621-pci/Kconfig | 8 - > > drivers/staging/mt7621-pci/Makefile | 2 - > > drivers/staging/mt7621-pci/TODO | 4 - > > .../mt7621-pci/mediatek,mt7621-pci.txt | 104 ------------ > > 11 files changed, 164 insertions(+), 122 deletions(-) > > create mode 100644 Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > rename {drivers/staging/mt7621-pci => arch/mips/pci}/pci-mt7621.c (100%) > > delete mode 100644 drivers/staging/mt7621-pci/Kconfig > > delete mode 100644 drivers/staging/mt7621-pci/Makefile > > delete mode 100644 drivers/staging/mt7621-pci/TODO > > delete mode 100644 drivers/staging/mt7621-pci/mediatek,mt7621-pci.txt > > > > -- > > 2.25.1
On Wed, May 19, 2021 at 11:18:36PM +0200, Sergio Paracuellos wrote: > On Wed, May 19, 2021 at 10:36 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > But most of the similar drivers are in drivers/pci/controller/, where > > I think it's easier to keep them up to date with changes in the PCI > > core. Have you considered putting this one there? > > Most pci drivers in 'arch/mips/pci' are still using PCI_LEGACY stuff. > In contrast mt7621-pci is using current pci generic apis but even most > of the code is generic enough, there is one remaining thing which > depends on mips architecture which is the iocu region configuration > which must be done in the driver itself. This is the only reason to > move this driver into 'arch/mips/pci' instead of > 'drivers/pci/controller/'. So... I am all ears to listen to > suggestions for the proper place for this driver. Thomas, do you have > any thoughts on this? I tried to put a pci-xtalk driver into drivers/pci/controller, but Lorenzo didn't want it there for being MIPS and not DT based. So this one is DT based, but still MIPS. I'm perfectly fine putting this driver into drivers/pci/controller/ Lorenzo, what's your opinion ? Thomas. -- Crap can work. Given enough thrust pigs will fly, but it's not necessarily a good idea. [ RFC1925, 2.3 ]
Hi Rob, On Sat, May 15, 2021 at 2:40 PM Sergio Paracuellos <sergio.paracuellos@gmail.com> wrote: > > Add device tree binding documentation for PCIe in MT7621 SoCs. > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > --- > .../bindings/pci/mediatek,mt7621-pci.yaml | 149 ++++++++++++++++++ > 1 file changed, 149 insertions(+) > create mode 100644 Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > diff --git a/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > new file mode 100644 > index 000000000000..7f5f9d583032 > --- /dev/null > +++ b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > @@ -0,0 +1,149 @@ > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/pci/mediatek,mt7621-pci.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: MediaTek MT7621 PCIe controller > + > +maintainers: > + - Sergio Paracuellos <sergio.paracuellos@gmail.com> > + > +description: |+ > + MediaTek MT7621 PCIe subsys supports single Root complex (RC) > + with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link > + > +allOf: > + - $ref: /schemas/pci/pci-bus.yaml# > + > +properties: > + compatible: > + const: mediatek,mt7621-pci > + > + reg: > + items: > + - description: host-pci bridge registers > + - description: pcie port 0 RC control registers > + - description: pcie port 1 RC control registers > + - description: pcie port 2 RC control registers > + > + ranges: > + maxItems: 2 > + > + resets: > + items: > + - description: pcie port 0 reset. > + - description: pcie port 1 reset. > + - description: pcie port 2 reset. > + > + reset-names: > + items: > + - const: pcie0 > + - const: pcie1 > + - const: pcie2 > + > + clocks: > + items: > + - description: pcie port 0 clock. > + - description: pcie port 1 clock. > + - description: pcie port 2 clock. > + > + clock-names: > + items: > + - const: pcie0 > + - const: pcie1 > + - const: pcie2 > + > + phys: > + items: > + - description: Dual-ported phy for pcie port 0 and 1. > + - description: Phy for pcie port 2. > + > + phy-names: > + items: > + - const: pcie-phy0 > + - const: pcie-phy2 > + > +required: > + - compatible > + - reg > + - ranges > + - "#interrupt-cells" > + - interrupt-map-mask > + - interrupt-map > + - resets > + - reset-names > + - clocks > + - clock-names > + - phys > + - phy-names > + - reset-gpios > + > +unevaluatedProperties: false > + > +examples: > + - | > + #include <dt-bindings/gpio/gpio.h> > + #include <dt-bindings/interrupt-controller/mips-gic.h> > + > + pcie: pcie@1e140000 { > + compatible = "mediatek,mt7621-pci"; > + reg = <0x1e140000 0x100>, > + <0x1e142000 0x100>, > + <0x1e143000 0x100>, > + <0x1e144000 0x100>; > + > + #address-cells = <3>; > + #size-cells = <2>; > + pinctrl-names = "default"; > + pinctrl-0 = <&pcie_pins>; > + device_type = "pci"; > + ranges = <0x02000000 0 0x00000000 0x60000000 0 0x10000000>, /* pci memory */ > + <0x01000000 0 0x00000000 0x1e160000 0 0x00010000>; /* io space */ > + #interrupt-cells = <1>; > + interrupt-map-mask = <0xF800 0 0 0>; > + interrupt-map = <0x0000 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>, > + <0x0800 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>, > + <0x1000 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > + resets = <&rstctrl 24>, <&rstctrl 25>, <&rstctrl 26>; > + reset-names = "pcie0", "pcie1", "pcie2"; > + clocks = <&clkctrl 24>, <&clkctrl 25>, <&clkctrl 26>; > + clock-names = "pcie0", "pcie1", "pcie2"; > + phys = <&pcie0_phy 1>, <&pcie2_phy 0>; > + phy-names = "pcie-phy0", "pcie-phy2"; > + reset-gpios = <&gpio 19 GPIO_ACTIVE_LOW>; > + > + pcie@0,0 { > + reg = <0x0000 0 0 0 0>; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + #interrupt-cells = <1>; > + interrupt-map-mask = <0 0 0 0>; > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>; > + ranges; > + }; > + > + pcie@1,0 { > + reg = <0x0800 0 0 0 0>; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + #interrupt-cells = <1>; > + interrupt-map-mask = <0 0 0 0>; > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>; > + ranges; > + }; > + > + pcie@2,0 { > + reg = <0x1000 0 0 0 0>; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + #interrupt-cells = <1>; > + interrupt-map-mask = <0 0 0 0>; > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > + ranges; > + }; > + }; > +... > -- > 2.25.1 > Any comments on this binding? It's been a while in devicetree patchwork review list without comments. Thanks in advance for your time. Best regards, Sergio Paracuellos
On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > This patch adds a driver for the PCIe controller of MT7621 SoC. > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > --- > arch/mips/pci/Makefile | 1 + > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > arch/mips/ralink/Kconfig | 9 +- > 3 files changed, 633 insertions(+), 1 deletion(-) > create mode 100644 arch/mips/pci/pci-mt7621.c > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > index f3eecc065e5c..178c550739c4 100644 > --- a/arch/mips/pci/Makefile > +++ b/arch/mips/pci/Makefile > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > # > # These are still pretty much in the old state, watch, go blind. > # > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > new file mode 100644 > index 000000000000..fe1945819d25 > --- /dev/null > +++ b/arch/mips/pci/pci-mt7621.c ... > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > +{ > + struct device *dev = pcie->dev; > + struct mt7621_pcie_port *port; > + u8 num_slots_enabled = 0; > + u32 slot; > + u32 val; > + int err; > + > + /* Setup MEMWIN and IOWIN */ > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > + > + list_for_each_entry(port, &pcie->ports, list) { > + if (port->enabled) { > + err = clk_prepare_enable(port->clk); > + if (err) { > + dev_err(dev, "enabling clk pcie%d\n", slot); > + return err; > + } > + > + mt7621_pcie_enable_port(port); > + dev_info(dev, "PCIE%d enabled\n", port->slot); > + num_slots_enabled++; > + } > + } > + > + for (slot = 0; slot < num_slots_enabled; slot++) { > + val = read_config(pcie, slot, PCI_COMMAND); > + val |= PCI_COMMAND_MASTER; > + write_config(pcie, slot, PCI_COMMAND, val); Hello! Is this part of code correct? Because it looks strange if PCIe controller driver automatically enables PCI bus mastering, prior device driver initialize itself. Moreover kernel has already function pci_set_master() for this purpose which is used by device drivers. So I think this code can confuse some device drivers... > + /* configure RC FTS number to 250 when it leaves L0s */ > + val = read_config(pcie, slot, PCIE_FTS_NUM); > + val &= ~PCIE_FTS_NUM_MASK; > + val |= PCIE_FTS_NUM_L0(0x50); > + write_config(pcie, slot, PCIE_FTS_NUM, val); > + } > + > + return 0; > +}
On Friday 21 May 2021 12:23:38 Thomas Bogendoerfer wrote: > On Wed, May 19, 2021 at 11:18:36PM +0200, Sergio Paracuellos wrote: > > On Wed, May 19, 2021 at 10:36 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > > But most of the similar drivers are in drivers/pci/controller/, where > > > I think it's easier to keep them up to date with changes in the PCI > > > core. Have you considered putting this one there? > > > > Most pci drivers in 'arch/mips/pci' are still using PCI_LEGACY stuff. > > In contrast mt7621-pci is using current pci generic apis but even most > > of the code is generic enough, there is one remaining thing which > > depends on mips architecture which is the iocu region configuration > > which must be done in the driver itself. This is the only reason to > > move this driver into 'arch/mips/pci' instead of > > 'drivers/pci/controller/'. So... I am all ears to listen to > > suggestions for the proper place for this driver. Thomas, do you have > > any thoughts on this? > > I tried to put a pci-xtalk driver into drivers/pci/controller, but > Lorenzo didn't want it there for being MIPS and not DT based. So this > one is DT based, but still MIPS. I'm perfectly fine putting this > driver into drivers/pci/controller/ In my personal opinion this driver could go into drivers/pci/controller/
Hi Pali, Thanks for your comments. On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > --- > > arch/mips/pci/Makefile | 1 + > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > arch/mips/ralink/Kconfig | 9 +- > > 3 files changed, 633 insertions(+), 1 deletion(-) > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > index f3eecc065e5c..178c550739c4 100644 > > --- a/arch/mips/pci/Makefile > > +++ b/arch/mips/pci/Makefile > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > # > > # These are still pretty much in the old state, watch, go blind. > > # > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > new file mode 100644 > > index 000000000000..fe1945819d25 > > --- /dev/null > > +++ b/arch/mips/pci/pci-mt7621.c > ... > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > +{ > > + struct device *dev = pcie->dev; > > + struct mt7621_pcie_port *port; > > + u8 num_slots_enabled = 0; > > + u32 slot; > > + u32 val; > > + int err; > > + > > + /* Setup MEMWIN and IOWIN */ > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > + > > + list_for_each_entry(port, &pcie->ports, list) { > > + if (port->enabled) { > > + err = clk_prepare_enable(port->clk); > > + if (err) { > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > + return err; > > + } > > + > > + mt7621_pcie_enable_port(port); > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > + num_slots_enabled++; > > + } > > + } > > + > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > + val = read_config(pcie, slot, PCI_COMMAND); > > + val |= PCI_COMMAND_MASTER; > > + write_config(pcie, slot, PCI_COMMAND, val); > > Hello! Is this part of code correct? Because it looks strange if PCIe > controller driver automatically enables PCI bus mastering, prior device > driver initialize itself. > > Moreover kernel has already function pci_set_master() for this purpose > which is used by device drivers. > > So I think this code can confuse some device drivers... I agree that we have pci_set_master() to be used in pci device driver code. Original controller driver set this bit for enabled slots. Since there is no documentation at all for the PCI in this SoC I have maintained the setting in the driver in a cleaner way. See original driver code and the setting here [0]. There is no other reason than that. I am ok with removing this from here and testing with my two devices that everything is still ok if having this setting in the pci controller driver is a real problem. [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 Best regards, Sergio Paracuellos > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > + val &= ~PCIE_FTS_NUM_MASK; > > + val |= PCIE_FTS_NUM_L0(0x50); > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > + } > > + > > + return 0; > > +}
On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > Hi Pali, > > Thanks for your comments. > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > --- > > > arch/mips/pci/Makefile | 1 + > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > arch/mips/ralink/Kconfig | 9 +- > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > index f3eecc065e5c..178c550739c4 100644 > > > --- a/arch/mips/pci/Makefile > > > +++ b/arch/mips/pci/Makefile > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > # > > > # These are still pretty much in the old state, watch, go blind. > > > # > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > new file mode 100644 > > > index 000000000000..fe1945819d25 > > > --- /dev/null > > > +++ b/arch/mips/pci/pci-mt7621.c > > ... > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > +{ > > > + struct device *dev = pcie->dev; > > > + struct mt7621_pcie_port *port; > > > + u8 num_slots_enabled = 0; > > > + u32 slot; > > > + u32 val; > > > + int err; > > > + > > > + /* Setup MEMWIN and IOWIN */ > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > + > > > + list_for_each_entry(port, &pcie->ports, list) { > > > + if (port->enabled) { > > > + err = clk_prepare_enable(port->clk); > > > + if (err) { > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > + return err; > > > + } > > > + > > > + mt7621_pcie_enable_port(port); > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > + num_slots_enabled++; > > > + } > > > + } > > > + > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > + val |= PCI_COMMAND_MASTER; > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > controller driver automatically enables PCI bus mastering, prior device > > driver initialize itself. > > > > Moreover kernel has already function pci_set_master() for this purpose > > which is used by device drivers. > > > > So I think this code can confuse some device drivers... > > I agree that we have pci_set_master() to be used in pci device driver > code. Original controller driver set this bit for enabled slots. Since > there is no documentation at all for the PCI in this SoC I see... this is really a big problem to do any driver development... > I have > maintained the setting in the driver in a cleaner way. See original > driver code and the setting here [0]. There is no other reason than > that. I am ok with removing this from here and testing with my two > devices that everything is still ok if having this setting in the pci > controller driver is a real problem. You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and then compare outputs. Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is needed, so it is possible that there would be no difference in lspci output. > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > Best regards, > Sergio Paracuellos > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > + val &= ~PCIE_FTS_NUM_MASK; > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > + } > > > + > > > + return 0; > > > +}
On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > Hi Pali, > > > > Thanks for your comments. > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > --- > > > > arch/mips/pci/Makefile | 1 + > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > arch/mips/ralink/Kconfig | 9 +- > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > index f3eecc065e5c..178c550739c4 100644 > > > > --- a/arch/mips/pci/Makefile > > > > +++ b/arch/mips/pci/Makefile > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > # > > > > # These are still pretty much in the old state, watch, go blind. > > > > # > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > new file mode 100644 > > > > index 000000000000..fe1945819d25 > > > > --- /dev/null > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > ... > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > +{ > > > > + struct device *dev = pcie->dev; > > > > + struct mt7621_pcie_port *port; > > > > + u8 num_slots_enabled = 0; > > > > + u32 slot; > > > > + u32 val; > > > > + int err; > > > > + > > > > + /* Setup MEMWIN and IOWIN */ > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > + > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > + if (port->enabled) { > > > > + err = clk_prepare_enable(port->clk); > > > > + if (err) { > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > + return err; > > > > + } > > > > + > > > > + mt7621_pcie_enable_port(port); > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > + num_slots_enabled++; > > > > + } > > > > + } > > > > + > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > + val |= PCI_COMMAND_MASTER; > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > controller driver automatically enables PCI bus mastering, prior device > > > driver initialize itself. > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > which is used by device drivers. > > > > > > So I think this code can confuse some device drivers... > > > > I agree that we have pci_set_master() to be used in pci device driver > > code. Original controller driver set this bit for enabled slots. Since > > there is no documentation at all for the PCI in this SoC > > I see... this is really a big problem to do any driver development... For sure it is :(. > > > I have > > maintained the setting in the driver in a cleaner way. See original > > driver code and the setting here [0]. There is no other reason than > > that. I am ok with removing this from here and testing with my two > > devices that everything is still ok if having this setting in the pci > > controller driver is a real problem. > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > then compare outputs. > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > needed, so it is possible that there would be no difference in lspci > output. Thanks. I will take this into account when v2 is submitted after more review comments come :). > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > Best regards, > > Sergio Paracuellos > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > + } > > > > + > > > > + return 0; > > > > +}
Hi Pali, On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos <sergio.paracuellos@gmail.com> wrote: > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > Hi Pali, > > > > > > Thanks for your comments. > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > --- > > > > > arch/mips/pci/Makefile | 1 + > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > --- a/arch/mips/pci/Makefile > > > > > +++ b/arch/mips/pci/Makefile > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > # > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > # > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > new file mode 100644 > > > > > index 000000000000..fe1945819d25 > > > > > --- /dev/null > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > ... > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > +{ > > > > > + struct device *dev = pcie->dev; > > > > > + struct mt7621_pcie_port *port; > > > > > + u8 num_slots_enabled = 0; > > > > > + u32 slot; > > > > > + u32 val; > > > > > + int err; > > > > > + > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > + > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > + if (port->enabled) { > > > > > + err = clk_prepare_enable(port->clk); > > > > > + if (err) { > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > + return err; > > > > > + } > > > > > + > > > > > + mt7621_pcie_enable_port(port); > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > + num_slots_enabled++; > > > > > + } > > > > > + } > > > > > + > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > + val |= PCI_COMMAND_MASTER; > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > controller driver automatically enables PCI bus mastering, prior device > > > > driver initialize itself. > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > which is used by device drivers. > > > > > > > > So I think this code can confuse some device drivers... > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > code. Original controller driver set this bit for enabled slots. Since > > > there is no documentation at all for the PCI in this SoC > > > > I see... this is really a big problem to do any driver development... > > For sure it is :(. > > > > > > I have > > > maintained the setting in the driver in a cleaner way. See original > > > driver code and the setting here [0]. There is no other reason than > > > that. I am ok with removing this from here and testing with my two > > > devices that everything is still ok if having this setting in the pci > > > controller driver is a real problem. > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > then compare outputs. > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > needed, so it is possible that there would be no difference in lspci > > output. > > Thanks. I will take this into account when v2 is submitted after more > review comments come :). I have tested to remove this and check lspci -nnvv output with and without PCI_COMMAND_MASTER code and, as you pointed out, there is no difference between them. Also, both boards are working without regressions at all. So I will remove this code for next version. Thanks, Sergio Paracuellos > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > Best regards, > > > Sergio Paracuellos > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > > + } > > > > > + > > > > > + return 0; > > > > > +}
On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > Hi Pali, > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > <sergio.paracuellos@gmail.com> wrote: > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > Hi Pali, > > > > > > > > Thanks for your comments. > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > --- > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > --- a/arch/mips/pci/Makefile > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > # > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > # > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > new file mode 100644 > > > > > > index 000000000000..fe1945819d25 > > > > > > --- /dev/null > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > ... > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > +{ > > > > > > + struct device *dev = pcie->dev; > > > > > > + struct mt7621_pcie_port *port; > > > > > > + u8 num_slots_enabled = 0; > > > > > > + u32 slot; > > > > > > + u32 val; > > > > > > + int err; > > > > > > + > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > + > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > + if (port->enabled) { > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > + if (err) { > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > + return err; > > > > > > + } > > > > > > + > > > > > > + mt7621_pcie_enable_port(port); > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > + num_slots_enabled++; > > > > > > + } > > > > > > + } > > > > > > + > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > driver initialize itself. > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > which is used by device drivers. > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > code. Original controller driver set this bit for enabled slots. Since > > > > there is no documentation at all for the PCI in this SoC > > > > > > I see... this is really a big problem to do any driver development... > > > > For sure it is :(. > > > > > > > > > I have > > > > maintained the setting in the driver in a cleaner way. See original > > > > driver code and the setting here [0]. There is no other reason than > > > > that. I am ok with removing this from here and testing with my two > > > > devices that everything is still ok if having this setting in the pci > > > > controller driver is a real problem. > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > then compare outputs. > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > needed, so it is possible that there would be no difference in lspci > > > output. > > > > Thanks. I will take this into account when v2 is submitted after more > > review comments come :). > > I have tested to remove this and check lspci -nnvv output with and > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > difference between them. Also, both boards are working without > regressions at all. So I will remove this code for next version. Perfect! > Thanks, > Sergio Paracuellos > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > Best regards, > > > > Sergio Paracuellos > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked as MT specific register. But from this code for me it looks like that it just access config space of some device and therefore it could be some standard PCIe register. Just with hardcoded calculated offset. Could you provide output from lspci -nnvv? So other people could look at it and maybe we decode what is this code doing and if it is needed. > > > > > > + } > > > > > > + > > > > > > + return 0; > > > > > > +}
Hi Pali, On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > Hi Pali, > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > <sergio.paracuellos@gmail.com> wrote: > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > Hi Pali, > > > > > > > > > > Thanks for your comments. > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > --- > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > # > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > # > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > new file mode 100644 > > > > > > > index 000000000000..fe1945819d25 > > > > > > > --- /dev/null > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > ... > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > +{ > > > > > > > + struct device *dev = pcie->dev; > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > + u32 slot; > > > > > > > + u32 val; > > > > > > > + int err; > > > > > > > + > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > + > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > + if (port->enabled) { > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > + if (err) { > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > + return err; > > > > > > > + } > > > > > > > + > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > + num_slots_enabled++; > > > > > > > + } > > > > > > > + } > > > > > > > + > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > driver initialize itself. > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > which is used by device drivers. > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > For sure it is :(. > > > > > > > > > > > > I have > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > driver code and the setting here [0]. There is no other reason than > > > > > that. I am ok with removing this from here and testing with my two > > > > > devices that everything is still ok if having this setting in the pci > > > > > controller driver is a real problem. > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > then compare outputs. > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > needed, so it is possible that there would be no difference in lspci > > > > output. > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > review comments come :). > > > > I have tested to remove this and check lspci -nnvv output with and > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > difference between them. Also, both boards are working without > > regressions at all. So I will remove this code for next version. > > Perfect! > > > Thanks, > > Sergio Paracuellos > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > Best regards, > > > > > Sergio Paracuellos > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > as MT specific register. But from this code for me it looks like that it > just access config space of some device and therefore it could be some > standard PCIe register. Just with hardcoded calculated offset. > > Could you provide output from lspci -nnvv? So other people could look at > it and maybe we decode what is this code doing and if it is needed. # lspci -nnvv 00:02.0 PCI bridge [0604]: Device [0e8d:0801] (rev 01) (prog-if 00 [Normal decode]) Device tree node: /sys/firmware/devicetree/base/pcie@1e140000/pcie@2,0 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 255 Region 1: Memory at 60200000 (32-bit, non-prefetchable) [size=64K] Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff [size=4K] Memory behind bridge: 60000000-600fffff [size=1M] Prefetchable memory behind bridge: 60100000-601fffff [size=1M] Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us ClockPM- Surprise- LLActRep+ BwNot- ASPMOptComp- LnkCtl: ASPM Disabled; RCB 128 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s (ok), Width x1 (ok) TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- RootCap: CRSVisible- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP- LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- AtomicOpsCtl: ReqEn- EgressBlck- LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 RootCmd: CERptEn- NFERptEn- FERptEn- RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 Capabilities: [140 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- lspci: Unable to load libkmod resources: error -12 01:00.0 Network controller [0280]: MEDIATEK Corp. Device [14c3:7612] Subsystem: MEDIATEK Corp. Device [14c3:7612] Device tree node: /sys/firmware/devicetree/base/pcie@1e140000/pcie@2,0/wifi@0,0 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 20 Region 0: Memory at 60000000 (64-bit, non-prefetchable) [size=1M] Expansion ROM at 60100000 [virtual] [disabled] [size=64K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 unlimited ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s (ok), Width x1 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [158 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [160 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=50us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=10us Kernel driver in use: mt76x2e Best regards, Sergio Paracuellos > > > > > > > > + } > > > > > > > + > > > > > > > + return 0; > > > > > > > +}
On Wednesday 02 June 2021 14:43:53 Sergio Paracuellos wrote: > Hi Pali, > > On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > > Hi Pali, > > > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > > <sergio.paracuellos@gmail.com> wrote: > > > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > > Hi Pali, > > > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > > --- > > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > > # > > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > > # > > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > > new file mode 100644 > > > > > > > > index 000000000000..fe1945819d25 > > > > > > > > --- /dev/null > > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > > ... > > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > > +{ > > > > > > > > + struct device *dev = pcie->dev; > > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > > + u32 slot; > > > > > > > > + u32 val; > > > > > > > > + int err; > > > > > > > > + > > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > > + > > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > > + if (port->enabled) { > > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > > + if (err) { > > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > > + return err; > > > > > > > > + } > > > > > > > > + > > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > > + num_slots_enabled++; > > > > > > > > + } > > > > > > > > + } > > > > > > > > + > > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > > driver initialize itself. > > > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > > which is used by device drivers. > > > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > > > For sure it is :(. > > > > > > > > > > > > > > > I have > > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > > driver code and the setting here [0]. There is no other reason than > > > > > > that. I am ok with removing this from here and testing with my two > > > > > > devices that everything is still ok if having this setting in the pci > > > > > > controller driver is a real problem. > > > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > > then compare outputs. > > > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > > needed, so it is possible that there would be no difference in lspci > > > > > output. > > > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > > review comments come :). > > > > > > I have tested to remove this and check lspci -nnvv output with and > > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > > difference between them. Also, both boards are working without > > > regressions at all. So I will remove this code for next version. > > > > Perfect! > > > > > Thanks, > > > Sergio Paracuellos > > > > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > > > Best regards, > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > > as MT specific register. But from this code for me it looks like that it > > just access config space of some device and therefore it could be some > > standard PCIe register. Just with hardcoded calculated offset. So based on your lspci output, there is no PCIe capability register at address PCIE_FTS_NUM (0x70c), right? It seems strange to trying access capability register outside of capability list. > > Could you provide output from lspci -nnvv? So other people could look at > > it and maybe we decode what is this code doing and if it is needed. > > # lspci -nnvv > 00:02.0 PCI bridge [0604]: Device [0e8d:0801] (rev 01) (prog-if 00 > [Normal decode]) Hm... Device address is 02. But in your code is: u8 num_slots_enabled = 0; ... list_for_each_entry(port, &pcie->ports, list) { if (port->enabled) { ... num_slots_enabled++; ... } } ... for (slot = 0; slot < num_slots_enabled; slot++) { val = read_config(pcie, slot, ...); ... write_config(pcie, slot, ...); } Which means that this code writes to config space of wrong device 0 (instead of 2)! In function write_config() can be seen that second parameter specify device of BDF address for bus=0 and function=0. > Device tree node: /sys/firmware/devicetree/base/pcie@1e140000/pcie@2,0 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0 > Interrupt: pin A routed to IRQ 255 > Region 1: Memory at 60200000 (32-bit, non-prefetchable) [size=64K] > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > I/O behind bridge: 00000000-00000fff [size=4K] > Memory behind bridge: 60000000-600fffff [size=1M] > Prefetchable memory behind bridge: 60100000-601fffff [size=1M] > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- <SERR- <PERR- > BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA > PME(D0+,D1+,D2-,D3hot+,D3cold-) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ > Address: 0000000000000000 Data: 0000 > Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 > DevCap: MaxPayload 128 bytes, PhantFunc 0 > ExtTag- RBE+ > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq- > AuxPwr- TransPend- > LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, > Exit Latency L0s <512ns, L1 <64us > ClockPM- Surprise- LLActRep+ BwNot- ASPMOptComp- > LnkCtl: ASPM Disabled; RCB 128 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s (ok), Width x1 (ok) > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- > RootCap: CRSVisible- > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- > PMEIntEna- CRSVisible- > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > DevCap2: Completion Timeout: Not Supported, > TimeoutDis+ NROPrPrP- LTR- > 10BitTagComp- 10BitTagReq- OBFF Not > Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, > EmergencyPowerReductionInit- > FRS- LN System CLS Not Supported, TPHComp- > ExtTPHComp- ARIFwd- > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > LTR- OBFF Disabled, ARIFwd- > AtomicOpsCtl: ReqEn- EgressBlck- > LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- > Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, > EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB, > EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- > LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v1] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- > UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ > ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > RootCmd: CERptEn- NFERptEn- FERptEn- > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- > FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 > Capabilities: [140 v1] Virtual Channel > Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 > Arb: Fixed- WRR32- WRR64- WRR128- > Ctrl: ArbSelect=Fixed > Status: InProgress- > VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- > Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- > Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff > Status: NegoPending- InProgress- > lspci: Unable to load libkmod resources: error -12 > > 01:00.0 Network controller [0280]: MEDIATEK Corp. Device [14c3:7612] > Subsystem: MEDIATEK Corp. Device [14c3:7612] > Device tree node: > /sys/firmware/devicetree/base/pcie@1e140000/pcie@2,0/wifi@0,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0 > Interrupt: pin A routed to IRQ 20 > Region 0: Memory at 60000000 (64-bit, non-prefetchable) [size=1M] > Expansion ROM at 60100000 [virtual] [disabled] [size=64K] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA > PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ > Address: 0000000000000000 Data: 0000 > Capabilities: [70] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s > unlimited, L1 unlimited > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ > FLReset- SlotPowerLimit 0.000W > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- > AuxPwr+ TransPend- > LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, > Exit Latency L0s <2us, L1 unlimited > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s (ok), Width x1 (ok) > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ > NROPrPrP- LTR- > 10BitTagComp- 10BitTagReq- OBFF Not > Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, > EmergencyPowerReductionInit- > FRS- TPHComp- ExtTPHComp- > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > LTR- OBFF Disabled, > AtomicOpsCtl: ReqEn- > LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- > Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, > EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -3.5dB, > EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- > LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- > UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ > ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > Capabilities: [158 v1] Latency Tolerance Reporting > Max snoop latency: 0ns > Max no snoop latency: 0ns > Capabilities: [160 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ > ASPM_L1.1+ L1_PM_Substates+ > PortCommonModeRestoreTime=50us PortTPowerOnTime=10us > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > T_CommonMode=0us LTR1.2_Threshold=0ns > L1SubCtl2: T_PwrOn=10us > Kernel driver in use: mt76x2e > > Best regards, > Sergio Paracuellos > > > > > > > > > > > + } > > > > > > > > + > > > > > > > > + return 0; > > > > > > > > +}
Hi Pali, Thanks for your comments. On Fri, Jun 4, 2021 at 6:55 PM Pali Rohár <pali@kernel.org> wrote: > > On Wednesday 02 June 2021 14:43:53 Sergio Paracuellos wrote: > > Hi Pali, > > > > On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > > > Hi Pali, > > > > > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > > > <sergio.paracuellos@gmail.com> wrote: > > > > > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > > > Hi Pali, > > > > > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > > > --- > > > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > > > # > > > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > > > # > > > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > > > new file mode 100644 > > > > > > > > > index 000000000000..fe1945819d25 > > > > > > > > > --- /dev/null > > > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > > > ... > > > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > > > +{ > > > > > > > > > + struct device *dev = pcie->dev; > > > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > > > + u32 slot; > > > > > > > > > + u32 val; > > > > > > > > > + int err; > > > > > > > > > + > > > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > > > + > > > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > > > + if (port->enabled) { > > > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > > > + if (err) { > > > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > > > + return err; > > > > > > > > > + } > > > > > > > > > + > > > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > > > + num_slots_enabled++; > > > > > > > > > + } > > > > > > > > > + } > > > > > > > > > + > > > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > > > driver initialize itself. > > > > > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > > > which is used by device drivers. > > > > > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > > > > > For sure it is :(. > > > > > > > > > > > > > > > > > > I have > > > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > > > driver code and the setting here [0]. There is no other reason than > > > > > > > that. I am ok with removing this from here and testing with my two > > > > > > > devices that everything is still ok if having this setting in the pci > > > > > > > controller driver is a real problem. > > > > > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > > > then compare outputs. > > > > > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > > > needed, so it is possible that there would be no difference in lspci > > > > > > output. > > > > > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > > > review comments come :). > > > > > > > > I have tested to remove this and check lspci -nnvv output with and > > > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > > > difference between them. Also, both boards are working without > > > > regressions at all. So I will remove this code for next version. > > > > > > Perfect! > > > > > > > Thanks, > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > > > > > Best regards, > > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > > > as MT specific register. But from this code for me it looks like that it > > > just access config space of some device and therefore it could be some > > > standard PCIe register. Just with hardcoded calculated offset. > > So based on your lspci output, there is no PCIe capability register at > address PCIE_FTS_NUM (0x70c), right? It seems strange to trying access > capability register outside of capability list. This setting is also in the original driver code from mediatek SDK. See [0]. Anyway, I have tried to remove this code and test what happens with my two boards and both of them seem to work properly just by deleting this code. > > > > Could you provide output from lspci -nnvv? So other people could look at > > > it and maybe we decode what is this code doing and if it is needed. > > > > # lspci -nnvv > > 00:02.0 PCI bridge [0604]: Device [0e8d:0801] (rev 01) (prog-if 00 > > [Normal decode]) > > Hm... Device address is 02. But in your code is: > > u8 num_slots_enabled = 0; > ... > list_for_each_entry(port, &pcie->ports, list) { > if (port->enabled) { > ... > num_slots_enabled++; > ... > } > } > ... > for (slot = 0; slot < num_slots_enabled; slot++) { > val = read_config(pcie, slot, ...); > ... > write_config(pcie, slot, ...); > } > > Which means that this code writes to config space of wrong device 0 > (instead of 2)! In function write_config() can be seen that second > parameter specify device of BDF address for bus=0 and function=0. Bridge enumeration depends on a virtual bridge register configuration. But at the end devices connected to the bridge are enumerated as 01:00.0, 02:00.0 and 03:00.0. So in this case the phy used is the one for "pcie2" (00:02.0) and the device connected to it is 01:00.0. For example a board using all the virtual bridges will get an output similar to: [ 16.487166] mt7621-pci 1e140000.pcie: host bridge /pcie@1e140000 ranges: [ 16.500627] mt7621-pci 1e140000.pcie: No bus range found for /pcie@1e140000, using [bus 00-ff] [ 16.518212] mt7621-pci 1e140000.pcie: MEM 0x0060000000..0x006fffffff -> 0x0000000000 [ 16.534531] mt7621-pci 1e140000.pcie: IO 0x001e160000..0x001e16ffff -> 0x0000000000 [ 16.786498] mt7621-pci 1e140000.pcie: PCIE0 enabled [ 16.796220] mt7621-pci 1e140000.pcie: PCIE1 enabled [ 16.805943] mt7621-pci 1e140000.pcie: PCIE2 enabled [ 16.815664] mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002 [ 16.834398] mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00 [ 16.847098] pci_bus 0000:00: root bus resource [io 0x1e160000-0x1e16ffff] [ 16.860806] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff] [ 16.874504] pci_bus 0000:00: root bus resource [bus 00-ff] [ 16.885441] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff] (bus address [0x00000000-0x0fffffff]) [ 16.905773] pci 0000:00:00.0: [0e8d:0801] type 01 class 0x060400 [ 16.917772] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x7fffffff] [ 16.930260] pci 0000:00:00.0: reg 0x14: initial BAR value 0x00000000 invalid [ 16.944304] pci 0000:00:00.0: reg 0x14: [mem size 0x00010000] [ 16.955841] pci 0000:00:00.0: supports D1 [ 16.963837] pci 0000:00:00.0: PME# supported from D0 D1 D3hot [ 16.975747] pci 0000:00:01.0: [0e8d:0801] type 01 class 0x060400 [ 16.987772] pci 0000:00:01.0: reg 0x10: [mem 0x00000000-0x7fffffff] [ 17.000273] pci 0000:00:01.0: reg 0x14: initial BAR value 0x00000000 invalid [ 17.014311] pci 0000:00:01.0: reg 0x14: [mem size 0x00010000] [ 17.025838] pci 0000:00:01.0: supports D1 [ 17.033828] pci 0000:00:01.0: PME# supported from D0 D1 D3hot [ 17.045699] pci 0000:00:02.0: [0e8d:0801] type 01 class 0x060400 [ 17.057726] pci 0000:00:02.0: reg 0x10: [mem 0x00000000-0x7fffffff] [ 17.070218] pci 0000:00:02.0: reg 0x14: initial BAR value 0x00000000 invalid [ 17.084260] pci 0000:00:02.0: reg 0x14: [mem size 0x00010000] [ 17.095788] pci 0000:00:02.0: supports D1 [ 17.103785] pci 0000:00:02.0: PME# supported from D0 D1 D3hot [ 17.116598] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring [ 17.132566] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring [ 17.148514] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring [ 17.164739] pci 0000:01:00.0: [1b21:0611] type 00 class 0x010185 [ 17.176775] pci 0000:01:00.0: reg 0x10: [io 0x0000-0x0007] [ 17.187892] pci 0000:01:00.0: reg 0x14: [io 0x0000-0x0003] [ 17.199009] pci 0000:01:00.0: reg 0x18: [io 0x0000-0x0007] [ 17.210130] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x0003] [ 17.221246] pci 0000:01:00.0: reg 0x20: [io 0x0000-0x000f] [ 17.232357] pci 0000:01:00.0: reg 0x24: initial BAR value 0x00000000 invalid [ 17.246411] pci 0000:01:00.0: reg 0x24: [mem size 0x00000200] [ 17.258031] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:00.0 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link) [ 17.317563] pci 0000:00:00.0: PCI bridge to [bus 01-ff] [ 17.328033] pci 0000:00:00.0: bridge window [io 0x0000-0x0fff] [ 17.340179] pci 0000:00:00.0: bridge window [mem 0x60000000-0x600fffff] [ 17.353703] pci 0000:00:00.0: bridge window [mem 0x60000000-0x600fffff pref] [ 17.368097] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 [ 17.381550] pci 0000:02:00.0: [1b21:0611] type 00 class 0x010185 [ 17.393573] pci 0000:02:00.0: reg 0x10: [io 0x0000-0x0007] [ 17.404694] pci 0000:02:00.0: reg 0x14: [io 0x0000-0x0003] [ 17.415804] pci 0000:02:00.0: reg 0x18: [io 0x0000-0x0007] [ 17.426916] pci 0000:02:00.0: reg 0x1c: [io 0x0000-0x0003] [ 17.438040] pci 0000:02:00.0: reg 0x20: [io 0x0000-0x000f] [ 17.449148] pci 0000:02:00.0: reg 0x24: initial BAR value 0x00000000 invalid [ 17.463193] pci 0000:02:00.0: reg 0x24: [mem size 0x00000200] [ 17.474803] pci 0000:02:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:01.0 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link) [ 17.527554] pci 0000:00:01.0: PCI bridge to [bus 02-ff] [ 17.538014] pci 0000:00:01.0: bridge window [io 0x0000-0x0fff] [ 17.550159] pci 0000:00:01.0: bridge window [mem 0x60000000-0x600fffff] [ 17.563682] pci 0000:00:01.0: bridge window [mem 0x60000000-0x600fffff pref] [ 17.578078] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02 [ 17.591529] pci 0000:03:00.0: [1b21:0611] type 00 class 0x010185 [ 17.603545] pci 0000:03:00.0: reg 0x10: [io 0x0000-0x0007] [ 17.614665] pci 0000:03:00.0: reg 0x14: [io 0x0000-0x0003] [ 17.625775] pci 0000:03:00.0: reg 0x18: [io 0x0000-0x0007] [ 17.636887] pci 0000:03:00.0: reg 0x1c: [io 0x0000-0x0003] [ 17.648009] pci 0000:03:00.0: reg 0x20: [io 0x0000-0x000f] [ 17.659119] pci 0000:03:00.0: reg 0x24: initial BAR value 0x00000000 invalid [ 17.673162] pci 0000:03:00.0: reg 0x24: [mem size 0x00000200] [ 17.684777] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:02.0 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link) [ 17.737561] pci 0000:00:02.0: PCI bridge to [bus 03-ff] [ 17.748022] pci 0000:00:02.0: bridge window [io 0x0000-0x0fff] [ 17.760167] pci 0000:00:02.0: bridge window [mem 0x60000000-0x600fffff] [ 17.773690] pci 0000:00:02.0: bridge window [mem 0x60000000-0x600fffff pref] [ 17.788085] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03 [ 17.801341] pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000] [ 17.814518] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000] [ 17.828392] pci 0000:00:01.0: BAR 0: no space for [mem size 0x80000000] [ 17.841571] pci 0000:00:01.0: BAR 0: failed to assign [mem size 0x80000000] [ 17.855443] pci 0000:00:02.0: BAR 0: no space for [mem size 0x80000000] [ 17.868626] pci 0000:00:02.0: BAR 0: failed to assign [mem size 0x80000000] [ 17.882502] pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x600fffff] [ 17.896028] pci 0000:00:00.0: BAR 9: assigned [mem 0x60100000-0x601fffff pref] [ 17.910419] pci 0000:00:01.0: BAR 8: assigned [mem 0x60200000-0x602fffff] [ 17.923951] pci 0000:00:01.0: BAR 9: assigned [mem 0x60300000-0x603fffff pref] [ 17.938353] pci 0000:00:02.0: BAR 8: assigned [mem 0x60400000-0x604fffff] [ 17.951879] pci 0000:00:02.0: BAR 9: assigned [mem 0x60500000-0x605fffff pref] [ 17.966273] pci 0000:00:00.0: BAR 1: assigned [mem 0x60600000-0x6060ffff] [ 17.979809] pci 0000:00:01.0: BAR 1: assigned [mem 0x60610000-0x6061ffff] [ 17.993341] pci 0000:00:02.0: BAR 1: assigned [mem 0x60620000-0x6062ffff] [ 18.006877] pci 0000:00:00.0: BAR 7: assigned [io 0x1e160000-0x1e160fff] [ 18.020408] pci 0000:00:01.0: BAR 7: assigned [io 0x1e161000-0x1e161fff] [ 18.033932] pci 0000:00:02.0: BAR 7: assigned [io 0x1e162000-0x1e162fff] [ 18.047472] pci 0000:01:00.0: BAR 5: assigned [mem 0x60000000-0x600001ff] [ 18.061005] pci 0000:01:00.0: BAR 4: assigned [io 0x1e160000-0x1e16000f] [ 18.074540] pci 0000:01:00.0: BAR 0: assigned [io 0x1e160010-0x1e160017] [ 18.088070] pci 0000:01:00.0: BAR 2: assigned [io 0x1e160018-0x1e16001f] [ 18.101606] pci 0000:01:00.0: BAR 1: assigned [io 0x1e160020-0x1e160023] [ 18.115139] pci 0000:01:00.0: BAR 3: assigned [io 0x1e160024-0x1e160027] [ 18.128676] pci 0000:00:00.0: PCI bridge to [bus 01] [ 18.138577] pci 0000:00:00.0: bridge window [io 0x1e160000-0x1e160fff] [ 18.152100] pci 0000:00:00.0: bridge window [mem 0x60000000-0x600fffff] [ 18.165627] pci 0000:00:00.0: bridge window [mem 0x60100000-0x601fffff pref] [ 18.180030] pci 0000:02:00.0: BAR 5: assigned [mem 0x60200000-0x602001ff] [ 18.193566] pci 0000:02:00.0: BAR 4: assigned [io 0x1e161000-0x1e16100f] [ 18.207110] pci 0000:02:00.0: BAR 0: assigned [io 0x1e161010-0x1e161017] [ 18.220648] pci 0000:02:00.0: BAR 2: assigned [io 0x1e161018-0x1e16101f] [ 18.234183] pci 0000:02:00.0: BAR 1: assigned [io 0x1e161020-0x1e161023] [ 18.247723] pci 0000:02:00.0: BAR 3: assigned [io 0x1e161024-0x1e161027] [ 18.261257] pci 0000:00:01.0: PCI bridge to [bus 02] [ 18.271165] pci 0000:00:01.0: bridge window [io 0x1e161000-0x1e161fff] [ 18.284695] pci 0000:00:01.0: bridge window [mem 0x60200000-0x602fffff] [ 18.298225] pci 0000:00:01.0: bridge window [mem 0x60300000-0x603fffff pref] [ 18.312630] pci 0000:03:00.0: BAR 5: assigned [mem 0x60400000-0x604001ff] [ 18.326166] pci 0000:03:00.0: BAR 4: assigned [io 0x1e162000-0x1e16200f] [ 18.339702] pci 0000:03:00.0: BAR 0: assigned [io 0x1e162010-0x1e162017] [ 18.353237] pci 0000:03:00.0: BAR 2: assigned [io 0x1e162018-0x1e16201f] [ 18.366775] pci 0000:03:00.0: BAR 1: assigned [io 0x1e162020-0x1e162023] [ 18.380311] pci 0000:03:00.0: BAR 3: assigned [io 0x1e162024-0x1e162027] [ 18.393841] pci 0000:00:02.0: PCI bridge to [bus 03] [ 18.403740] pci 0000:00:02.0: bridge window [io 0x1e162000-0x1e162fff] [ 18.417270] pci 0000:00:02.0: bridge window [mem 0x60400000-0x604fffff] [ 18.430801] pci 0000:00:02.0: bridge window [mem 0x60500000-0x605fffff pref] [ 18.445529] ahci 0000:01:00.0: version 3.0 [ 18.445559] pci 0000:00:00.0: enabling device (0000 -> 0003) [ 18.456853] ahci 0000:01:00.0: enabling device (0000 -> 0003) [ 18.468455] ahci 0000:01:00.0: SSS flag set, parallel bus scan disabled [ 18.481700] ahci 0000:01:00.0: AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl IDE mode [ 18.497662] ahci 0000:01:00.0: flags: 64bit ncq sntf stag led clo pmp pio slum part ccc sxs [ 18.516777] scsi host0: ahci [ 18.523629] scsi host1: ahci [ 18.529829] ata1: SATA max UDMA/133 abar m512@0x60000000 port 0x60000100 irq 22 [ 18.544447] ata2: SATA max UDMA/133 abar m512@0x60000000 port 0x60000180 irq 22 [ 18.559465] pci 0000:00:01.0: enabling device (0000 -> 0003) [ 18.570786] ahci 0000:02:00.0: enabling device (0000 -> 0003) [ 18.582414] ahci 0000:02:00.0: SSS flag set, parallel bus scan disabled [ 18.595665] ahci 0000:02:00.0: AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl IDE mode [ 18.611614] ahci 0000:02:00.0: flags: 64bit ncq sntf stag led clo pmp pio slum part ccc sxs [ 18.631053] scsi host2: ahci [ 18.637983] scsi host3: ahci [ 18.644138] ata3: SATA max UDMA/133 abar m512@0x60200000 port 0x60200100 irq 23 [ 18.658792] ata4: SATA max UDMA/133 abar m512@0x60200000 port 0x60200180 irq 23 [ 18.673827] pci 0000:00:02.0: enabling device (0000 -> 0003) [ 18.685151] ahci 0000:03:00.0: enabling device (0000 -> 0003) [ 18.696782] ahci 0000:03:00.0: SSS flag set, parallel bus scan disabled [ 18.710025] ahci 0000:03:00.0: AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl IDE mode [ 18.725972] ahci 0000:03:00.0: flags: 64bit ncq sntf stag led clo pmp pio slum part ccc sxs And you are totally right, the setting is writing in the wrong place. I changed the device tree and the way interrupts are mapped to avoid using a custom 'map_irq' function [1]. Before that commit the pci virtual bridge register was reordering the buses enumeration depending on link status, so there I should also properly rewrite the code in question. I can rewrite the code to read and write config properly using the slot moving the code into 'mt7621_pcie_enable_port' as follows: static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port) { struct mt7621_pcie *pcie = port->pcie; u32 slot = port->slot; u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT); u32 val; /* enable pcie interrupt */ val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR); val |= PCIE_PORT_INT_EN(slot); /* map 2G DDR region */ pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE, offset + RALINK_PCI_BAR0SETUP_ADDR); /* configure class code and revision ID */ pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID, offset + RALINK_PCI_CLASS); /* configure RC FTS number to 250 when it leaves L0s */ val = read_config(pcie, slot, PCIE_FTS_NUM); val &= ~PCIE_FTS_NUM_MASK; val |= PCIE_FTS_NUM_L0(0x50); write_config(pcie, slot, PCIE_FTS_NUM, val); } static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) { struct device *dev = pcie->dev; struct mt7621_pcie_port *port; int err; /* Setup MEMWIN and IOWIN */ pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); list_for_each_entry(port, &pcie->ports, list) { if (port->enabled) { err = clk_prepare_enable(port->clk); if (err) { dev_err(dev, "enabling clk pcie%d\n", port->slot); return err; } mt7621_pcie_enable_port(port); dev_info(dev, "PCIE%d enabled\n", port->slot); } } return 0; } Or just delete the setting and the read and write config functions since they are not being used in any other place. My two boards work without this setting but I don't know about other boards. What do you think? Best regards, Sergio Paracuellos [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n663 [1]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/commit/drivers/staging/mt7621-pci?h=staging-testing&id=b99cc3a2b6b62cf994acac5cced03298d9908c9b > > > Device tree node: /sys/firmware/devicetree/base/pcie@1e140000/pcie@2,0 > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > > ParErr- Stepping- SERR- FastB2B- DisINTx- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > > <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0 > > Interrupt: pin A routed to IRQ 255 > > Region 1: Memory at 60200000 (32-bit, non-prefetchable) [size=64K] > > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > > I/O behind bridge: 00000000-00000fff [size=4K] > > Memory behind bridge: 60000000-600fffff [size=1M] > > Prefetchable memory behind bridge: 60100000-601fffff [size=1M] > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- > > <TAbort- <MAbort- <SERR- <PERR- > > BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA > > PME(D0+,D1+,D2-,D3hot+,D3cold-) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ > > Address: 0000000000000000 Data: 0000 > > Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 > > DevCap: MaxPayload 128 bytes, PhantFunc 0 > > ExtTag- RBE+ > > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq- > > AuxPwr- TransPend- > > LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, > > Exit Latency L0s <512ns, L1 <64us > > ClockPM- Surprise- LLActRep+ BwNot- ASPMOptComp- > > LnkCtl: ASPM Disabled; RCB 128 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s (ok), Width x1 (ok) > > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- > > RootCap: CRSVisible- > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- > > PMEIntEna- CRSVisible- > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > DevCap2: Completion Timeout: Not Supported, > > TimeoutDis+ NROPrPrP- LTR- > > 10BitTagComp- 10BitTagReq- OBFF Not > > Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, > > EmergencyPowerReductionInit- > > FRS- LN System CLS Not Supported, TPHComp- > > ExtTPHComp- ARIFwd- > > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > > LTR- OBFF Disabled, ARIFwd- > > AtomicOpsCtl: ReqEn- EgressBlck- > > LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- > > Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, > > EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -6dB, > > EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- > > LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v1] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- > > UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- > > AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > > AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ > > ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > RootCmd: CERptEn- NFERptEn- FERptEn- > > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- > > FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 > > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 > > Capabilities: [140 v1] Virtual Channel > > Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 > > Arb: Fixed- WRR32- WRR64- WRR128- > > Ctrl: ArbSelect=Fixed > > Status: InProgress- > > VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- > > Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- > > Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff > > Status: NegoPending- InProgress- > > lspci: Unable to load libkmod resources: error -12 > > > > 01:00.0 Network controller [0280]: MEDIATEK Corp. Device [14c3:7612] > > Subsystem: MEDIATEK Corp. Device [14c3:7612] > > Device tree node: > > /sys/firmware/devicetree/base/pcie@1e140000/pcie@2,0/wifi@0,0 > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > > ParErr- Stepping- SERR- FastB2B- DisINTx- > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > > <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0 > > Interrupt: pin A routed to IRQ 20 > > Region 0: Memory at 60000000 (64-bit, non-prefetchable) [size=1M] > > Expansion ROM at 60100000 [virtual] [disabled] [size=64K] > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA > > PME(D0+,D1-,D2-,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ > > Address: 0000000000000000 Data: 0000 > > Capabilities: [70] Express (v2) Endpoint, MSI 00 > > DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s > > unlimited, L1 unlimited > > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ > > FLReset- SlotPowerLimit 0.000W > > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- > > AuxPwr+ TransPend- > > LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, > > Exit Latency L0s <2us, L1 unlimited > > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s (ok), Width x1 (ok) > > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ > > NROPrPrP- LTR- > > 10BitTagComp- 10BitTagReq- OBFF Not > > Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, > > EmergencyPowerReductionInit- > > FRS- TPHComp- ExtTPHComp- > > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > > LTR- OBFF Disabled, > > AtomicOpsCtl: ReqEn- > > LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- > > Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, > > EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -3.5dB, > > EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- > > LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v2] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > > UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- > > UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > > AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > > AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ > > ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > > Capabilities: [158 v1] Latency Tolerance Reporting > > Max snoop latency: 0ns > > Max no snoop latency: 0ns > > Capabilities: [160 v1] L1 PM Substates > > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ > > ASPM_L1.1+ L1_PM_Substates+ > > PortCommonModeRestoreTime=50us PortTPowerOnTime=10us > > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > > T_CommonMode=0us LTR1.2_Threshold=0ns > > L1SubCtl2: T_PwrOn=10us > > Kernel driver in use: mt76x2e > > > > Best regards, > > Sergio Paracuellos > > > > > > > > > > > > > > + } > > > > > > > > > + > > > > > > > > > + return 0; > > > > > > > > > +}
On Fri, Jun 04, 2021 at 06:55:25PM +0200, Pali Rohár wrote: > On Wednesday 02 June 2021 14:43:53 Sergio Paracuellos wrote: > > Hi Pali, > > > > On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > > > Hi Pali, > > > > > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > > > <sergio.paracuellos@gmail.com> wrote: > > > > > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > > > Hi Pali, > > > > > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > > > --- > > > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > > > # > > > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > > > # > > > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > > > new file mode 100644 > > > > > > > > > index 000000000000..fe1945819d25 > > > > > > > > > --- /dev/null > > > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > > > ... > > > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > > > +{ > > > > > > > > > + struct device *dev = pcie->dev; > > > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > > > + u32 slot; > > > > > > > > > + u32 val; > > > > > > > > > + int err; > > > > > > > > > + > > > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > > > + > > > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > > > + if (port->enabled) { > > > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > > > + if (err) { > > > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > > > + return err; > > > > > > > > > + } > > > > > > > > > + > > > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > > > + num_slots_enabled++; > > > > > > > > > + } > > > > > > > > > + } > > > > > > > > > + > > > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > > > driver initialize itself. > > > > > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > > > which is used by device drivers. > > > > > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > > > > > For sure it is :(. > > > > > > > > > > > > > > > > > > I have > > > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > > > driver code and the setting here [0]. There is no other reason than > > > > > > > that. I am ok with removing this from here and testing with my two > > > > > > > devices that everything is still ok if having this setting in the pci > > > > > > > controller driver is a real problem. > > > > > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > > > then compare outputs. > > > > > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > > > needed, so it is possible that there would be no difference in lspci > > > > > > output. > > > > > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > > > review comments come :). > > > > > > > > I have tested to remove this and check lspci -nnvv output with and > > > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > > > difference between them. Also, both boards are working without > > > > regressions at all. So I will remove this code for next version. > > > > > > Perfect! > > > > > > > Thanks, > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > > > > > Best regards, > > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > > > as MT specific register. But from this code for me it looks like that it > > > just access config space of some device and therefore it could be some > > > standard PCIe register. Just with hardcoded calculated offset. > > So based on your lspci output, there is no PCIe capability register at > address PCIE_FTS_NUM (0x70c), right? It seems strange to trying access > capability register outside of capability list. Looks like a DW PCIe port logic register: drivers/pci/controller/dwc/pcie-designware.h-/* Synopsys-specific PCIe configuration registers */ drivers/pci/controller/dwc/pcie-designware.h:#define PCIE_PORT_AFR 0x70C drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_N_FTS_MASK GENMASK(15, 8) drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_N_FTS(n) FIELD_PREP(PORT_AFR_N_FTS_MASK, n) drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_CC_N_FTS_MASK GENMASK(23, 16) drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_CC_N_FTS(n) FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, n) drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_ENTER_ASPM BIT(30) drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L0S_ENTRANCE_LAT_SHIFT 24 drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L0S_ENTRANCE_LAT_MASK GENMASK(26, 24) drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L1_ENTRANCE_LAT_SHIFT 27 drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L1_ENTRANCE_LAT_MASK GENMASK(29, 27) Rob
On Sat, May 15, 2021 at 02:40:53PM +0200, Sergio Paracuellos wrote: > This patch adds a driver for the PCIe controller of MT7621 SoC. > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > --- > arch/mips/pci/Makefile | 1 + > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > arch/mips/ralink/Kconfig | 9 +- > 3 files changed, 633 insertions(+), 1 deletion(-) > create mode 100644 arch/mips/pci/pci-mt7621.c > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > index f3eecc065e5c..178c550739c4 100644 > --- a/arch/mips/pci/Makefile > +++ b/arch/mips/pci/Makefile > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > # > # These are still pretty much in the old state, watch, go blind. > # > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > new file mode 100644 > index 000000000000..fe1945819d25 > --- /dev/null > +++ b/arch/mips/pci/pci-mt7621.c > @@ -0,0 +1,624 @@ > +// SPDX-License-Identifier: GPL-2.0+ > +/* > + * BRIEF MODULE DESCRIPTION > + * PCI init for Ralink RT2880 solution > + * > + * Copyright 2007 Ralink Inc. (bruce_chang@ralinktech.com.tw) > + * > + * May 2007 Bruce Chang > + * Initial Release > + * > + * May 2009 Bruce Chang > + * support RT2880/RT3883 PCIe > + * > + * May 2011 Bruce Chang > + * support RT6855/MT7620 PCIe > + */ > + > +#include <linux/bitops.h> > +#include <linux/clk.h> > +#include <linux/delay.h> > +#include <linux/gpio/consumer.h> > +#include <linux/module.h> > +#include <linux/of.h> > +#include <linux/of_address.h> > +#include <linux/of_pci.h> > +#include <linux/of_platform.h> > +#include <linux/pci.h> > +#include <linux/phy/phy.h> > +#include <linux/platform_device.h> > +#include <linux/reset.h> > +#include <linux/sys_soc.h> > + > +/* MediaTek specific configuration registers */ > +#define PCIE_FTS_NUM 0x70c > +#define PCIE_FTS_NUM_MASK GENMASK(15, 8) > +#define PCIE_FTS_NUM_L0(x) (((x) & 0xff) << 8) > + > +/* Host-PCI bridge registers */ > +#define RALINK_PCI_PCICFG_ADDR 0x0000 > +#define RALINK_PCI_PCIMSK_ADDR 0x000C > +#define RALINK_PCI_CONFIG_ADDR 0x0020 > +#define RALINK_PCI_CONFIG_DATA 0x0024 > +#define RALINK_PCI_MEMBASE 0x0028 > +#define RALINK_PCI_IOBASE 0x002C > + > +/* PCIe RC control registers */ > +#define MT7621_PCIE_OFFSET 0x2000 > +#define MT7621_NEXT_PORT 0x1000 > + > +#define RALINK_PCI_BAR0SETUP_ADDR 0x0010 Standard BAR0 register? > +#define RALINK_PCI_ID 0x0030 > +#define RALINK_PCI_CLASS 0x0034 > +#define RALINK_PCI_SUBID 0x0038 > +#define RALINK_PCI_STATUS 0x0050 > + > +/* Some definition values */ > +#define PCIE_REVISION_ID BIT(0) > +#define PCIE_CLASS_CODE (0x60400 << 8) > +#define PCIE_BAR_MAP_MAX GENMASK(30, 16) > +#define PCIE_BAR_ENABLE BIT(0) > +#define PCIE_PORT_INT_EN(x) BIT(20 + (x)) > +#define PCIE_PORT_LINKUP BIT(0) > + > +#define PERST_DELAY_MS 100 > + > +/** > + * struct mt7621_pcie_port - PCIe port information > + * @base: I/O mapped register base > + * @list: port list > + * @pcie: pointer to PCIe host info > + * @clk: pointer to the port clock gate > + * @phy: pointer to PHY control block > + * @pcie_rst: pointer to port reset control > + * @gpio_rst: gpio reset > + * @slot: port slot > + * @enabled: indicates if port is enabled > + */ > +struct mt7621_pcie_port { > + void __iomem *base; > + struct list_head list; > + struct mt7621_pcie *pcie; > + struct clk *clk; > + struct phy *phy; > + struct reset_control *pcie_rst; > + struct gpio_desc *gpio_rst; > + u32 slot; > + bool enabled; > +}; > + > +/** > + * struct mt7621_pcie - PCIe host information > + * @base: IO Mapped Register Base > + * @io: IO resource > + * @mem: pointer to non-prefetchable memory resource > + * @dev: Pointer to PCIe device > + * @io_map_base: virtual memory base address for io > + * @ports: pointer to PCIe port information > + * @resets_inverted: depends on chip revision > + * reset lines are inverted. > + */ > +struct mt7621_pcie { > + void __iomem *base; > + struct device *dev; > + struct resource io; > + struct resource *mem; > + unsigned long io_map_base; These are all stored in the host bridge struct, no need for you to store them. > + struct list_head ports; A list is kind of an overkill for 3 entries and you know how many ports. Just embed an array of struct mt7621_pcie_port. Then you only need 1 alloc. > + bool resets_inverted; > +}; > + > +static inline u32 pcie_read(struct mt7621_pcie *pcie, u32 reg) > +{ > + return readl(pcie->base + reg); Can use _relaxed variants here and through out. > +} > + > +static inline void pcie_write(struct mt7621_pcie *pcie, u32 val, u32 reg) > +{ > + writel(val, pcie->base + reg); > +} > + > +static inline void pcie_rmw(struct mt7621_pcie *pcie, u32 reg, u32 clr, u32 set) > +{ > + u32 val = readl(pcie->base + reg); > + > + val &= ~clr; > + val |= set; > + writel(val, pcie->base + reg); > +} > + > +static inline u32 pcie_port_read(struct mt7621_pcie_port *port, u32 reg) > +{ > + return readl(port->base + reg); > +} > + > +static inline void pcie_port_write(struct mt7621_pcie_port *port, > + u32 val, u32 reg) > +{ > + writel(val, port->base + reg); > +} > + > +static inline u32 mt7621_pci_get_cfgaddr(unsigned int bus, unsigned int slot, > + unsigned int func, unsigned int where) > +{ > + return (((where & 0xF00) >> 8) << 24) | (bus << 16) | (slot << 11) | > + (func << 8) | (where & 0xfc) | 0x80000000; > +} > + > +static void __iomem *mt7621_pcie_map_bus(struct pci_bus *bus, > + unsigned int devfn, int where) > +{ > + struct mt7621_pcie *pcie = bus->sysdata; > + u32 address = mt7621_pci_get_cfgaddr(bus->number, PCI_SLOT(devfn), > + PCI_FUNC(devfn), where); > + > + writel(address, pcie->base + RALINK_PCI_CONFIG_ADDR); > + > + return pcie->base + RALINK_PCI_CONFIG_DATA + (where & 3); > +} > + > +struct pci_ops mt7621_pci_ops = { > + .map_bus = mt7621_pcie_map_bus, > + .read = pci_generic_config_read, > + .write = pci_generic_config_write, > +}; > + > +static u32 read_config(struct mt7621_pcie *pcie, unsigned int dev, u32 reg) > +{ > + u32 address = mt7621_pci_get_cfgaddr(0, dev, 0, reg); > + > + pcie_write(pcie, address, RALINK_PCI_CONFIG_ADDR); > + return pcie_read(pcie, RALINK_PCI_CONFIG_DATA); > +} > + > +static void write_config(struct mt7621_pcie *pcie, unsigned int dev, > + u32 reg, u32 val) > +{ > + u32 address = mt7621_pci_get_cfgaddr(0, dev, 0, reg); > + > + pcie_write(pcie, address, RALINK_PCI_CONFIG_ADDR); > + pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA); > +} > + > +static inline void mt7621_rst_gpio_pcie_assert(struct mt7621_pcie_port *port) > +{ > + if (port->gpio_rst) Don't need the if. gpiod_set_value should work with NULL. > + gpiod_set_value(port->gpio_rst, 1); > +} > + > +static inline void mt7621_rst_gpio_pcie_deassert(struct mt7621_pcie_port *port) > +{ > + if (port->gpio_rst) > + gpiod_set_value(port->gpio_rst, 0); > +} > + > +static inline bool mt7621_pcie_port_is_linkup(struct mt7621_pcie_port *port) > +{ > + return (pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) != 0; > +} > + > +static inline void mt7621_control_assert(struct mt7621_pcie_port *port) > +{ > + struct mt7621_pcie *pcie = port->pcie; > + > + if (pcie->resets_inverted) > + reset_control_assert(port->pcie_rst); > + else > + reset_control_deassert(port->pcie_rst); > +} > + > +static inline void mt7621_control_deassert(struct mt7621_pcie_port *port) > +{ > + struct mt7621_pcie *pcie = port->pcie; > + > + if (pcie->resets_inverted) > + reset_control_deassert(port->pcie_rst); > + else > + reset_control_assert(port->pcie_rst); > +} > + > +static void setup_cm_memory_region(struct mt7621_pcie *pcie) > +{ > + struct resource *mem_resource = pcie->mem; > + struct device *dev = pcie->dev; > + resource_size_t mask; > + > + if (mips_cps_numiocu(0)) { > + /* > + * FIXME: hardware doesn't accept mask values with 1s after > + * 0s (e.g. 0xffef), so it would be great to warn if that's > + * about to happen > + */ > + mask = ~(mem_resource->end - mem_resource->start); > + > + write_gcr_reg1_base(mem_resource->start); > + write_gcr_reg1_mask(mask | CM_GCR_REGn_MASK_CMTGT_IOCU0); > + dev_info(dev, "PCI coherence region base: 0x%08llx, mask/settings: 0x%08llx\n", > + (unsigned long long)read_gcr_reg1_base(), > + (unsigned long long)read_gcr_reg1_mask()); > + } > +} > + > +static int mt7621_pci_parse_request_of_pci_ranges(struct pci_host_bridge *host) > +{ > + struct mt7621_pcie *pcie = pci_host_bridge_priv(host); > + struct device *dev = pcie->dev; > + struct device_node *node = dev->of_node; > + struct of_pci_range_parser parser; > + struct resource_entry *entry; > + struct of_pci_range range; > + LIST_HEAD(res); > + > + if (of_pci_range_parser_init(&parser, node)) { > + dev_err(dev, "missing \"ranges\" property\n"); > + return -EINVAL; > + } > + > + /* > + * IO_SPACE_LIMIT for MIPS is 0xffff but this platform uses IO at > + * upper address 0x001e160000. of_pci_range_to_resource does not work I think that's normal... > + * well for MIPS platforms that don't define PCI_IOBASE, so set the IO > + * resource manually instead. Can't this be fixed? > + */ > + for_each_of_pci_range(&parser, &range) { > + switch (range.flags & IORESOURCE_TYPE_BITS) { The core code already parses ranges for you. Try not to do it again. (Use the resource instead) > + case IORESOURCE_IO: > + pcie->io_map_base = > + (unsigned long)ioremap(range.cpu_addr, > + range.size); > + pcie->io.name = node->full_name; > + pcie->io.flags = range.flags; > + pcie->io.start = range.cpu_addr; > + pcie->io.end = range.cpu_addr + range.size - 1; > + pcie->io.parent = pcie->io.child = pcie->io.sibling = NULL; > + set_io_port_base(pcie->io_map_base); > + break; > + } > + } > + > + entry = resource_list_first_type(&host->windows, IORESOURCE_MEM); > + if (!entry) { > + dev_err(dev, "Cannot get memory resource"); > + return -EINVAL; > + } > + > + pcie->mem = entry->res; > + pci_add_resource(&res, &pcie->io); > + pci_add_resource(&res, entry->res); > + list_splice_init(&res, &host->windows); This should already be done for you. > + > + return 0; > +} > + > +static int mt7621_pcie_parse_port(struct mt7621_pcie *pcie, > + int slot) > +{ > + struct mt7621_pcie_port *port; > + struct device *dev = pcie->dev; > + struct platform_device *pdev = to_platform_device(dev); > + char name[10]; > + > + port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL); > + if (!port) > + return -ENOMEM; > + > + port->base = devm_platform_ioremap_resource(pdev, slot + 1); > + if (IS_ERR(port->base)) > + return PTR_ERR(port->base); > + > + snprintf(name, sizeof(name), "pcie%d", slot); > + port->clk = devm_clk_get(dev, name); > + if (IS_ERR(port->clk)) { > + dev_err(dev, "failed to get pcie%d clock\n", slot); > + return PTR_ERR(port->clk); > + } > + > + snprintf(name, sizeof(name), "pcie%d", slot); > + port->pcie_rst = devm_reset_control_get_exclusive(dev, name); > + if (PTR_ERR(port->pcie_rst) == -EPROBE_DEFER) { > + dev_err(dev, "failed to get pcie%d reset control\n", slot); > + return PTR_ERR(port->pcie_rst); > + } > + > + snprintf(name, sizeof(name), "pcie-phy%d", slot); > + port->phy = devm_phy_get(dev, name); > + if (IS_ERR(port->phy) && slot != 1) > + return PTR_ERR(port->phy); > + > + port->gpio_rst = devm_gpiod_get_index_optional(dev, "reset", slot, > + GPIOD_OUT_LOW); > + if (IS_ERR(port->gpio_rst)) { > + dev_err(dev, "Failed to get GPIO for PCIe%d\n", slot); > + return PTR_ERR(port->gpio_rst); > + } > + > + port->slot = slot; > + port->pcie = pcie; > + > + INIT_LIST_HEAD(&port->list); > + list_add_tail(&port->list, &pcie->ports); > + > + return 0; > +} > + > +static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie) > +{ > + struct device *dev = pcie->dev; > + struct platform_device *pdev = to_platform_device(dev); > + struct device_node *node = dev->of_node, *child; > + int err; > + > + pcie->base = devm_platform_ioremap_resource(pdev, 0); > + if (IS_ERR(pcie->base)) > + return PTR_ERR(pcie->base); > + > + for_each_available_child_of_node(node, child) { > + int slot; > + > + err = of_pci_get_devfn(child); > + if (err < 0) { > + of_node_put(child); > + dev_err(dev, "failed to parse devfn: %d\n", err); > + return err; > + } > + > + slot = PCI_SLOT(err); > + > + err = mt7621_pcie_parse_port(pcie, slot); > + if (err) { > + of_node_put(child); > + return err; > + } > + } > + > + return 0; > +} > + > +static int mt7621_pcie_init_port(struct mt7621_pcie_port *port) > +{ > + struct mt7621_pcie *pcie = port->pcie; > + struct device *dev = pcie->dev; > + u32 slot = port->slot; > + int err; > + > + err = phy_init(port->phy); > + if (err) { > + dev_err(dev, "failed to initialize port%d phy\n", slot); > + return err; > + } > + > + err = phy_power_on(port->phy); > + if (err) { > + dev_err(dev, "failed to power on port%d phy\n", slot); > + phy_exit(port->phy); > + return err; > + } > + > + port->enabled = true; > + > + return 0; > +} > + > +static void mt7621_pcie_reset_assert(struct mt7621_pcie *pcie) > +{ > + struct mt7621_pcie_port *port; > + > + list_for_each_entry(port, &pcie->ports, list) { > + /* PCIe RC reset assert */ > + mt7621_control_assert(port); > + > + /* PCIe EP reset assert */ > + mt7621_rst_gpio_pcie_assert(port); > + } > + > + msleep(PERST_DELAY_MS); > +} > + > +static void mt7621_pcie_reset_rc_deassert(struct mt7621_pcie *pcie) > +{ > + struct mt7621_pcie_port *port; > + > + list_for_each_entry(port, &pcie->ports, list) > + mt7621_control_deassert(port); > +} > + > +static void mt7621_pcie_reset_ep_deassert(struct mt7621_pcie *pcie) > +{ > + struct mt7621_pcie_port *port; > + > + list_for_each_entry(port, &pcie->ports, list) > + mt7621_rst_gpio_pcie_deassert(port); > + > + msleep(PERST_DELAY_MS); > +} > + > +static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie) > +{ > + struct device *dev = pcie->dev; > + struct mt7621_pcie_port *port, *tmp; > + int err; > + > + mt7621_pcie_reset_assert(pcie); > + mt7621_pcie_reset_rc_deassert(pcie); > + > + list_for_each_entry_safe(port, tmp, &pcie->ports, list) { > + u32 slot = port->slot; > + > + if (slot == 1) { > + port->enabled = true; > + continue; > + } > + > + err = mt7621_pcie_init_port(port); > + if (err) { > + dev_err(dev, "Initiating port %d failed\n", slot); > + list_del(&port->list); > + } > + } > + > + mt7621_pcie_reset_ep_deassert(pcie); > + > + tmp = NULL; > + list_for_each_entry(port, &pcie->ports, list) { > + u32 slot = port->slot; > + > + if (!mt7621_pcie_port_is_linkup(port)) { > + dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", > + slot); > + mt7621_control_assert(port); > + clk_disable_unprepare(port->clk); > + port->enabled = false; > + > + if (slot == 0) { > + tmp = port; > + continue; > + } > + > + if (slot == 1 && tmp && !tmp->enabled) > + phy_power_off(tmp->phy); > + } > + } > +} > + > +static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port) > +{ > + struct mt7621_pcie *pcie = port->pcie; > + u32 slot = port->slot; > + u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT); I don't see how this works unless the ports happen to be at the same VA offset. The writes below are to the 'common' registers which are 0x100 bytes long based on the DT example, but the offset lines up with the port offsets. > + u32 val; > + > + /* enable pcie interrupt */ > + val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR); > + val |= PCIE_PORT_INT_EN(slot); > + pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR); > + > + /* map 2G DDR region */ > + pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE, > + offset + RALINK_PCI_BAR0SETUP_ADDR); > + > + /* configure class code and revision ID */ > + pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID, > + offset + RALINK_PCI_CLASS); > +} > + > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > +{ > + struct device *dev = pcie->dev; > + struct mt7621_pcie_port *port; > + u8 num_slots_enabled = 0; > + u32 slot; > + u32 val; > + int err; > + > + /* Setup MEMWIN and IOWIN */ > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > + > + list_for_each_entry(port, &pcie->ports, list) { > + if (port->enabled) { > + err = clk_prepare_enable(port->clk); > + if (err) { > + dev_err(dev, "enabling clk pcie%d\n", slot); > + return err; > + } > + > + mt7621_pcie_enable_port(port); > + dev_info(dev, "PCIE%d enabled\n", port->slot); > + num_slots_enabled++; > + } > + } > + > + for (slot = 0; slot < num_slots_enabled; slot++) { > + val = read_config(pcie, slot, PCI_COMMAND); > + val |= PCI_COMMAND_MASTER; > + write_config(pcie, slot, PCI_COMMAND, val); > + /* configure RC FTS number to 250 when it leaves L0s */ > + val = read_config(pcie, slot, PCIE_FTS_NUM); > + val &= ~PCIE_FTS_NUM_MASK; > + val |= PCIE_FTS_NUM_L0(0x50); > + write_config(pcie, slot, PCIE_FTS_NUM, val); > + } > + > + return 0; > +} > + > +static int mt7621_pcie_register_host(struct pci_host_bridge *host) > +{ > + struct mt7621_pcie *pcie = pci_host_bridge_priv(host); > + > + host->ops = &mt7621_pci_ops; > + host->sysdata = pcie; > + return pci_host_probe(host); > +} > + > +static const struct soc_device_attribute mt7621_pci_quirks_match[] = { > + { .soc_id = "mt7621", .revision = "E2" } > +}; > + > +static int mt7621_pci_probe(struct platform_device *pdev) > +{ > + struct device *dev = &pdev->dev; > + const struct soc_device_attribute *attr; > + struct mt7621_pcie *pcie; > + struct pci_host_bridge *bridge; > + int err; > + > + if (!dev->of_node) > + return -ENODEV; > + > + bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie)); > + if (!bridge) > + return -ENOMEM; > + > + pcie = pci_host_bridge_priv(bridge); > + pcie->dev = dev; > + platform_set_drvdata(pdev, pcie); > + INIT_LIST_HEAD(&pcie->ports); > + > + attr = soc_device_match(mt7621_pci_quirks_match); > + if (attr) > + pcie->resets_inverted = true; > + > + err = mt7621_pcie_parse_dt(pcie); > + if (err) { > + dev_err(dev, "Parsing DT failed\n"); > + return err; > + } > + > + err = mt7621_pci_parse_request_of_pci_ranges(bridge); > + if (err) { > + dev_err(dev, "Error requesting pci resources from ranges"); > + return err; > + } > + > + /* set resources limits */ > + ioport_resource.start = pcie->io.start; > + ioport_resource.end = pcie->io.end; > + > + mt7621_pcie_init_ports(pcie); > + > + err = mt7621_pcie_enable_ports(pcie); Really need 2 functions here? > + if (err) { > + dev_err(dev, "Error enabling pcie ports\n"); > + return err; > + } > + > + setup_cm_memory_region(pcie); > + > + return mt7621_pcie_register_host(bridge); > +} > + > +static const struct of_device_id mt7621_pci_ids[] = { > + { .compatible = "mediatek,mt7621-pci" }, > + {}, > +}; > +MODULE_DEVICE_TABLE(of, mt7621_pci_ids); > + > +static struct platform_driver mt7621_pci_driver = { > + .probe = mt7621_pci_probe, > + .driver = { > + .name = "mt7621-pci", > + .of_match_table = of_match_ptr(mt7621_pci_ids), > + }, > +}; > +builtin_platform_driver(mt7621_pci_driver); > diff --git a/arch/mips/ralink/Kconfig b/arch/mips/ralink/Kconfig > index ec4daa63c5e3..50e5a54f7d9e 100644 > --- a/arch/mips/ralink/Kconfig > +++ b/arch/mips/ralink/Kconfig > @@ -56,7 +56,7 @@ choice > select MIPS_GIC > select COMMON_CLK > select CLKSRC_MIPS_GIC > - select HAVE_PCI if PCI_MT7621 > + select HAVE_PCI > select SOC_BUS > endchoice > > @@ -101,4 +101,11 @@ choice > > endchoice > > +config PCI_MT7621 > + bool "MediaTek MT7621 PCI Controller" > + depends on RALINK && SOC_MT7621 Ideally this should also have (|| COMPILE_TEST), but looks like there are a few MIPS dependencies in here. So maybe (|| (MIPS && COMPILE_TEST)? If it's not built by allmodconfig or defconfig, it's hard for people to compile test it. > + select PCI_DRIVERS_GENERIC > + help > + This selects a driver for the MediaTek MT7621 PCI Controller. > + > endif > -- > 2.25.1
On Sat, May 15, 2021 at 02:40:52PM +0200, Sergio Paracuellos wrote: > Add device tree binding documentation for PCIe in MT7621 SoCs. > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > --- > .../bindings/pci/mediatek,mt7621-pci.yaml | 149 ++++++++++++++++++ > 1 file changed, 149 insertions(+) > create mode 100644 Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > diff --git a/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > new file mode 100644 > index 000000000000..7f5f9d583032 > --- /dev/null > +++ b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > @@ -0,0 +1,149 @@ > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/pci/mediatek,mt7621-pci.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: MediaTek MT7621 PCIe controller > + > +maintainers: > + - Sergio Paracuellos <sergio.paracuellos@gmail.com> > + > +description: |+ > + MediaTek MT7621 PCIe subsys supports single Root complex (RC) > + with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link > + > +allOf: > + - $ref: /schemas/pci/pci-bus.yaml# > + > +properties: > + compatible: > + const: mediatek,mt7621-pci > + > + reg: > + items: > + - description: host-pci bridge registers > + - description: pcie port 0 RC control registers > + - description: pcie port 1 RC control registers > + - description: pcie port 2 RC control registers > + > + ranges: > + maxItems: 2 > + > + resets: > + items: > + - description: pcie port 0 reset. > + - description: pcie port 1 reset. > + - description: pcie port 2 reset. > + > + reset-names: > + items: > + - const: pcie0 > + - const: pcie1 > + - const: pcie2 > + > + clocks: > + items: > + - description: pcie port 0 clock. > + - description: pcie port 1 clock. > + - description: pcie port 2 clock. > + > + clock-names: > + items: > + - const: pcie0 > + - const: pcie1 > + - const: pcie2 > + > + phys: > + items: > + - description: Dual-ported phy for pcie port 0 and 1. > + - description: Phy for pcie port 2. > + > + phy-names: > + items: > + - const: pcie-phy0 > + - const: pcie-phy2 If you're going to keep the ports (and I think that's right because there's only a single PCI address space AFAICT), then I think you should move resets, clocks, and phys into each port node. So you'll need to define 'pcie@[0-2],0' node with those properties under it. > + > +required: > + - compatible > + - reg > + - ranges > + - "#interrupt-cells" > + - interrupt-map-mask > + - interrupt-map > + - resets > + - reset-names > + - clocks > + - clock-names > + - phys > + - phy-names > + - reset-gpios > + > +unevaluatedProperties: false > + > +examples: > + - | > + #include <dt-bindings/gpio/gpio.h> > + #include <dt-bindings/interrupt-controller/mips-gic.h> > + > + pcie: pcie@1e140000 { > + compatible = "mediatek,mt7621-pci"; > + reg = <0x1e140000 0x100>, > + <0x1e142000 0x100>, > + <0x1e143000 0x100>, > + <0x1e144000 0x100>; > + > + #address-cells = <3>; > + #size-cells = <2>; > + pinctrl-names = "default"; > + pinctrl-0 = <&pcie_pins>; > + device_type = "pci"; > + ranges = <0x02000000 0 0x00000000 0x60000000 0 0x10000000>, /* pci memory */ > + <0x01000000 0 0x00000000 0x1e160000 0 0x00010000>; /* io space */ > + #interrupt-cells = <1>; > + interrupt-map-mask = <0xF800 0 0 0>; > + interrupt-map = <0x0000 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>, > + <0x0800 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>, > + <0x1000 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > + resets = <&rstctrl 24>, <&rstctrl 25>, <&rstctrl 26>; > + reset-names = "pcie0", "pcie1", "pcie2"; > + clocks = <&clkctrl 24>, <&clkctrl 25>, <&clkctrl 26>; > + clock-names = "pcie0", "pcie1", "pcie2"; > + phys = <&pcie0_phy 1>, <&pcie2_phy 0>; > + phy-names = "pcie-phy0", "pcie-phy2"; > + reset-gpios = <&gpio 19 GPIO_ACTIVE_LOW>; > + > + pcie@0,0 { > + reg = <0x0000 0 0 0 0>; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + #interrupt-cells = <1>; > + interrupt-map-mask = <0 0 0 0>; > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>; > + ranges; > + }; > + > + pcie@1,0 { > + reg = <0x0800 0 0 0 0>; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + #interrupt-cells = <1>; > + interrupt-map-mask = <0 0 0 0>; > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>; > + ranges; > + }; > + > + pcie@2,0 { > + reg = <0x1000 0 0 0 0>; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + #interrupt-cells = <1>; > + interrupt-map-mask = <0 0 0 0>; > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > + ranges; > + }; > + }; > +... > -- > 2.25.1
On Mon, May 31, 2021 at 03:18:45PM +0200, Pali Rohár wrote: > On Friday 21 May 2021 12:23:38 Thomas Bogendoerfer wrote: > > On Wed, May 19, 2021 at 11:18:36PM +0200, Sergio Paracuellos wrote: > > > On Wed, May 19, 2021 at 10:36 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > > > But most of the similar drivers are in drivers/pci/controller/, where > > > > I think it's easier to keep them up to date with changes in the PCI > > > > core. Have you considered putting this one there? > > > > > > Most pci drivers in 'arch/mips/pci' are still using PCI_LEGACY stuff. > > > In contrast mt7621-pci is using current pci generic apis but even most > > > of the code is generic enough, there is one remaining thing which > > > depends on mips architecture which is the iocu region configuration > > > which must be done in the driver itself. This is the only reason to > > > move this driver into 'arch/mips/pci' instead of > > > 'drivers/pci/controller/'. So... I am all ears to listen to > > > suggestions for the proper place for this driver. Thomas, do you have > > > any thoughts on this? > > > > I tried to put a pci-xtalk driver into drivers/pci/controller, but > > Lorenzo didn't want it there for being MIPS and not DT based. So this > > one is DT based, but still MIPS. I'm perfectly fine putting this > > driver into drivers/pci/controller/ > > In my personal opinion this driver could go into drivers/pci/controller/ I'm not sure exactly what "PCI_LEGACY" above refers to. I don't see any direct #includes of arch/mips in the driver. I do see that it uses mips_cps_numiocu(), which is certainly MIPS-specific. But we do have some things in drivers/pci/controller/ that only build on certain arches, enforced mostly by Kconfig rules, so I think you could do that. We try to make so things can at least be *compiled* on any arch, but I know that's not always possible. So I think it would be useful to put this in drivers/pci/controller/ somewhere because it will make it easier to see common patterns and refactoring opportunities. Bjorn
Hi Bjorn, On Fri, Jun 4, 2021 at 9:43 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > On Mon, May 31, 2021 at 03:18:45PM +0200, Pali Rohár wrote: > > On Friday 21 May 2021 12:23:38 Thomas Bogendoerfer wrote: > > > On Wed, May 19, 2021 at 11:18:36PM +0200, Sergio Paracuellos wrote: > > > > On Wed, May 19, 2021 at 10:36 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > > > > But most of the similar drivers are in drivers/pci/controller/, where > > > > > I think it's easier to keep them up to date with changes in the PCI > > > > > core. Have you considered putting this one there? > > > > > > > > Most pci drivers in 'arch/mips/pci' are still using PCI_LEGACY stuff. > > > > In contrast mt7621-pci is using current pci generic apis but even most > > > > of the code is generic enough, there is one remaining thing which > > > > depends on mips architecture which is the iocu region configuration > > > > which must be done in the driver itself. This is the only reason to > > > > move this driver into 'arch/mips/pci' instead of > > > > 'drivers/pci/controller/'. So... I am all ears to listen to > > > > suggestions for the proper place for this driver. Thomas, do you have > > > > any thoughts on this? > > > > > > I tried to put a pci-xtalk driver into drivers/pci/controller, but > > > Lorenzo didn't want it there for being MIPS and not DT based. So this > > > one is DT based, but still MIPS. I'm perfectly fine putting this > > > driver into drivers/pci/controller/ > > > > In my personal opinion this driver could go into drivers/pci/controller/ > > I'm not sure exactly what "PCI_LEGACY" above refers to. I meant most of the drivers there are not using current generic pci apis but still using pci legacy ones. > > I don't see any direct #includes of arch/mips in the driver. I do see > that it uses mips_cps_numiocu(), which is certainly MIPS-specific. Yes, mips_cps_numiocu is the only stuff needed and arch specific used by this driver. > > But we do have some things in drivers/pci/controller/ that only build > on certain arches, enforced mostly by Kconfig rules, so I think you > could do that. We try to make so things can at least be *compiled* on > any arch, but I know that's not always possible. > > So I think it would be useful to put this in drivers/pci/controller/ > somewhere because it will make it easier to see common patterns and > refactoring opportunities. Ok, so I will move the driver into 'drivers/pci/controller/' in v2. Thanks, Sergio Paracuellos > > Bjorn
Hi Rob, Thanks for the review. On Fri, Jun 4, 2021 at 9:35 PM Rob Herring <robh@kernel.org> wrote: > > On Sat, May 15, 2021 at 02:40:52PM +0200, Sergio Paracuellos wrote: > > Add device tree binding documentation for PCIe in MT7621 SoCs. > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > --- > > .../bindings/pci/mediatek,mt7621-pci.yaml | 149 ++++++++++++++++++ > > 1 file changed, 149 insertions(+) > > create mode 100644 Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > > > diff --git a/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > new file mode 100644 > > index 000000000000..7f5f9d583032 > > --- /dev/null > > +++ b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > @@ -0,0 +1,149 @@ > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > > +%YAML 1.2 > > +--- > > +$id: http://devicetree.org/schemas/pci/mediatek,mt7621-pci.yaml# > > +$schema: http://devicetree.org/meta-schemas/core.yaml# > > + > > +title: MediaTek MT7621 PCIe controller > > + > > +maintainers: > > + - Sergio Paracuellos <sergio.paracuellos@gmail.com> > > + > > +description: |+ > > + MediaTek MT7621 PCIe subsys supports single Root complex (RC) > > + with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link > > + > > +allOf: > > + - $ref: /schemas/pci/pci-bus.yaml# > > + > > +properties: > > + compatible: > > + const: mediatek,mt7621-pci > > + > > + reg: > > + items: > > + - description: host-pci bridge registers > > + - description: pcie port 0 RC control registers > > + - description: pcie port 1 RC control registers > > + - description: pcie port 2 RC control registers > > + > > + ranges: > > + maxItems: 2 > > + > > + resets: > > + items: > > + - description: pcie port 0 reset. > > + - description: pcie port 1 reset. > > + - description: pcie port 2 reset. > > + > > + reset-names: > > + items: > > + - const: pcie0 > > + - const: pcie1 > > + - const: pcie2 > > + > > + clocks: > > + items: > > + - description: pcie port 0 clock. > > + - description: pcie port 1 clock. > > + - description: pcie port 2 clock. > > + > > + clock-names: > > + items: > > + - const: pcie0 > > + - const: pcie1 > > + - const: pcie2 > > + > > + phys: > > + items: > > + - description: Dual-ported phy for pcie port 0 and 1. > > + - description: Phy for pcie port 2. > > + > > + phy-names: > > + items: > > + - const: pcie-phy0 > > + - const: pcie-phy2 > > If you're going to keep the ports (and I think that's right because > there's only a single PCI address space AFAICT), then I think you should > move resets, clocks, and phys into each port node. > > So you'll need to define 'pcie@[0-2],0' node with those properties under > it. Ok I will move these stuff to each port node. So each port node will be similar to: pcie@0,0 { reg = <0x0000 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; device_type = "pci"; #interrupt-cells = <1>; clocks = <&clkctrl 24>; resets = <&rstctrl 24>; phys = <&pcie0_phy 1>; interrupt-map-mask = <0 0 0 0>; interrupt-map = <0 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>; ranges; }; How can I be sure by schema that the clocks, reset and phy properties are in each port node if I move them from the parent? By now each port node is just validating because of ' $ref: /schemas/pci/pci-bus.yaml#'. Thanks in advance for your time. Best regards, Sergio Paracuellos > > > + > > +required: > > + - compatible > > + - reg > > + - ranges > > + - "#interrupt-cells" > > + - interrupt-map-mask > > + - interrupt-map > > + - resets > > + - reset-names > > + - clocks > > + - clock-names > > + - phys > > + - phy-names > > + - reset-gpios > > + > > +unevaluatedProperties: false > > + > > +examples: > > + - | > > + #include <dt-bindings/gpio/gpio.h> > > + #include <dt-bindings/interrupt-controller/mips-gic.h> > > + > > + pcie: pcie@1e140000 { > > + compatible = "mediatek,mt7621-pci"; > > + reg = <0x1e140000 0x100>, > > + <0x1e142000 0x100>, > > + <0x1e143000 0x100>, > > + <0x1e144000 0x100>; > > + > > + #address-cells = <3>; > > + #size-cells = <2>; > > + pinctrl-names = "default"; > > + pinctrl-0 = <&pcie_pins>; > > + device_type = "pci"; > > + ranges = <0x02000000 0 0x00000000 0x60000000 0 0x10000000>, /* pci memory */ > > + <0x01000000 0 0x00000000 0x1e160000 0 0x00010000>; /* io space */ > > + #interrupt-cells = <1>; > > + interrupt-map-mask = <0xF800 0 0 0>; > > + interrupt-map = <0x0000 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>, > > + <0x0800 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>, > > + <0x1000 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > > + resets = <&rstctrl 24>, <&rstctrl 25>, <&rstctrl 26>; > > + reset-names = "pcie0", "pcie1", "pcie2"; > > + clocks = <&clkctrl 24>, <&clkctrl 25>, <&clkctrl 26>; > > + clock-names = "pcie0", "pcie1", "pcie2"; > > + phys = <&pcie0_phy 1>, <&pcie2_phy 0>; > > + phy-names = "pcie-phy0", "pcie-phy2"; > > + reset-gpios = <&gpio 19 GPIO_ACTIVE_LOW>; > > + > > + pcie@0,0 { > > + reg = <0x0000 0 0 0 0>; > > + #address-cells = <3>; > > + #size-cells = <2>; > > + device_type = "pci"; > > + #interrupt-cells = <1>; > > + interrupt-map-mask = <0 0 0 0>; > > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>; > > + ranges; > > + }; > > + > > + pcie@1,0 { > > + reg = <0x0800 0 0 0 0>; > > + #address-cells = <3>; > > + #size-cells = <2>; > > + device_type = "pci"; > > + #interrupt-cells = <1>; > > + interrupt-map-mask = <0 0 0 0>; > > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>; > > + ranges; > > + }; > > + > > + pcie@2,0 { > > + reg = <0x1000 0 0 0 0>; > > + #address-cells = <3>; > > + #size-cells = <2>; > > + device_type = "pci"; > > + #interrupt-cells = <1>; > > + interrupt-map-mask = <0 0 0 0>; > > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > > + ranges; > > + }; > > + }; > > +... > > -- > > 2.25.1
Hi Rob, Thanks for the review. On Fri, Jun 4, 2021 at 9:30 PM Rob Herring <robh@kernel.org> wrote: > > On Sat, May 15, 2021 at 02:40:53PM +0200, Sergio Paracuellos wrote: > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > --- > > arch/mips/pci/Makefile | 1 + > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > arch/mips/ralink/Kconfig | 9 +- > > 3 files changed, 633 insertions(+), 1 deletion(-) > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > index f3eecc065e5c..178c550739c4 100644 > > --- a/arch/mips/pci/Makefile > > +++ b/arch/mips/pci/Makefile > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > # > > # These are still pretty much in the old state, watch, go blind. > > # > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > new file mode 100644 > > index 000000000000..fe1945819d25 > > --- /dev/null > > +++ b/arch/mips/pci/pci-mt7621.c > > @@ -0,0 +1,624 @@ > > +// SPDX-License-Identifier: GPL-2.0+ > > +/* > > + * BRIEF MODULE DESCRIPTION > > + * PCI init for Ralink RT2880 solution > > + * > > + * Copyright 2007 Ralink Inc. (bruce_chang@ralinktech.com.tw) > > + * > > + * May 2007 Bruce Chang > > + * Initial Release > > + * > > + * May 2009 Bruce Chang > > + * support RT2880/RT3883 PCIe > > + * > > + * May 2011 Bruce Chang > > + * support RT6855/MT7620 PCIe > > + */ > > + > > +#include <linux/bitops.h> > > +#include <linux/clk.h> > > +#include <linux/delay.h> > > +#include <linux/gpio/consumer.h> > > +#include <linux/module.h> > > +#include <linux/of.h> > > +#include <linux/of_address.h> > > +#include <linux/of_pci.h> > > +#include <linux/of_platform.h> > > +#include <linux/pci.h> > > +#include <linux/phy/phy.h> > > +#include <linux/platform_device.h> > > +#include <linux/reset.h> > > +#include <linux/sys_soc.h> > > + > > +/* MediaTek specific configuration registers */ > > +#define PCIE_FTS_NUM 0x70c > > +#define PCIE_FTS_NUM_MASK GENMASK(15, 8) > > +#define PCIE_FTS_NUM_L0(x) (((x) & 0xff) << 8) > > + > > +/* Host-PCI bridge registers */ > > +#define RALINK_PCI_PCICFG_ADDR 0x0000 > > +#define RALINK_PCI_PCIMSK_ADDR 0x000C > > +#define RALINK_PCI_CONFIG_ADDR 0x0020 > > +#define RALINK_PCI_CONFIG_DATA 0x0024 > > +#define RALINK_PCI_MEMBASE 0x0028 > > +#define RALINK_PCI_IOBASE 0x002C > > + > > +/* PCIe RC control registers */ > > +#define MT7621_PCIE_OFFSET 0x2000 > > +#define MT7621_NEXT_PORT 0x1000 > > + > > +#define RALINK_PCI_BAR0SETUP_ADDR 0x0010 > > Standard BAR0 register? Ok, I will remove this and use 'PCI_BASE_ADDRESS_0' instead. > > > +#define RALINK_PCI_ID 0x0030 > > +#define RALINK_PCI_CLASS 0x0034 > > +#define RALINK_PCI_SUBID 0x0038 > > +#define RALINK_PCI_STATUS 0x0050 > > + > > +/* Some definition values */ > > +#define PCIE_REVISION_ID BIT(0) > > +#define PCIE_CLASS_CODE (0x60400 << 8) > > +#define PCIE_BAR_MAP_MAX GENMASK(30, 16) > > +#define PCIE_BAR_ENABLE BIT(0) > > +#define PCIE_PORT_INT_EN(x) BIT(20 + (x)) > > +#define PCIE_PORT_LINKUP BIT(0) > > + > > +#define PERST_DELAY_MS 100 > > + > > +/** > > + * struct mt7621_pcie_port - PCIe port information > > + * @base: I/O mapped register base > > + * @list: port list > > + * @pcie: pointer to PCIe host info > > + * @clk: pointer to the port clock gate > > + * @phy: pointer to PHY control block > > + * @pcie_rst: pointer to port reset control > > + * @gpio_rst: gpio reset > > + * @slot: port slot > > + * @enabled: indicates if port is enabled > > + */ > > +struct mt7621_pcie_port { > > + void __iomem *base; > > + struct list_head list; > > + struct mt7621_pcie *pcie; > > + struct clk *clk; > > + struct phy *phy; > > + struct reset_control *pcie_rst; > > + struct gpio_desc *gpio_rst; > > + u32 slot; > > + bool enabled; > > +}; > > + > > +/** > > + * struct mt7621_pcie - PCIe host information > > + * @base: IO Mapped Register Base > > + * @io: IO resource > > + * @mem: pointer to non-prefetchable memory resource > > + * @dev: Pointer to PCIe device > > + * @io_map_base: virtual memory base address for io > > + * @ports: pointer to PCIe port information > > + * @resets_inverted: depends on chip revision > > + * reset lines are inverted. > > + */ > > +struct mt7621_pcie { > > + void __iomem *base; > > + struct device *dev; > > > + struct resource io; > > + struct resource *mem; > > + unsigned long io_map_base; > > These are all stored in the host bridge struct, no need for you to store > them. IO resources must be requested and mapped manually (see my explanation below) and I use mem resource to also setup mips iocu regions, that is why I am storing it also here. > > > + struct list_head ports; > > A list is kind of an overkill for 3 entries and you know how many ports. > Just embed an array of struct mt7621_pcie_port. Then you only need 1 > alloc. Since ports are at most three but can be one or two also depending on the board I ended up using a list instead of a fixed array of three ports. This list is dynamically updated depending on link status. If some of the ports are not present in the board nodes initially stored after device tree parsing are deleted for the list. If it is not a big problem I prefer to maintain this list as it is. > > > + bool resets_inverted; > > +}; > > + > > +static inline u32 pcie_read(struct mt7621_pcie *pcie, u32 reg) > > +{ > > + return readl(pcie->base + reg); > > Can use _relaxed variants here and through out. Ok will change into _relaxed variants if is preferred. > > > +} > > + > > +static inline void pcie_write(struct mt7621_pcie *pcie, u32 val, u32 reg) > > +{ > > + writel(val, pcie->base + reg); > > +} > > + > > +static inline void pcie_rmw(struct mt7621_pcie *pcie, u32 reg, u32 clr, u32 set) > > +{ > > + u32 val = readl(pcie->base + reg); > > + > > + val &= ~clr; > > + val |= set; > > + writel(val, pcie->base + reg); > > +} > > + > > +static inline u32 pcie_port_read(struct mt7621_pcie_port *port, u32 reg) > > +{ > > + return readl(port->base + reg); > > +} > > + > > +static inline void pcie_port_write(struct mt7621_pcie_port *port, > > + u32 val, u32 reg) > > +{ > > + writel(val, port->base + reg); > > +} > > + > > +static inline u32 mt7621_pci_get_cfgaddr(unsigned int bus, unsigned int slot, > > + unsigned int func, unsigned int where) > > +{ > > + return (((where & 0xF00) >> 8) << 24) | (bus << 16) | (slot << 11) | > > + (func << 8) | (where & 0xfc) | 0x80000000; > > +} > > + > > +static void __iomem *mt7621_pcie_map_bus(struct pci_bus *bus, > > + unsigned int devfn, int where) > > +{ > > + struct mt7621_pcie *pcie = bus->sysdata; > > + u32 address = mt7621_pci_get_cfgaddr(bus->number, PCI_SLOT(devfn), > > + PCI_FUNC(devfn), where); > > + > > + writel(address, pcie->base + RALINK_PCI_CONFIG_ADDR); > > + > > + return pcie->base + RALINK_PCI_CONFIG_DATA + (where & 3); > > +} > > + > > +struct pci_ops mt7621_pci_ops = { > > + .map_bus = mt7621_pcie_map_bus, > > + .read = pci_generic_config_read, > > + .write = pci_generic_config_write, > > +}; > > + > > +static u32 read_config(struct mt7621_pcie *pcie, unsigned int dev, u32 reg) > > +{ > > + u32 address = mt7621_pci_get_cfgaddr(0, dev, 0, reg); > > + > > + pcie_write(pcie, address, RALINK_PCI_CONFIG_ADDR); > > + return pcie_read(pcie, RALINK_PCI_CONFIG_DATA); > > +} > > + > > +static void write_config(struct mt7621_pcie *pcie, unsigned int dev, > > + u32 reg, u32 val) > > +{ > > + u32 address = mt7621_pci_get_cfgaddr(0, dev, 0, reg); > > + > > + pcie_write(pcie, address, RALINK_PCI_CONFIG_ADDR); > > + pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA); > > +} > > + > > +static inline void mt7621_rst_gpio_pcie_assert(struct mt7621_pcie_port *port) > > +{ > > + if (port->gpio_rst) > > Don't need the if. gpiod_set_value should work with NULL. Gpio resets can be optional. Some boards use one gpio reset pin for each port but others only one for all of them. So if no reset is not requested for a port I don't want to do anything. Hence, the check. > > > + gpiod_set_value(port->gpio_rst, 1); > > +} > > + > > +static inline void mt7621_rst_gpio_pcie_deassert(struct mt7621_pcie_port *port) > > +{ > > + if (port->gpio_rst) > > + gpiod_set_value(port->gpio_rst, 0); > > +} > > + > > +static inline bool mt7621_pcie_port_is_linkup(struct mt7621_pcie_port *port) > > +{ > > + return (pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) != 0; > > +} > > + > > +static inline void mt7621_control_assert(struct mt7621_pcie_port *port) > > +{ > > + struct mt7621_pcie *pcie = port->pcie; > > + > > + if (pcie->resets_inverted) > > + reset_control_assert(port->pcie_rst); > > + else > > + reset_control_deassert(port->pcie_rst); > > +} > > + > > +static inline void mt7621_control_deassert(struct mt7621_pcie_port *port) > > +{ > > + struct mt7621_pcie *pcie = port->pcie; > > + > > + if (pcie->resets_inverted) > > + reset_control_deassert(port->pcie_rst); > > + else > > + reset_control_assert(port->pcie_rst); > > +} > > + > > +static void setup_cm_memory_region(struct mt7621_pcie *pcie) > > +{ > > + struct resource *mem_resource = pcie->mem; > > + struct device *dev = pcie->dev; > > + resource_size_t mask; > > + > > + if (mips_cps_numiocu(0)) { > > + /* > > + * FIXME: hardware doesn't accept mask values with 1s after > > + * 0s (e.g. 0xffef), so it would be great to warn if that's > > + * about to happen > > + */ > > + mask = ~(mem_resource->end - mem_resource->start); > > + > > + write_gcr_reg1_base(mem_resource->start); > > + write_gcr_reg1_mask(mask | CM_GCR_REGn_MASK_CMTGT_IOCU0); > > + dev_info(dev, "PCI coherence region base: 0x%08llx, mask/settings: 0x%08llx\n", > > + (unsigned long long)read_gcr_reg1_base(), > > + (unsigned long long)read_gcr_reg1_mask()); > > + } > > +} > > + > > +static int mt7621_pci_parse_request_of_pci_ranges(struct pci_host_bridge *host) > > +{ > > + struct mt7621_pcie *pcie = pci_host_bridge_priv(host); > > + struct device *dev = pcie->dev; > > + struct device_node *node = dev->of_node; > > + struct of_pci_range_parser parser; > > + struct resource_entry *entry; > > + struct of_pci_range range; > > + LIST_HEAD(res); > > + > > + if (of_pci_range_parser_init(&parser, node)) { > > + dev_err(dev, "missing \"ranges\" property\n"); > > + return -EINVAL; > > + } > > + > > + /* > > + * IO_SPACE_LIMIT for MIPS is 0xffff but this platform uses IO at > > + * upper address 0x001e160000. of_pci_range_to_resource does not work > > I think that's normal... No, pci_address_to_pio will fail and io resources are not properly assigned at all. > > > + * well for MIPS platforms that don't define PCI_IOBASE, so set the IO > > + * resource manually instead. > > Can't this be fixed? This is the way the current PCI architecture is in mips... Maybe Thomas has a strong opinion on this. > > > + */ > > + for_each_of_pci_range(&parser, &range) { > > + switch (range.flags & IORESOURCE_TYPE_BITS) { > > The core code already parses ranges for you. Try not to do it again. > (Use the resource instead) See below... > > > + case IORESOURCE_IO: > > + pcie->io_map_base = > > + (unsigned long)ioremap(range.cpu_addr, > > + range.size); > > + pcie->io.name = node->full_name; > > + pcie->io.flags = range.flags; > > + pcie->io.start = range.cpu_addr; > > + pcie->io.end = range.cpu_addr + range.size - 1; > > + pcie->io.parent = pcie->io.child = pcie->io.sibling = NULL; > > + set_io_port_base(pcie->io_map_base); > > + break; > > + } > > + } > > + > > + entry = resource_list_first_type(&host->windows, IORESOURCE_MEM); > > + if (!entry) { > > + dev_err(dev, "Cannot get memory resource"); > > + return -EINVAL; > > + } > > + > > + pcie->mem = entry->res; > > + pci_add_resource(&res, &pcie->io); > > + pci_add_resource(&res, entry->res); > > + list_splice_init(&res, &host->windows); > > This should already be done for you. Most MIPS platforms do not define PCI_IOBASE, nor implement pci_address_to_pio(). Moreover, IO_SPACE_LIMIT is 0xffff for most MIPS platforms. of_pci_range_to_resource passes the _start address_ of the IO range into pci_address_to_pio, which then checks it against IO_SPACE_LIMIT and fails. So I have to do this manually or nothing will work properly... > > > + > > + return 0; > > +} > > + > > +static int mt7621_pcie_parse_port(struct mt7621_pcie *pcie, > > + int slot) > > +{ > > + struct mt7621_pcie_port *port; > > + struct device *dev = pcie->dev; > > + struct platform_device *pdev = to_platform_device(dev); > > + char name[10]; > > + > > + port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL); > > + if (!port) > > + return -ENOMEM; > > + > > + port->base = devm_platform_ioremap_resource(pdev, slot + 1); > > + if (IS_ERR(port->base)) > > + return PTR_ERR(port->base); > > + > > + snprintf(name, sizeof(name), "pcie%d", slot); > > + port->clk = devm_clk_get(dev, name); > > + if (IS_ERR(port->clk)) { > > + dev_err(dev, "failed to get pcie%d clock\n", slot); > > + return PTR_ERR(port->clk); > > + } > > + > > + snprintf(name, sizeof(name), "pcie%d", slot); > > + port->pcie_rst = devm_reset_control_get_exclusive(dev, name); > > + if (PTR_ERR(port->pcie_rst) == -EPROBE_DEFER) { > > + dev_err(dev, "failed to get pcie%d reset control\n", slot); > > + return PTR_ERR(port->pcie_rst); > > + } > > + > > + snprintf(name, sizeof(name), "pcie-phy%d", slot); > > + port->phy = devm_phy_get(dev, name); > > + if (IS_ERR(port->phy) && slot != 1) > > + return PTR_ERR(port->phy); > > + > > + port->gpio_rst = devm_gpiod_get_index_optional(dev, "reset", slot, > > + GPIOD_OUT_LOW); > > + if (IS_ERR(port->gpio_rst)) { > > + dev_err(dev, "Failed to get GPIO for PCIe%d\n", slot); > > + return PTR_ERR(port->gpio_rst); > > + } > > + > > + port->slot = slot; > > + port->pcie = pcie; > > + > > + INIT_LIST_HEAD(&port->list); > > + list_add_tail(&port->list, &pcie->ports); > > + > > + return 0; > > +} > > + > > +static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie) > > +{ > > + struct device *dev = pcie->dev; > > + struct platform_device *pdev = to_platform_device(dev); > > + struct device_node *node = dev->of_node, *child; > > + int err; > > + > > + pcie->base = devm_platform_ioremap_resource(pdev, 0); > > + if (IS_ERR(pcie->base)) > > + return PTR_ERR(pcie->base); > > + > > + for_each_available_child_of_node(node, child) { > > + int slot; > > + > > + err = of_pci_get_devfn(child); > > + if (err < 0) { > > + of_node_put(child); > > + dev_err(dev, "failed to parse devfn: %d\n", err); > > + return err; > > + } > > + > > + slot = PCI_SLOT(err); > > + > > + err = mt7621_pcie_parse_port(pcie, slot); > > + if (err) { > > + of_node_put(child); > > + return err; > > + } > > + } > > + > > + return 0; > > +} > > + > > +static int mt7621_pcie_init_port(struct mt7621_pcie_port *port) > > +{ > > + struct mt7621_pcie *pcie = port->pcie; > > + struct device *dev = pcie->dev; > > + u32 slot = port->slot; > > + int err; > > + > > + err = phy_init(port->phy); > > + if (err) { > > + dev_err(dev, "failed to initialize port%d phy\n", slot); > > + return err; > > + } > > + > > + err = phy_power_on(port->phy); > > + if (err) { > > + dev_err(dev, "failed to power on port%d phy\n", slot); > > + phy_exit(port->phy); > > + return err; > > + } > > + > > + port->enabled = true; > > + > > + return 0; > > +} > > + > > +static void mt7621_pcie_reset_assert(struct mt7621_pcie *pcie) > > +{ > > + struct mt7621_pcie_port *port; > > + > > + list_for_each_entry(port, &pcie->ports, list) { > > + /* PCIe RC reset assert */ > > + mt7621_control_assert(port); > > + > > + /* PCIe EP reset assert */ > > + mt7621_rst_gpio_pcie_assert(port); > > + } > > + > > + msleep(PERST_DELAY_MS); > > +} > > + > > +static void mt7621_pcie_reset_rc_deassert(struct mt7621_pcie *pcie) > > +{ > > + struct mt7621_pcie_port *port; > > + > > + list_for_each_entry(port, &pcie->ports, list) > > + mt7621_control_deassert(port); > > +} > > + > > +static void mt7621_pcie_reset_ep_deassert(struct mt7621_pcie *pcie) > > +{ > > + struct mt7621_pcie_port *port; > > + > > + list_for_each_entry(port, &pcie->ports, list) > > + mt7621_rst_gpio_pcie_deassert(port); > > + > > + msleep(PERST_DELAY_MS); > > +} > > + > > +static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie) > > +{ > > + struct device *dev = pcie->dev; > > + struct mt7621_pcie_port *port, *tmp; > > + int err; > > + > > + mt7621_pcie_reset_assert(pcie); > > + mt7621_pcie_reset_rc_deassert(pcie); > > + > > + list_for_each_entry_safe(port, tmp, &pcie->ports, list) { > > + u32 slot = port->slot; > > + > > + if (slot == 1) { > > + port->enabled = true; > > + continue; > > + } > > + > > + err = mt7621_pcie_init_port(port); > > + if (err) { > > + dev_err(dev, "Initiating port %d failed\n", slot); > > + list_del(&port->list); > > + } > > + } > > + > > + mt7621_pcie_reset_ep_deassert(pcie); > > + > > + tmp = NULL; > > + list_for_each_entry(port, &pcie->ports, list) { > > + u32 slot = port->slot; > > + > > + if (!mt7621_pcie_port_is_linkup(port)) { > > + dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", > > + slot); > > + mt7621_control_assert(port); > > + clk_disable_unprepare(port->clk); > > + port->enabled = false; > > + > > + if (slot == 0) { > > + tmp = port; > > + continue; > > + } > > + > > + if (slot == 1 && tmp && !tmp->enabled) > > + phy_power_off(tmp->phy) > > + } > > + } > > +} > > + > > +static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port) > > +{ > > + struct mt7621_pcie *pcie = port->pcie; > > + u32 slot = port->slot; > > + u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT); > > I don't see how this works unless the ports happen to be at the same VA > offset. The writes below are to the 'common' registers which are 0x100 > bytes long based on the DT example, but the offset lines up with the > port offsets. RC registers for port 0 start at physical address 0x1e142000, for port 1 0x1e143000 and for port 2 0x1e144000. We are using 'pcie_write' which internally uses pcie->base which is 0x1e140000, but all the previous ones have been already requested and mapped for each port so we calculate the offset from this 0x1e140000 and write there. > > > + u32 val; > > + > > + /* enable pcie interrupt */ > > + val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR); > > + val |= PCIE_PORT_INT_EN(slot); > > + pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR); > > + > > + /* map 2G DDR region */ > > + pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE, > > + offset + RALINK_PCI_BAR0SETUP_ADDR); > > + > > + /* configure class code and revision ID */ > > + pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID, > > + offset + RALINK_PCI_CLASS); > > +} > > + > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > +{ > > + struct device *dev = pcie->dev; > > + struct mt7621_pcie_port *port; > > + u8 num_slots_enabled = 0; > > + u32 slot; > > + u32 val; > > + int err; > > + > > + /* Setup MEMWIN and IOWIN */ > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > + > > + list_for_each_entry(port, &pcie->ports, list) { > > + if (port->enabled) { > > + err = clk_prepare_enable(port->clk); > > + if (err) { > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > + return err; > > + } > > + > > + mt7621_pcie_enable_port(port); > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > + num_slots_enabled++; > > + } > > + } > > + > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > + val = read_config(pcie, slot, PCI_COMMAND); > > + val |= PCI_COMMAND_MASTER; > > + write_config(pcie, slot, PCI_COMMAND, val); > > + /* configure RC FTS number to 250 when it leaves L0s */ > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > + val &= ~PCIE_FTS_NUM_MASK; > > + val |= PCIE_FTS_NUM_L0(0x50); > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > + } > > + > > + return 0; > > +} > > + > > +static int mt7621_pcie_register_host(struct pci_host_bridge *host) > > +{ > > + struct mt7621_pcie *pcie = pci_host_bridge_priv(host); > > + > > + host->ops = &mt7621_pci_ops; > > + host->sysdata = pcie; > > + return pci_host_probe(host); > > +} > > + > > +static const struct soc_device_attribute mt7621_pci_quirks_match[] = { > > + { .soc_id = "mt7621", .revision = "E2" } > > +}; > > + > > +static int mt7621_pci_probe(struct platform_device *pdev) > > +{ > > + struct device *dev = &pdev->dev; > > + const struct soc_device_attribute *attr; > > + struct mt7621_pcie *pcie; > > + struct pci_host_bridge *bridge; > > + int err; > > + > > + if (!dev->of_node) > > + return -ENODEV; > > + > > + bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie)); > > + if (!bridge) > > + return -ENOMEM; > > + > > + pcie = pci_host_bridge_priv(bridge); > > + pcie->dev = dev; > > + platform_set_drvdata(pdev, pcie); > > + INIT_LIST_HEAD(&pcie->ports); > > + > > + attr = soc_device_match(mt7621_pci_quirks_match); > > + if (attr) > > + pcie->resets_inverted = true; > > + > > + err = mt7621_pcie_parse_dt(pcie); > > + if (err) { > > + dev_err(dev, "Parsing DT failed\n"); > > + return err; > > + } > > + > > + err = mt7621_pci_parse_request_of_pci_ranges(bridge); > > + if (err) { > > + dev_err(dev, "Error requesting pci resources from ranges"); > > + return err; > > + } > > + > > + /* set resources limits */ > > + ioport_resource.start = pcie->io.start; > > + ioport_resource.end = pcie->io.end; > > + > > + mt7621_pcie_init_ports(pcie); > > + > > + err = mt7621_pcie_enable_ports(pcie); > > Really need 2 functions here? mt7621_pcie_init_ports is in charge of power on the phy and checks for link status to see which ports are in use and mt7621_pcie_enable_ports just enable all the stuff to be ready to be used. > > > + if (err) { > > + dev_err(dev, "Error enabling pcie ports\n"); > > + return err; > > + } > > + > > + setup_cm_memory_region(pcie); > > + > > + return mt7621_pcie_register_host(bridge); > > +} > > + > > +static const struct of_device_id mt7621_pci_ids[] = { > > + { .compatible = "mediatek,mt7621-pci" }, > > + {}, > > +}; > > +MODULE_DEVICE_TABLE(of, mt7621_pci_ids); > > + > > +static struct platform_driver mt7621_pci_driver = { > > + .probe = mt7621_pci_probe, > > + .driver = { > > + .name = "mt7621-pci", > > + .of_match_table = of_match_ptr(mt7621_pci_ids), > > + }, > > +}; > > +builtin_platform_driver(mt7621_pci_driver); > > diff --git a/arch/mips/ralink/Kconfig b/arch/mips/ralink/Kconfig > > index ec4daa63c5e3..50e5a54f7d9e 100644 > > --- a/arch/mips/ralink/Kconfig > > +++ b/arch/mips/ralink/Kconfig > > @@ -56,7 +56,7 @@ choice > > select MIPS_GIC > > select COMMON_CLK > > select CLKSRC_MIPS_GIC > > - select HAVE_PCI if PCI_MT7621 > > + select HAVE_PCI > > select SOC_BUS > > endchoice > > > > @@ -101,4 +101,11 @@ choice > > > > endchoice > > > > +config PCI_MT7621 > > + bool "MediaTek MT7621 PCI Controller" > > + depends on RALINK && SOC_MT7621 > > Ideally this should also have (|| COMPILE_TEST), but looks like there > are a few MIPS dependencies in here. So maybe (|| (MIPS && > COMPILE_TEST)? If it's not built by allmodconfig or defconfig, it's hard > for people to compile test it. Ok, I will add (|| (MIPS && COMPILE_TEST) here. Best regards, Sergio Paracuellos > > > > + select PCI_DRIVERS_GENERIC > > + help > > + This selects a driver for the MediaTek MT7621 PCI Controller. > > + > > endif > > -- > > 2.25.1
On Friday 04 June 2021 13:49:39 Rob Herring wrote: > On Fri, Jun 04, 2021 at 06:55:25PM +0200, Pali Rohár wrote: > > On Wednesday 02 June 2021 14:43:53 Sergio Paracuellos wrote: > > > Hi Pali, > > > > > > On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > > > > Hi Pali, > > > > > > > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > > > > <sergio.paracuellos@gmail.com> wrote: > > > > > > > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > > > > Hi Pali, > > > > > > > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > > > > --- > > > > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > > > > # > > > > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > > > > # > > > > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > > > > new file mode 100644 > > > > > > > > > > index 000000000000..fe1945819d25 > > > > > > > > > > --- /dev/null > > > > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > > > > ... > > > > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > > > > +{ > > > > > > > > > > + struct device *dev = pcie->dev; > > > > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > > > > + u32 slot; > > > > > > > > > > + u32 val; > > > > > > > > > > + int err; > > > > > > > > > > + > > > > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > > > > + > > > > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > > > > + if (port->enabled) { > > > > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > > > > + if (err) { > > > > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > > > > + return err; > > > > > > > > > > + } > > > > > > > > > > + > > > > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > > > > + num_slots_enabled++; > > > > > > > > > > + } > > > > > > > > > > + } > > > > > > > > > > + > > > > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > > > > driver initialize itself. > > > > > > > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > > > > which is used by device drivers. > > > > > > > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > > > > > > > For sure it is :(. > > > > > > > > > > > > > > > > > > > > > I have > > > > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > > > > driver code and the setting here [0]. There is no other reason than > > > > > > > > that. I am ok with removing this from here and testing with my two > > > > > > > > devices that everything is still ok if having this setting in the pci > > > > > > > > controller driver is a real problem. > > > > > > > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > > > > then compare outputs. > > > > > > > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > > > > needed, so it is possible that there would be no difference in lspci > > > > > > > output. > > > > > > > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > > > > review comments come :). > > > > > > > > > > I have tested to remove this and check lspci -nnvv output with and > > > > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > > > > difference between them. Also, both boards are working without > > > > > regressions at all. So I will remove this code for next version. > > > > > > > > Perfect! > > > > > > > > > Thanks, > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > > > > > > > Best regards, > > > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > > > > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > > > > as MT specific register. But from this code for me it looks like that it > > > > just access config space of some device and therefore it could be some > > > > standard PCIe register. Just with hardcoded calculated offset. > > > > So based on your lspci output, there is no PCIe capability register at > > address PCIE_FTS_NUM (0x70c), right? It seems strange to trying access > > capability register outside of capability list. > > Looks like a DW PCIe port logic register: > > drivers/pci/controller/dwc/pcie-designware.h-/* Synopsys-specific PCIe configuration registers */ > drivers/pci/controller/dwc/pcie-designware.h:#define PCIE_PORT_AFR 0x70C > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_N_FTS_MASK GENMASK(15, 8) > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_N_FTS(n) FIELD_PREP(PORT_AFR_N_FTS_MASK, n) > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_CC_N_FTS_MASK GENMASK(23, 16) > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_CC_N_FTS(n) FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, n) > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_ENTER_ASPM BIT(30) > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L0S_ENTRANCE_LAT_SHIFT 24 > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L0S_ENTRANCE_LAT_MASK GENMASK(26, 24) > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L1_ENTRANCE_LAT_SHIFT 27 > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L1_ENTRANCE_LAT_MASK GENMASK(29, 27) > > Rob Rob: nice catch! Does it mean that this MT7621 SoC has dwc controller and driver can be theoretically in future rewritten to use common dwc code? Sergio: I have tried to find some information about it and seems that MT7620, MT7621 and MT7628 SoC are really using some designware dwc IP. Some details are available in section "Embedded/kernel developer friendliness" in following blog post: https://www.abclinuxu.cz/blog/GardenOfEdenConfiguration/2019/10/opus-magnum And seems that "programming guide" documentation for MT7620 is available on internet with description of PCIe registers. I do not know how MT7620 and MT7621 are different but maybe it could help to develop or understand driver.
On Friday 04 June 2021 20:44:42 Sergio Paracuellos wrote: > Hi Pali, > > Thanks for your comments. > > On Fri, Jun 4, 2021 at 6:55 PM Pali Rohár <pali@kernel.org> wrote: > > > > On Wednesday 02 June 2021 14:43:53 Sergio Paracuellos wrote: > > > Hi Pali, > > > > > > On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > > > > Hi Pali, > > > > > > > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > > > > <sergio.paracuellos@gmail.com> wrote: > > > > > > > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > > > > Hi Pali, > > > > > > > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > > > > --- > > > > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > > > > # > > > > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > > > > # > > > > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > > > > new file mode 100644 > > > > > > > > > > index 000000000000..fe1945819d25 > > > > > > > > > > --- /dev/null > > > > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > > > > ... > > > > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > > > > +{ > > > > > > > > > > + struct device *dev = pcie->dev; > > > > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > > > > + u32 slot; > > > > > > > > > > + u32 val; > > > > > > > > > > + int err; > > > > > > > > > > + > > > > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > > > > + > > > > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > > > > + if (port->enabled) { > > > > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > > > > + if (err) { > > > > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > > > > + return err; > > > > > > > > > > + } > > > > > > > > > > + > > > > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > > > > + num_slots_enabled++; > > > > > > > > > > + } > > > > > > > > > > + } > > > > > > > > > > + > > > > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > > > > driver initialize itself. > > > > > > > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > > > > which is used by device drivers. > > > > > > > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > > > > > > > For sure it is :(. > > > > > > > > > > > > > > > > > > > > > I have > > > > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > > > > driver code and the setting here [0]. There is no other reason than > > > > > > > > that. I am ok with removing this from here and testing with my two > > > > > > > > devices that everything is still ok if having this setting in the pci > > > > > > > > controller driver is a real problem. > > > > > > > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > > > > then compare outputs. > > > > > > > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > > > > needed, so it is possible that there would be no difference in lspci > > > > > > > output. > > > > > > > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > > > > review comments come :). > > > > > > > > > > I have tested to remove this and check lspci -nnvv output with and > > > > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > > > > difference between them. Also, both boards are working without > > > > > regressions at all. So I will remove this code for next version. > > > > > > > > Perfect! > > > > > > > > > Thanks, > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > > > > > > > Best regards, > > > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > > > > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > > > > as MT specific register. But from this code for me it looks like that it > > > > just access config space of some device and therefore it could be some > > > > standard PCIe register. Just with hardcoded calculated offset. > > > > So based on your lspci output, there is no PCIe capability register at > > address PCIE_FTS_NUM (0x70c), right? It seems strange to trying access > > capability register outside of capability list. > > This setting is also in the original driver code from mediatek SDK. See [0]. > Anyway, I have tried to remove this code and test what happens with my > two boards and both of them seem to > work properly just by deleting this code. Looks like that this code is some designware stuff which seems to be needed. So it would be better to let it here as is. > > > > Could you provide output from lspci -nnvv? So other people could look at > > > > it and maybe we decode what is this code doing and if it is needed. > > > > > > # lspci -nnvv > > > 00:02.0 PCI bridge [0604]: Device [0e8d:0801] (rev 01) (prog-if 00 > > > [Normal decode]) > > > > Hm... Device address is 02. But in your code is: > > > > u8 num_slots_enabled = 0; > > ... > > list_for_each_entry(port, &pcie->ports, list) { > > if (port->enabled) { > > ... > > num_slots_enabled++; > > ... > > } > > } > > ... > > for (slot = 0; slot < num_slots_enabled; slot++) { > > val = read_config(pcie, slot, ...); > > ... > > write_config(pcie, slot, ...); > > } > > > > Which means that this code writes to config space of wrong device 0 > > (instead of 2)! In function write_config() can be seen that second > > parameter specify device of BDF address for bus=0 and function=0. > > Bridge enumeration depends on a virtual bridge register configuration. > But at the end devices connected to the bridge > are enumerated as 01:00.0, 02:00.0 and 03:00.0. So in this case the > phy used is the one for "pcie2" (00:02.0) and the device connected to > it > is 01:00.0. For example a board using all the virtual bridges will get > an output similar to: > > [ 16.487166] mt7621-pci 1e140000.pcie: host bridge /pcie@1e140000 ranges: > [ 16.500627] mt7621-pci 1e140000.pcie: No bus range found for > /pcie@1e140000, using [bus 00-ff] > [ 16.518212] mt7621-pci 1e140000.pcie: MEM > 0x0060000000..0x006fffffff -> 0x0000000000 > [ 16.534531] mt7621-pci 1e140000.pcie: IO > 0x001e160000..0x001e16ffff -> 0x0000000000 > [ 16.786498] mt7621-pci 1e140000.pcie: PCIE0 enabled > [ 16.796220] mt7621-pci 1e140000.pcie: PCIE1 enabled > [ 16.805943] mt7621-pci 1e140000.pcie: PCIE2 enabled > [ 16.815664] mt7621-pci 1e140000.pcie: PCI coherence region base: > 0x60000000, mask/settings: 0xf0000002 > [ 16.834398] mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00 > [ 16.847098] pci_bus 0000:00: root bus resource [io 0x1e160000-0x1e16ffff] > [ 16.860806] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff] > [ 16.874504] pci_bus 0000:00: root bus resource [bus 00-ff] > [ 16.885441] pci_bus 0000:00: root bus resource [mem > 0x60000000-0x6fffffff] (bus address [0x00000000-0x0fffffff]) > [ 16.905773] pci 0000:00:00.0: [0e8d:0801] type 01 class 0x060400 > [ 16.917772] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x7fffffff] > [ 16.930260] pci 0000:00:00.0: reg 0x14: initial BAR value 0x00000000 invalid > [ 16.944304] pci 0000:00:00.0: reg 0x14: [mem size 0x00010000] > [ 16.955841] pci 0000:00:00.0: supports D1 > [ 16.963837] pci 0000:00:00.0: PME# supported from D0 D1 D3hot > [ 16.975747] pci 0000:00:01.0: [0e8d:0801] type 01 class 0x060400 > [ 16.987772] pci 0000:00:01.0: reg 0x10: [mem 0x00000000-0x7fffffff] > [ 17.000273] pci 0000:00:01.0: reg 0x14: initial BAR value 0x00000000 invalid > [ 17.014311] pci 0000:00:01.0: reg 0x14: [mem size 0x00010000] > [ 17.025838] pci 0000:00:01.0: supports D1 > [ 17.033828] pci 0000:00:01.0: PME# supported from D0 D1 D3hot > [ 17.045699] pci 0000:00:02.0: [0e8d:0801] type 01 class 0x060400 > [ 17.057726] pci 0000:00:02.0: reg 0x10: [mem 0x00000000-0x7fffffff] > [ 17.070218] pci 0000:00:02.0: reg 0x14: initial BAR value 0x00000000 invalid > [ 17.084260] pci 0000:00:02.0: reg 0x14: [mem size 0x00010000] > [ 17.095788] pci 0000:00:02.0: supports D1 > [ 17.103785] pci 0000:00:02.0: PME# supported from D0 D1 D3hot > [ 17.116598] pci 0000:00:00.0: bridge configuration invalid ([bus > 00-00]), reconfiguring > [ 17.132566] pci 0000:00:01.0: bridge configuration invalid ([bus > 00-00]), reconfiguring > [ 17.148514] pci 0000:00:02.0: bridge configuration invalid ([bus > 00-00]), reconfiguring > [ 17.164739] pci 0000:01:00.0: [1b21:0611] type 00 class 0x010185 > [ 17.176775] pci 0000:01:00.0: reg 0x10: [io 0x0000-0x0007] > [ 17.187892] pci 0000:01:00.0: reg 0x14: [io 0x0000-0x0003] > [ 17.199009] pci 0000:01:00.0: reg 0x18: [io 0x0000-0x0007] > [ 17.210130] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x0003] > [ 17.221246] pci 0000:01:00.0: reg 0x20: [io 0x0000-0x000f] > [ 17.232357] pci 0000:01:00.0: reg 0x24: initial BAR value 0x00000000 invalid > [ 17.246411] pci 0000:01:00.0: reg 0x24: [mem size 0x00000200] > [ 17.258031] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, > limited by 2.5 GT/s PCIe x1 link at 0000:00:00.0 (capable of 4.000 > Gb/s with 5.0 GT/s PCIe x1 link) > [ 17.317563] pci 0000:00:00.0: PCI bridge to [bus 01-ff] > [ 17.328033] pci 0000:00:00.0: bridge window [io 0x0000-0x0fff] > [ 17.340179] pci 0000:00:00.0: bridge window [mem 0x60000000-0x600fffff] > [ 17.353703] pci 0000:00:00.0: bridge window [mem > 0x60000000-0x600fffff pref] > [ 17.368097] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 > [ 17.381550] pci 0000:02:00.0: [1b21:0611] type 00 class 0x010185 > [ 17.393573] pci 0000:02:00.0: reg 0x10: [io 0x0000-0x0007] > [ 17.404694] pci 0000:02:00.0: reg 0x14: [io 0x0000-0x0003] > [ 17.415804] pci 0000:02:00.0: reg 0x18: [io 0x0000-0x0007] > [ 17.426916] pci 0000:02:00.0: reg 0x1c: [io 0x0000-0x0003] > [ 17.438040] pci 0000:02:00.0: reg 0x20: [io 0x0000-0x000f] > [ 17.449148] pci 0000:02:00.0: reg 0x24: initial BAR value 0x00000000 invalid > [ 17.463193] pci 0000:02:00.0: reg 0x24: [mem size 0x00000200] > [ 17.474803] pci 0000:02:00.0: 2.000 Gb/s available PCIe bandwidth, > limited by 2.5 GT/s PCIe x1 link at 0000:00:01.0 (capable of 4.000 > Gb/s with 5.0 GT/s PCIe x1 link) > [ 17.527554] pci 0000:00:01.0: PCI bridge to [bus 02-ff] > [ 17.538014] pci 0000:00:01.0: bridge window [io 0x0000-0x0fff] > [ 17.550159] pci 0000:00:01.0: bridge window [mem 0x60000000-0x600fffff] > [ 17.563682] pci 0000:00:01.0: bridge window [mem > 0x60000000-0x600fffff pref] > [ 17.578078] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02 > [ 17.591529] pci 0000:03:00.0: [1b21:0611] type 00 class 0x010185 > [ 17.603545] pci 0000:03:00.0: reg 0x10: [io 0x0000-0x0007] > [ 17.614665] pci 0000:03:00.0: reg 0x14: [io 0x0000-0x0003] > [ 17.625775] pci 0000:03:00.0: reg 0x18: [io 0x0000-0x0007] > [ 17.636887] pci 0000:03:00.0: reg 0x1c: [io 0x0000-0x0003] > [ 17.648009] pci 0000:03:00.0: reg 0x20: [io 0x0000-0x000f] > [ 17.659119] pci 0000:03:00.0: reg 0x24: initial BAR value 0x00000000 invalid > [ 17.673162] pci 0000:03:00.0: reg 0x24: [mem size 0x00000200] > [ 17.684777] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, > limited by 2.5 GT/s PCIe x1 link at 0000:00:02.0 (capable of 4.000 > Gb/s with 5.0 GT/s PCIe x1 link) > [ 17.737561] pci 0000:00:02.0: PCI bridge to [bus 03-ff] > [ 17.748022] pci 0000:00:02.0: bridge window [io 0x0000-0x0fff] > [ 17.760167] pci 0000:00:02.0: bridge window [mem 0x60000000-0x600fffff] > [ 17.773690] pci 0000:00:02.0: bridge window [mem > 0x60000000-0x600fffff pref] > [ 17.788085] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03 > [ 17.801341] pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000] > [ 17.814518] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000] > [ 17.828392] pci 0000:00:01.0: BAR 0: no space for [mem size 0x80000000] > [ 17.841571] pci 0000:00:01.0: BAR 0: failed to assign [mem size 0x80000000] > [ 17.855443] pci 0000:00:02.0: BAR 0: no space for [mem size 0x80000000] > [ 17.868626] pci 0000:00:02.0: BAR 0: failed to assign [mem size 0x80000000] > [ 17.882502] pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x600fffff] > [ 17.896028] pci 0000:00:00.0: BAR 9: assigned [mem > 0x60100000-0x601fffff pref] > [ 17.910419] pci 0000:00:01.0: BAR 8: assigned [mem 0x60200000-0x602fffff] > [ 17.923951] pci 0000:00:01.0: BAR 9: assigned [mem > 0x60300000-0x603fffff pref] > [ 17.938353] pci 0000:00:02.0: BAR 8: assigned [mem 0x60400000-0x604fffff] > [ 17.951879] pci 0000:00:02.0: BAR 9: assigned [mem > 0x60500000-0x605fffff pref] > [ 17.966273] pci 0000:00:00.0: BAR 1: assigned [mem 0x60600000-0x6060ffff] > [ 17.979809] pci 0000:00:01.0: BAR 1: assigned [mem 0x60610000-0x6061ffff] > [ 17.993341] pci 0000:00:02.0: BAR 1: assigned [mem 0x60620000-0x6062ffff] > [ 18.006877] pci 0000:00:00.0: BAR 7: assigned [io 0x1e160000-0x1e160fff] > [ 18.020408] pci 0000:00:01.0: BAR 7: assigned [io 0x1e161000-0x1e161fff] > [ 18.033932] pci 0000:00:02.0: BAR 7: assigned [io 0x1e162000-0x1e162fff] > [ 18.047472] pci 0000:01:00.0: BAR 5: assigned [mem 0x60000000-0x600001ff] > [ 18.061005] pci 0000:01:00.0: BAR 4: assigned [io 0x1e160000-0x1e16000f] > [ 18.074540] pci 0000:01:00.0: BAR 0: assigned [io 0x1e160010-0x1e160017] > [ 18.088070] pci 0000:01:00.0: BAR 2: assigned [io 0x1e160018-0x1e16001f] > [ 18.101606] pci 0000:01:00.0: BAR 1: assigned [io 0x1e160020-0x1e160023] > [ 18.115139] pci 0000:01:00.0: BAR 3: assigned [io 0x1e160024-0x1e160027] > [ 18.128676] pci 0000:00:00.0: PCI bridge to [bus 01] > [ 18.138577] pci 0000:00:00.0: bridge window [io 0x1e160000-0x1e160fff] > [ 18.152100] pci 0000:00:00.0: bridge window [mem 0x60000000-0x600fffff] > [ 18.165627] pci 0000:00:00.0: bridge window [mem > 0x60100000-0x601fffff pref] > [ 18.180030] pci 0000:02:00.0: BAR 5: assigned [mem 0x60200000-0x602001ff] > [ 18.193566] pci 0000:02:00.0: BAR 4: assigned [io 0x1e161000-0x1e16100f] > [ 18.207110] pci 0000:02:00.0: BAR 0: assigned [io 0x1e161010-0x1e161017] > [ 18.220648] pci 0000:02:00.0: BAR 2: assigned [io 0x1e161018-0x1e16101f] > [ 18.234183] pci 0000:02:00.0: BAR 1: assigned [io 0x1e161020-0x1e161023] > [ 18.247723] pci 0000:02:00.0: BAR 3: assigned [io 0x1e161024-0x1e161027] > [ 18.261257] pci 0000:00:01.0: PCI bridge to [bus 02] > [ 18.271165] pci 0000:00:01.0: bridge window [io 0x1e161000-0x1e161fff] > [ 18.284695] pci 0000:00:01.0: bridge window [mem 0x60200000-0x602fffff] > [ 18.298225] pci 0000:00:01.0: bridge window [mem > 0x60300000-0x603fffff pref] > [ 18.312630] pci 0000:03:00.0: BAR 5: assigned [mem 0x60400000-0x604001ff] > [ 18.326166] pci 0000:03:00.0: BAR 4: assigned [io 0x1e162000-0x1e16200f] > [ 18.339702] pci 0000:03:00.0: BAR 0: assigned [io 0x1e162010-0x1e162017] > [ 18.353237] pci 0000:03:00.0: BAR 2: assigned [io 0x1e162018-0x1e16201f] > [ 18.366775] pci 0000:03:00.0: BAR 1: assigned [io 0x1e162020-0x1e162023] > [ 18.380311] pci 0000:03:00.0: BAR 3: assigned [io 0x1e162024-0x1e162027] > [ 18.393841] pci 0000:00:02.0: PCI bridge to [bus 03] > [ 18.403740] pci 0000:00:02.0: bridge window [io 0x1e162000-0x1e162fff] > [ 18.417270] pci 0000:00:02.0: bridge window [mem 0x60400000-0x604fffff] > [ 18.430801] pci 0000:00:02.0: bridge window [mem > 0x60500000-0x605fffff pref] > [ 18.445529] ahci 0000:01:00.0: version 3.0 > [ 18.445559] pci 0000:00:00.0: enabling device (0000 -> 0003) > [ 18.456853] ahci 0000:01:00.0: enabling device (0000 -> 0003) > [ 18.468455] ahci 0000:01:00.0: SSS flag set, parallel bus scan disabled > [ 18.481700] ahci 0000:01:00.0: AHCI 0001.0200 32 slots 2 ports 6 > Gbps 0x3 impl IDE mode > [ 18.497662] ahci 0000:01:00.0: flags: 64bit ncq sntf stag led clo > pmp pio slum part ccc sxs > [ 18.516777] scsi host0: ahci > [ 18.523629] scsi host1: ahci > [ 18.529829] ata1: SATA max UDMA/133 abar m512@0x60000000 port > 0x60000100 irq 22 > [ 18.544447] ata2: SATA max UDMA/133 abar m512@0x60000000 port > 0x60000180 irq 22 > [ 18.559465] pci 0000:00:01.0: enabling device (0000 -> 0003) > [ 18.570786] ahci 0000:02:00.0: enabling device (0000 -> 0003) > [ 18.582414] ahci 0000:02:00.0: SSS flag set, parallel bus scan disabled > [ 18.595665] ahci 0000:02:00.0: AHCI 0001.0200 32 slots 2 ports 6 > Gbps 0x3 impl IDE mode > [ 18.611614] ahci 0000:02:00.0: flags: 64bit ncq sntf stag led clo > pmp pio slum part ccc sxs > [ 18.631053] scsi host2: ahci > [ 18.637983] scsi host3: ahci > [ 18.644138] ata3: SATA max UDMA/133 abar m512@0x60200000 port > 0x60200100 irq 23 > [ 18.658792] ata4: SATA max UDMA/133 abar m512@0x60200000 port > 0x60200180 irq 23 > [ 18.673827] pci 0000:00:02.0: enabling device (0000 -> 0003) > [ 18.685151] ahci 0000:03:00.0: enabling device (0000 -> 0003) > [ 18.696782] ahci 0000:03:00.0: SSS flag set, parallel bus scan disabled > [ 18.710025] ahci 0000:03:00.0: AHCI 0001.0200 32 slots 2 ports 6 > Gbps 0x3 impl IDE mode > [ 18.725972] ahci 0000:03:00.0: flags: 64bit ncq sntf stag led clo > pmp pio slum part ccc sxs > > And you are totally right, the setting is writing in the wrong place. > I changed the device tree and the way interrupts are mapped > to avoid using a custom 'map_irq' function [1]. Before that commit the > pci virtual bridge register was reordering the > buses enumeration depending on link status, so there I should also > properly rewrite the code in question. > > I can rewrite the code to read and write config properly using the > slot moving the code into 'mt7621_pcie_enable_port' as follows: > > static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port) > { > struct mt7621_pcie *pcie = port->pcie; > u32 slot = port->slot; > u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT); > u32 val; > > /* enable pcie interrupt */ > val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR); > val |= PCIE_PORT_INT_EN(slot); > > /* map 2G DDR region */ > pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE, > offset + RALINK_PCI_BAR0SETUP_ADDR); > > /* configure class code and revision ID */ > pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID, > offset + RALINK_PCI_CLASS); > > /* configure RC FTS number to 250 when it leaves L0s */ > val = read_config(pcie, slot, PCIE_FTS_NUM); > val &= ~PCIE_FTS_NUM_MASK; > val |= PCIE_FTS_NUM_L0(0x50); > write_config(pcie, slot, PCIE_FTS_NUM, val); > } > > static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > { > struct device *dev = pcie->dev; > struct mt7621_pcie_port *port; > int err; > > /* Setup MEMWIN and IOWIN */ > pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > list_for_each_entry(port, &pcie->ports, list) { > if (port->enabled) { > err = clk_prepare_enable(port->clk); > if (err) { > dev_err(dev, "enabling clk pcie%d\n", > port->slot); > return err; > } > > mt7621_pcie_enable_port(port); > dev_info(dev, "PCIE%d enabled\n", port->slot); > } > } > > return 0; > } > > Or just delete the setting and the read and write config functions > since they are not being used in any other place. My two boards work > without this setting but I don't know about other boards. > > What do you think? I think that your above modification is fine and make code more straightforward. Thanks for looking at it! > Best regards, > Sergio Paracuellos > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n663 > [1]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/commit/drivers/staging/mt7621-pci?h=staging-testing&id=b99cc3a2b6b62cf994acac5cced03298d9908c9b
Hi Pali, On Sat, Jun 5, 2021 at 12:58 AM Pali Rohár <pali@kernel.org> wrote: > > On Friday 04 June 2021 13:49:39 Rob Herring wrote: > > On Fri, Jun 04, 2021 at 06:55:25PM +0200, Pali Rohár wrote: > > > On Wednesday 02 June 2021 14:43:53 Sergio Paracuellos wrote: > > > > Hi Pali, > > > > > > > > On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > > > > > Hi Pali, > > > > > > > > > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > > > > > <sergio.paracuellos@gmail.com> wrote: > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > > > > > Hi Pali, > > > > > > > > > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > > > > > --- > > > > > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > > > > > # > > > > > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > > > > > # > > > > > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > > > > > new file mode 100644 > > > > > > > > > > > index 000000000000..fe1945819d25 > > > > > > > > > > > --- /dev/null > > > > > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > > > > > ... > > > > > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > > > > > +{ > > > > > > > > > > > + struct device *dev = pcie->dev; > > > > > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > > > > > + u32 slot; > > > > > > > > > > > + u32 val; > > > > > > > > > > > + int err; > > > > > > > > > > > + > > > > > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > > > > > + > > > > > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > > > > > + if (port->enabled) { > > > > > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > > > > > + if (err) { > > > > > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > > > > > + return err; > > > > > > > > > > > + } > > > > > > > > > > > + > > > > > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > > > > > + num_slots_enabled++; > > > > > > > > > > > + } > > > > > > > > > > > + } > > > > > > > > > > > + > > > > > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > > > > > driver initialize itself. > > > > > > > > > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > > > > > which is used by device drivers. > > > > > > > > > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > > > > > > > > > For sure it is :(. > > > > > > > > > > > > > > > > > > > > > > > > I have > > > > > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > > > > > driver code and the setting here [0]. There is no other reason than > > > > > > > > > that. I am ok with removing this from here and testing with my two > > > > > > > > > devices that everything is still ok if having this setting in the pci > > > > > > > > > controller driver is a real problem. > > > > > > > > > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > > > > > then compare outputs. > > > > > > > > > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > > > > > needed, so it is possible that there would be no difference in lspci > > > > > > > > output. > > > > > > > > > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > > > > > review comments come :). > > > > > > > > > > > > I have tested to remove this and check lspci -nnvv output with and > > > > > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > > > > > difference between them. Also, both boards are working without > > > > > > regressions at all. So I will remove this code for next version. > > > > > > > > > > Perfect! > > > > > > > > > > > Thanks, > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > > > > > > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > > > > > as MT specific register. But from this code for me it looks like that it > > > > > just access config space of some device and therefore it could be some > > > > > standard PCIe register. Just with hardcoded calculated offset. > > > > > > So based on your lspci output, there is no PCIe capability register at > > > address PCIE_FTS_NUM (0x70c), right? It seems strange to trying access > > > capability register outside of capability list. > > > > Looks like a DW PCIe port logic register: > > > > drivers/pci/controller/dwc/pcie-designware.h-/* Synopsys-specific PCIe configuration registers */ > > drivers/pci/controller/dwc/pcie-designware.h:#define PCIE_PORT_AFR 0x70C > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_N_FTS_MASK GENMASK(15, 8) > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_N_FTS(n) FIELD_PREP(PORT_AFR_N_FTS_MASK, n) > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_CC_N_FTS_MASK GENMASK(23, 16) > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_CC_N_FTS(n) FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, n) > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_ENTER_ASPM BIT(30) > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L0S_ENTRANCE_LAT_SHIFT 24 > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L0S_ENTRANCE_LAT_MASK GENMASK(26, 24) > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L1_ENTRANCE_LAT_SHIFT 27 > > drivers/pci/controller/dwc/pcie-designware.h-#define PORT_AFR_L1_ENTRANCE_LAT_MASK GENMASK(29, 27) > > > > Rob > > Rob: nice catch! > Does it mean that this MT7621 SoC has dwc controller and driver can be > theoretically in future rewritten to use common dwc code? > > Sergio: I have tried to find some information about it and seems that > MT7620, MT7621 and MT7628 SoC are really using some designware dwc IP. > Some details are available in section "Embedded/kernel developer > friendliness" in following blog post: > https://www.abclinuxu.cz/blog/GardenOfEdenConfiguration/2019/10/opus-magnum Thanks. For what see in the datasheet linked there some of the registers looks pretty similar. It also defines registers taking in account three root ports but in mt7628 and mt7620 there is only one available. It seems tons of things are not listed there but it is better than nothing. > > And seems that "programming guide" documentation for MT7620 is available > on internet with description of PCIe registers. I do not know how MT7620 > and MT7621 are different but maybe it could help to develop or understand > driver. Yes, it is available but none of the dwc stuff is inside. Only the main register as the mt7628 datasheet has. Best regards, Sergio Paracuellos
Hi Pali, On Sat, Jun 5, 2021 at 1:07 AM Pali Rohár <pali@kernel.org> wrote: > > On Friday 04 June 2021 20:44:42 Sergio Paracuellos wrote: > > Hi Pali, > > > > Thanks for your comments. > > > > On Fri, Jun 4, 2021 at 6:55 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > On Wednesday 02 June 2021 14:43:53 Sergio Paracuellos wrote: > > > > Hi Pali, > > > > > > > > On Wed, Jun 2, 2021 at 2:23 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > On Wednesday 02 June 2021 14:16:26 Sergio Paracuellos wrote: > > > > > > Hi Pali, > > > > > > > > > > > > On Mon, May 31, 2021 at 4:19 PM Sergio Paracuellos > > > > > > <sergio.paracuellos@gmail.com> wrote: > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:50 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > On Monday 31 May 2021 15:39:55 Sergio Paracuellos wrote: > > > > > > > > > Hi Pali, > > > > > > > > > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > > > > > On Mon, May 31, 2021 at 3:14 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > > > > > > > > > > > > > On Saturday 15 May 2021 14:40:53 Sergio Paracuellos wrote: > > > > > > > > > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > > > > > > > > > > > > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > > > > > > > > > --- > > > > > > > > > > > arch/mips/pci/Makefile | 1 + > > > > > > > > > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > > > > > > > > > arch/mips/ralink/Kconfig | 9 +- > > > > > > > > > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > > > > > > > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > > > > > > > > > > > > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > > > > > > > > > index f3eecc065e5c..178c550739c4 100644 > > > > > > > > > > > --- a/arch/mips/pci/Makefile > > > > > > > > > > > +++ b/arch/mips/pci/Makefile > > > > > > > > > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > > > > > > > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > > > > > > > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > > > > > > > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > > > > > > > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > > > > > > > > > # > > > > > > > > > > > # These are still pretty much in the old state, watch, go blind. > > > > > > > > > > > # > > > > > > > > > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > > > > > > > > > new file mode 100644 > > > > > > > > > > > index 000000000000..fe1945819d25 > > > > > > > > > > > --- /dev/null > > > > > > > > > > > +++ b/arch/mips/pci/pci-mt7621.c > > > > > > > > > > ... > > > > > > > > > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > > > > > > > > > +{ > > > > > > > > > > > + struct device *dev = pcie->dev; > > > > > > > > > > > + struct mt7621_pcie_port *port; > > > > > > > > > > > + u8 num_slots_enabled = 0; > > > > > > > > > > > + u32 slot; > > > > > > > > > > > + u32 val; > > > > > > > > > > > + int err; > > > > > > > > > > > + > > > > > > > > > > > + /* Setup MEMWIN and IOWIN */ > > > > > > > > > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > > > > > > > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > > > > > > > > + > > > > > > > > > > > + list_for_each_entry(port, &pcie->ports, list) { > > > > > > > > > > > + if (port->enabled) { > > > > > > > > > > > + err = clk_prepare_enable(port->clk); > > > > > > > > > > > + if (err) { > > > > > > > > > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > > > > > > > > > + return err; > > > > > > > > > > > + } > > > > > > > > > > > + > > > > > > > > > > > + mt7621_pcie_enable_port(port); > > > > > > > > > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > > > > > > > > > + num_slots_enabled++; > > > > > > > > > > > + } > > > > > > > > > > > + } > > > > > > > > > > > + > > > > > > > > > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > > > > > > > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > > > > > > > > > + val |= PCI_COMMAND_MASTER; > > > > > > > > > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > > > > > > > > > > > > > > > > > > Hello! Is this part of code correct? Because it looks strange if PCIe > > > > > > > > > > controller driver automatically enables PCI bus mastering, prior device > > > > > > > > > > driver initialize itself. > > > > > > > > > > > > > > > > > > > > Moreover kernel has already function pci_set_master() for this purpose > > > > > > > > > > which is used by device drivers. > > > > > > > > > > > > > > > > > > > > So I think this code can confuse some device drivers... > > > > > > > > > > > > > > > > > > I agree that we have pci_set_master() to be used in pci device driver > > > > > > > > > code. Original controller driver set this bit for enabled slots. Since > > > > > > > > > there is no documentation at all for the PCI in this SoC > > > > > > > > > > > > > > > > I see... this is really a big problem to do any driver development... > > > > > > > > > > > > > > For sure it is :(. > > > > > > > > > > > > > > > > > > > > > > > > I have > > > > > > > > > maintained the setting in the driver in a cleaner way. See original > > > > > > > > > driver code and the setting here [0]. There is no other reason than > > > > > > > > > that. I am ok with removing this from here and testing with my two > > > > > > > > > devices that everything is still ok if having this setting in the pci > > > > > > > > > controller driver is a real problem. > > > > > > > > > > > > > > > > You can run lspci -nnvv with and without PCI_COMMAND_MASTER code and > > > > > > > > then compare outputs. > > > > > > > > > > > > > > > > Device drivers for sure enable PCI_COMMAND_MASTER at the time when it is > > > > > > > > needed, so it is possible that there would be no difference in lspci > > > > > > > > output. > > > > > > > > > > > > > > Thanks. I will take this into account when v2 is submitted after more > > > > > > > review comments come :). > > > > > > > > > > > > I have tested to remove this and check lspci -nnvv output with and > > > > > > without PCI_COMMAND_MASTER code and, as you pointed out, there is no > > > > > > difference between them. Also, both boards are working without > > > > > > regressions at all. So I will remove this code for next version. > > > > > > > > > > Perfect! > > > > > > > > > > > Thanks, > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > > > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n676 > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > Sergio Paracuellos > > > > > > > > > > > > > > > > > > > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > > > > > > > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > > > > > > > > > + val &= ~PCIE_FTS_NUM_MASK; > > > > > > > > > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > > > > > > > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > > > > > > > > Could you look also what is doing this code (PCIE_FTS_NUM)? It is marked > > > > > as MT specific register. But from this code for me it looks like that it > > > > > just access config space of some device and therefore it could be some > > > > > standard PCIe register. Just with hardcoded calculated offset. > > > > > > So based on your lspci output, there is no PCIe capability register at > > > address PCIE_FTS_NUM (0x70c), right? It seems strange to trying access > > > capability register outside of capability list. > > > > This setting is also in the original driver code from mediatek SDK. See [0]. > > Anyway, I have tried to remove this code and test what happens with my > > two boards and both of them seem to > > work properly just by deleting this code. > > Looks like that this code is some designware stuff which seems to be > needed. So it would be better to let it here as is. Ok, will do. > > > > > > Could you provide output from lspci -nnvv? So other people could look at > > > > > it and maybe we decode what is this code doing and if it is needed. > > > > > > > > # lspci -nnvv > > > > 00:02.0 PCI bridge [0604]: Device [0e8d:0801] (rev 01) (prog-if 00 > > > > [Normal decode]) > > > > > > Hm... Device address is 02. But in your code is: > > > > > > u8 num_slots_enabled = 0; > > > ... > > > list_for_each_entry(port, &pcie->ports, list) { > > > if (port->enabled) { > > > ... > > > num_slots_enabled++; > > > ... > > > } > > > } > > > ... > > > for (slot = 0; slot < num_slots_enabled; slot++) { > > > val = read_config(pcie, slot, ...); > > > ... > > > write_config(pcie, slot, ...); > > > } > > > > > > Which means that this code writes to config space of wrong device 0 > > > (instead of 2)! In function write_config() can be seen that second > > > parameter specify device of BDF address for bus=0 and function=0. > > > > Bridge enumeration depends on a virtual bridge register configuration. > > But at the end devices connected to the bridge > > are enumerated as 01:00.0, 02:00.0 and 03:00.0. So in this case the > > phy used is the one for "pcie2" (00:02.0) and the device connected to > > it > > is 01:00.0. For example a board using all the virtual bridges will get > > an output similar to: > > > > [ 16.487166] mt7621-pci 1e140000.pcie: host bridge /pcie@1e140000 ranges: > > [ 16.500627] mt7621-pci 1e140000.pcie: No bus range found for > > /pcie@1e140000, using [bus 00-ff] > > [ 16.518212] mt7621-pci 1e140000.pcie: MEM > > 0x0060000000..0x006fffffff -> 0x0000000000 > > [ 16.534531] mt7621-pci 1e140000.pcie: IO > > 0x001e160000..0x001e16ffff -> 0x0000000000 > > [ 16.786498] mt7621-pci 1e140000.pcie: PCIE0 enabled > > [ 16.796220] mt7621-pci 1e140000.pcie: PCIE1 enabled > > [ 16.805943] mt7621-pci 1e140000.pcie: PCIE2 enabled > > [ 16.815664] mt7621-pci 1e140000.pcie: PCI coherence region base: > > 0x60000000, mask/settings: 0xf0000002 > > [ 16.834398] mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00 > > [ 16.847098] pci_bus 0000:00: root bus resource [io 0x1e160000-0x1e16ffff] > > [ 16.860806] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff] > > [ 16.874504] pci_bus 0000:00: root bus resource [bus 00-ff] > > [ 16.885441] pci_bus 0000:00: root bus resource [mem > > 0x60000000-0x6fffffff] (bus address [0x00000000-0x0fffffff]) > > [ 16.905773] pci 0000:00:00.0: [0e8d:0801] type 01 class 0x060400 > > [ 16.917772] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x7fffffff] > > [ 16.930260] pci 0000:00:00.0: reg 0x14: initial BAR value 0x00000000 invalid > > [ 16.944304] pci 0000:00:00.0: reg 0x14: [mem size 0x00010000] > > [ 16.955841] pci 0000:00:00.0: supports D1 > > [ 16.963837] pci 0000:00:00.0: PME# supported from D0 D1 D3hot > > [ 16.975747] pci 0000:00:01.0: [0e8d:0801] type 01 class 0x060400 > > [ 16.987772] pci 0000:00:01.0: reg 0x10: [mem 0x00000000-0x7fffffff] > > [ 17.000273] pci 0000:00:01.0: reg 0x14: initial BAR value 0x00000000 invalid > > [ 17.014311] pci 0000:00:01.0: reg 0x14: [mem size 0x00010000] > > [ 17.025838] pci 0000:00:01.0: supports D1 > > [ 17.033828] pci 0000:00:01.0: PME# supported from D0 D1 D3hot > > [ 17.045699] pci 0000:00:02.0: [0e8d:0801] type 01 class 0x060400 > > [ 17.057726] pci 0000:00:02.0: reg 0x10: [mem 0x00000000-0x7fffffff] > > [ 17.070218] pci 0000:00:02.0: reg 0x14: initial BAR value 0x00000000 invalid > > [ 17.084260] pci 0000:00:02.0: reg 0x14: [mem size 0x00010000] > > [ 17.095788] pci 0000:00:02.0: supports D1 > > [ 17.103785] pci 0000:00:02.0: PME# supported from D0 D1 D3hot > > [ 17.116598] pci 0000:00:00.0: bridge configuration invalid ([bus > > 00-00]), reconfiguring > > [ 17.132566] pci 0000:00:01.0: bridge configuration invalid ([bus > > 00-00]), reconfiguring > > [ 17.148514] pci 0000:00:02.0: bridge configuration invalid ([bus > > 00-00]), reconfiguring > > [ 17.164739] pci 0000:01:00.0: [1b21:0611] type 00 class 0x010185 > > [ 17.176775] pci 0000:01:00.0: reg 0x10: [io 0x0000-0x0007] > > [ 17.187892] pci 0000:01:00.0: reg 0x14: [io 0x0000-0x0003] > > [ 17.199009] pci 0000:01:00.0: reg 0x18: [io 0x0000-0x0007] > > [ 17.210130] pci 0000:01:00.0: reg 0x1c: [io 0x0000-0x0003] > > [ 17.221246] pci 0000:01:00.0: reg 0x20: [io 0x0000-0x000f] > > [ 17.232357] pci 0000:01:00.0: reg 0x24: initial BAR value 0x00000000 invalid > > [ 17.246411] pci 0000:01:00.0: reg 0x24: [mem size 0x00000200] > > [ 17.258031] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, > > limited by 2.5 GT/s PCIe x1 link at 0000:00:00.0 (capable of 4.000 > > Gb/s with 5.0 GT/s PCIe x1 link) > > [ 17.317563] pci 0000:00:00.0: PCI bridge to [bus 01-ff] > > [ 17.328033] pci 0000:00:00.0: bridge window [io 0x0000-0x0fff] > > [ 17.340179] pci 0000:00:00.0: bridge window [mem 0x60000000-0x600fffff] > > [ 17.353703] pci 0000:00:00.0: bridge window [mem > > 0x60000000-0x600fffff pref] > > [ 17.368097] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 > > [ 17.381550] pci 0000:02:00.0: [1b21:0611] type 00 class 0x010185 > > [ 17.393573] pci 0000:02:00.0: reg 0x10: [io 0x0000-0x0007] > > [ 17.404694] pci 0000:02:00.0: reg 0x14: [io 0x0000-0x0003] > > [ 17.415804] pci 0000:02:00.0: reg 0x18: [io 0x0000-0x0007] > > [ 17.426916] pci 0000:02:00.0: reg 0x1c: [io 0x0000-0x0003] > > [ 17.438040] pci 0000:02:00.0: reg 0x20: [io 0x0000-0x000f] > > [ 17.449148] pci 0000:02:00.0: reg 0x24: initial BAR value 0x00000000 invalid > > [ 17.463193] pci 0000:02:00.0: reg 0x24: [mem size 0x00000200] > > [ 17.474803] pci 0000:02:00.0: 2.000 Gb/s available PCIe bandwidth, > > limited by 2.5 GT/s PCIe x1 link at 0000:00:01.0 (capable of 4.000 > > Gb/s with 5.0 GT/s PCIe x1 link) > > [ 17.527554] pci 0000:00:01.0: PCI bridge to [bus 02-ff] > > [ 17.538014] pci 0000:00:01.0: bridge window [io 0x0000-0x0fff] > > [ 17.550159] pci 0000:00:01.0: bridge window [mem 0x60000000-0x600fffff] > > [ 17.563682] pci 0000:00:01.0: bridge window [mem > > 0x60000000-0x600fffff pref] > > [ 17.578078] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02 > > [ 17.591529] pci 0000:03:00.0: [1b21:0611] type 00 class 0x010185 > > [ 17.603545] pci 0000:03:00.0: reg 0x10: [io 0x0000-0x0007] > > [ 17.614665] pci 0000:03:00.0: reg 0x14: [io 0x0000-0x0003] > > [ 17.625775] pci 0000:03:00.0: reg 0x18: [io 0x0000-0x0007] > > [ 17.636887] pci 0000:03:00.0: reg 0x1c: [io 0x0000-0x0003] > > [ 17.648009] pci 0000:03:00.0: reg 0x20: [io 0x0000-0x000f] > > [ 17.659119] pci 0000:03:00.0: reg 0x24: initial BAR value 0x00000000 invalid > > [ 17.673162] pci 0000:03:00.0: reg 0x24: [mem size 0x00000200] > > [ 17.684777] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, > > limited by 2.5 GT/s PCIe x1 link at 0000:00:02.0 (capable of 4.000 > > Gb/s with 5.0 GT/s PCIe x1 link) > > [ 17.737561] pci 0000:00:02.0: PCI bridge to [bus 03-ff] > > [ 17.748022] pci 0000:00:02.0: bridge window [io 0x0000-0x0fff] > > [ 17.760167] pci 0000:00:02.0: bridge window [mem 0x60000000-0x600fffff] > > [ 17.773690] pci 0000:00:02.0: bridge window [mem > > 0x60000000-0x600fffff pref] > > [ 17.788085] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03 > > [ 17.801341] pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000] > > [ 17.814518] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000] > > [ 17.828392] pci 0000:00:01.0: BAR 0: no space for [mem size 0x80000000] > > [ 17.841571] pci 0000:00:01.0: BAR 0: failed to assign [mem size 0x80000000] > > [ 17.855443] pci 0000:00:02.0: BAR 0: no space for [mem size 0x80000000] > > [ 17.868626] pci 0000:00:02.0: BAR 0: failed to assign [mem size 0x80000000] > > [ 17.882502] pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x600fffff] > > [ 17.896028] pci 0000:00:00.0: BAR 9: assigned [mem > > 0x60100000-0x601fffff pref] > > [ 17.910419] pci 0000:00:01.0: BAR 8: assigned [mem 0x60200000-0x602fffff] > > [ 17.923951] pci 0000:00:01.0: BAR 9: assigned [mem > > 0x60300000-0x603fffff pref] > > [ 17.938353] pci 0000:00:02.0: BAR 8: assigned [mem 0x60400000-0x604fffff] > > [ 17.951879] pci 0000:00:02.0: BAR 9: assigned [mem > > 0x60500000-0x605fffff pref] > > [ 17.966273] pci 0000:00:00.0: BAR 1: assigned [mem 0x60600000-0x6060ffff] > > [ 17.979809] pci 0000:00:01.0: BAR 1: assigned [mem 0x60610000-0x6061ffff] > > [ 17.993341] pci 0000:00:02.0: BAR 1: assigned [mem 0x60620000-0x6062ffff] > > [ 18.006877] pci 0000:00:00.0: BAR 7: assigned [io 0x1e160000-0x1e160fff] > > [ 18.020408] pci 0000:00:01.0: BAR 7: assigned [io 0x1e161000-0x1e161fff] > > [ 18.033932] pci 0000:00:02.0: BAR 7: assigned [io 0x1e162000-0x1e162fff] > > [ 18.047472] pci 0000:01:00.0: BAR 5: assigned [mem 0x60000000-0x600001ff] > > [ 18.061005] pci 0000:01:00.0: BAR 4: assigned [io 0x1e160000-0x1e16000f] > > [ 18.074540] pci 0000:01:00.0: BAR 0: assigned [io 0x1e160010-0x1e160017] > > [ 18.088070] pci 0000:01:00.0: BAR 2: assigned [io 0x1e160018-0x1e16001f] > > [ 18.101606] pci 0000:01:00.0: BAR 1: assigned [io 0x1e160020-0x1e160023] > > [ 18.115139] pci 0000:01:00.0: BAR 3: assigned [io 0x1e160024-0x1e160027] > > [ 18.128676] pci 0000:00:00.0: PCI bridge to [bus 01] > > [ 18.138577] pci 0000:00:00.0: bridge window [io 0x1e160000-0x1e160fff] > > [ 18.152100] pci 0000:00:00.0: bridge window [mem 0x60000000-0x600fffff] > > [ 18.165627] pci 0000:00:00.0: bridge window [mem > > 0x60100000-0x601fffff pref] > > [ 18.180030] pci 0000:02:00.0: BAR 5: assigned [mem 0x60200000-0x602001ff] > > [ 18.193566] pci 0000:02:00.0: BAR 4: assigned [io 0x1e161000-0x1e16100f] > > [ 18.207110] pci 0000:02:00.0: BAR 0: assigned [io 0x1e161010-0x1e161017] > > [ 18.220648] pci 0000:02:00.0: BAR 2: assigned [io 0x1e161018-0x1e16101f] > > [ 18.234183] pci 0000:02:00.0: BAR 1: assigned [io 0x1e161020-0x1e161023] > > [ 18.247723] pci 0000:02:00.0: BAR 3: assigned [io 0x1e161024-0x1e161027] > > [ 18.261257] pci 0000:00:01.0: PCI bridge to [bus 02] > > [ 18.271165] pci 0000:00:01.0: bridge window [io 0x1e161000-0x1e161fff] > > [ 18.284695] pci 0000:00:01.0: bridge window [mem 0x60200000-0x602fffff] > > [ 18.298225] pci 0000:00:01.0: bridge window [mem > > 0x60300000-0x603fffff pref] > > [ 18.312630] pci 0000:03:00.0: BAR 5: assigned [mem 0x60400000-0x604001ff] > > [ 18.326166] pci 0000:03:00.0: BAR 4: assigned [io 0x1e162000-0x1e16200f] > > [ 18.339702] pci 0000:03:00.0: BAR 0: assigned [io 0x1e162010-0x1e162017] > > [ 18.353237] pci 0000:03:00.0: BAR 2: assigned [io 0x1e162018-0x1e16201f] > > [ 18.366775] pci 0000:03:00.0: BAR 1: assigned [io 0x1e162020-0x1e162023] > > [ 18.380311] pci 0000:03:00.0: BAR 3: assigned [io 0x1e162024-0x1e162027] > > [ 18.393841] pci 0000:00:02.0: PCI bridge to [bus 03] > > [ 18.403740] pci 0000:00:02.0: bridge window [io 0x1e162000-0x1e162fff] > > [ 18.417270] pci 0000:00:02.0: bridge window [mem 0x60400000-0x604fffff] > > [ 18.430801] pci 0000:00:02.0: bridge window [mem > > 0x60500000-0x605fffff pref] > > [ 18.445529] ahci 0000:01:00.0: version 3.0 > > [ 18.445559] pci 0000:00:00.0: enabling device (0000 -> 0003) > > [ 18.456853] ahci 0000:01:00.0: enabling device (0000 -> 0003) > > [ 18.468455] ahci 0000:01:00.0: SSS flag set, parallel bus scan disabled > > [ 18.481700] ahci 0000:01:00.0: AHCI 0001.0200 32 slots 2 ports 6 > > Gbps 0x3 impl IDE mode > > [ 18.497662] ahci 0000:01:00.0: flags: 64bit ncq sntf stag led clo > > pmp pio slum part ccc sxs > > [ 18.516777] scsi host0: ahci > > [ 18.523629] scsi host1: ahci > > [ 18.529829] ata1: SATA max UDMA/133 abar m512@0x60000000 port > > 0x60000100 irq 22 > > [ 18.544447] ata2: SATA max UDMA/133 abar m512@0x60000000 port > > 0x60000180 irq 22 > > [ 18.559465] pci 0000:00:01.0: enabling device (0000 -> 0003) > > [ 18.570786] ahci 0000:02:00.0: enabling device (0000 -> 0003) > > [ 18.582414] ahci 0000:02:00.0: SSS flag set, parallel bus scan disabled > > [ 18.595665] ahci 0000:02:00.0: AHCI 0001.0200 32 slots 2 ports 6 > > Gbps 0x3 impl IDE mode > > [ 18.611614] ahci 0000:02:00.0: flags: 64bit ncq sntf stag led clo > > pmp pio slum part ccc sxs > > [ 18.631053] scsi host2: ahci > > [ 18.637983] scsi host3: ahci > > [ 18.644138] ata3: SATA max UDMA/133 abar m512@0x60200000 port > > 0x60200100 irq 23 > > [ 18.658792] ata4: SATA max UDMA/133 abar m512@0x60200000 port > > 0x60200180 irq 23 > > [ 18.673827] pci 0000:00:02.0: enabling device (0000 -> 0003) > > [ 18.685151] ahci 0000:03:00.0: enabling device (0000 -> 0003) > > [ 18.696782] ahci 0000:03:00.0: SSS flag set, parallel bus scan disabled > > [ 18.710025] ahci 0000:03:00.0: AHCI 0001.0200 32 slots 2 ports 6 > > Gbps 0x3 impl IDE mode > > [ 18.725972] ahci 0000:03:00.0: flags: 64bit ncq sntf stag led clo > > pmp pio slum part ccc sxs > > > > And you are totally right, the setting is writing in the wrong place. > > I changed the device tree and the way interrupts are mapped > > to avoid using a custom 'map_irq' function [1]. Before that commit the > > pci virtual bridge register was reordering the > > buses enumeration depending on link status, so there I should also > > properly rewrite the code in question. > > > > I can rewrite the code to read and write config properly using the > > slot moving the code into 'mt7621_pcie_enable_port' as follows: > > > > static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port) > > { > > struct mt7621_pcie *pcie = port->pcie; > > u32 slot = port->slot; > > u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT); > > u32 val; > > > > /* enable pcie interrupt */ > > val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR); > > val |= PCIE_PORT_INT_EN(slot); > > > > /* map 2G DDR region */ > > pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE, > > offset + RALINK_PCI_BAR0SETUP_ADDR); > > > > /* configure class code and revision ID */ > > pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID, > > offset + RALINK_PCI_CLASS); > > > > /* configure RC FTS number to 250 when it leaves L0s */ > > val = read_config(pcie, slot, PCIE_FTS_NUM); > > val &= ~PCIE_FTS_NUM_MASK; > > val |= PCIE_FTS_NUM_L0(0x50); > > write_config(pcie, slot, PCIE_FTS_NUM, val); > > } > > > > static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > { > > struct device *dev = pcie->dev; > > struct mt7621_pcie_port *port; > > int err; > > > > /* Setup MEMWIN and IOWIN */ > > pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > > list_for_each_entry(port, &pcie->ports, list) { > > if (port->enabled) { > > err = clk_prepare_enable(port->clk); > > if (err) { > > dev_err(dev, "enabling clk pcie%d\n", > > port->slot); > > return err; > > } > > > > mt7621_pcie_enable_port(port); > > dev_info(dev, "PCIE%d enabled\n", port->slot); > > } > > } > > > > return 0; > > } > > > > Or just delete the setting and the read and write config functions > > since they are not being used in any other place. My two boards work > > without this setting but I don't know about other boards. > > > > What do you think? > > I think that your above modification is fine and make code more > straightforward. Thanks for looking at it! Ok I will change this in this way, then. Thanks for your comments and help. Best regards, Sergio Paracuellos > > > Best regards, > > Sergio Paracuellos > > > > [0]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/tree/drivers/staging/mt7621-pci/pci-mt7621.c?h=v4.18#n663 > > [1]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/commit/drivers/staging/mt7621-pci?h=staging-testing&id=b99cc3a2b6b62cf994acac5cced03298d9908c9b
Hi Rob, On Fri, Jun 4, 2021 at 11:32 PM Sergio Paracuellos <sergio.paracuellos@gmail.com> wrote: > > Hi Rob, > > Thanks for the review. > > On Fri, Jun 4, 2021 at 9:35 PM Rob Herring <robh@kernel.org> wrote: > > > > On Sat, May 15, 2021 at 02:40:52PM +0200, Sergio Paracuellos wrote: > > > Add device tree binding documentation for PCIe in MT7621 SoCs. > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > --- > > > .../bindings/pci/mediatek,mt7621-pci.yaml | 149 ++++++++++++++++++ > > > 1 file changed, 149 insertions(+) > > > create mode 100644 Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > > > > > diff --git a/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > > new file mode 100644 > > > index 000000000000..7f5f9d583032 > > > --- /dev/null > > > +++ b/Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml > > > @@ -0,0 +1,149 @@ > > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > > > +%YAML 1.2 > > > +--- > > > +$id: http://devicetree.org/schemas/pci/mediatek,mt7621-pci.yaml# > > > +$schema: http://devicetree.org/meta-schemas/core.yaml# > > > + > > > +title: MediaTek MT7621 PCIe controller > > > + > > > +maintainers: > > > + - Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > + > > > +description: |+ > > > + MediaTek MT7621 PCIe subsys supports single Root complex (RC) > > > + with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link > > > + > > > +allOf: > > > + - $ref: /schemas/pci/pci-bus.yaml# > > > + > > > +properties: > > > + compatible: > > > + const: mediatek,mt7621-pci > > > + > > > + reg: > > > + items: > > > + - description: host-pci bridge registers > > > + - description: pcie port 0 RC control registers > > > + - description: pcie port 1 RC control registers > > > + - description: pcie port 2 RC control registers > > > + > > > + ranges: > > > + maxItems: 2 > > > + > > > + resets: > > > + items: > > > + - description: pcie port 0 reset. > > > + - description: pcie port 1 reset. > > > + - description: pcie port 2 reset. > > > + > > > + reset-names: > > > + items: > > > + - const: pcie0 > > > + - const: pcie1 > > > + - const: pcie2 > > > + > > > + clocks: > > > + items: > > > + - description: pcie port 0 clock. > > > + - description: pcie port 1 clock. > > > + - description: pcie port 2 clock. > > > + > > > + clock-names: > > > + items: > > > + - const: pcie0 > > > + - const: pcie1 > > > + - const: pcie2 > > > + > > > + phys: > > > + items: > > > + - description: Dual-ported phy for pcie port 0 and 1. > > > + - description: Phy for pcie port 2. > > > + > > > + phy-names: > > > + items: > > > + - const: pcie-phy0 > > > + - const: pcie-phy2 > > > > If you're going to keep the ports (and I think that's right because > > there's only a single PCI address space AFAICT), then I think you should > > move resets, clocks, and phys into each port node. > > > > So you'll need to define 'pcie@[0-2],0' node with those properties under > > it. > > Ok I will move these stuff to each port node. So each port node will > be similar to: > > pcie@0,0 { > reg = <0x0000 0 0 0 0>; > #address-cells = <3>; > #size-cells = <2>; > device_type = "pci"; > #interrupt-cells = <1>; > clocks = <&clkctrl 24>; > resets = <&rstctrl 24>; > phys = <&pcie0_phy 1>; > interrupt-map-mask = <0 0 0 0>; > interrupt-map = <0 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>; > ranges; > }; > > How can I be sure by schema that the clocks, reset and phy properties > are in each port node if I move them from the parent? By now each port > node is just validating because of ' $ref: > /schemas/pci/pci-bus.yaml#'. > > Thanks in advance for your time. > So I think, this should be enough: %YAML 1.2 --- $id: http://devicetree.org/schemas/pci/mediatek,mt7621-pci.yaml# $schema: http://devicetree.org/meta-schemas/core.yaml# title: MediaTek MT7621 PCIe controller maintainers: - Sergio Paracuellos <sergio.paracuellos@gmail.com> description: |+ MediaTek MT7621 PCIe subsys supports single Root complex (RC) with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link allOf: - $ref: /schemas/pci/pci-bus.yaml# properties: compatible: const: mediatek,mt7621-pci reg: items: - description: host-pci bridge registers - description: pcie port 0 RC control registers - description: pcie port 1 RC control registers - description: pcie port 2 RC control registers ranges: maxItems: 2 patternProperties: 'pcie@[0-2],0': type: object $ref: /schemas/pci/pci-bus.yaml# properties: resets: maxItems: 1 clocks: maxItems: 1 phys: maxItems: 1 required: - "#interrupt-cells" - interrupt-map-mask - interrupt-map - resets - clocks - phys - phy-names - ranges unevaluatedProperties: false required: - compatible - reg - ranges - "#interrupt-cells" - interrupt-map-mask - interrupt-map - reset-gpios unevaluatedProperties: false examples: - | #include <dt-bindings/gpio/gpio.h> #include <dt-bindings/interrupt-controller/mips-gic.h> pcie: pcie@1e140000 { compatible = "mediatek,mt7621-pci"; reg = <0x1e140000 0x100>, <0x1e142000 0x100>, <0x1e143000 0x100>, <0x1e144000 0x100>; #address-cells = <3>; #size-cells = <2>; pinctrl-names = "default"; pinctrl-0 = <&pcie_pins>; device_type = "pci"; ranges = <0x02000000 0 0x00000000 0x60000000 0 0x10000000>, /* pci memory */ <0x01000000 0 0x00000000 0x1e160000 0 0x00010000>; /* io space */ #interrupt-cells = <1>; interrupt-map-mask = <0xF800 0 0 0>; interrupt-map = <0x0000 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>, <0x0800 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>, <0x1000 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; reset-gpios = <&gpio 19 GPIO_ACTIVE_LOW>; pcie@0,0 { reg = <0x0000 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; device_type = "pci"; #interrupt-cells = <1>; interrupt-map-mask = <0 0 0 0>; interrupt-map = <0 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>; resets = <&rstctrl 24>; clocks = <&clkctrl 24>; phys = <&pcie0_phy 1>; phy-names = "pcie-phy0"; ranges; }; pcie@1,0 { reg = <0x0800 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; device_type = "pci"; #interrupt-cells = <1>; interrupt-map-mask = <0 0 0 0>; interrupt-map = <0 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>; resets = <&rstctrl 25>; clocks = <&clkctrl 25>; phys = <&pcie0_phy 1>; phy-names = "pcie-phy1"; ranges; }; pcie@2,0 { reg = <0x1000 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; device_type = "pci"; #interrupt-cells = <1>; interrupt-map-mask = <0 0 0 0>; interrupt-map = <0 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; resets = <&rstctrl 26>; clocks = <&clkctrl 26>; phys = <&pcie2_phy 0>; phy-names = "pcie-phy2"; ranges; }; }; ... I will send these bindings for v2 if you are ok with them. Thanks, Sergio Paracuellos > Best regards, > Sergio Paracuellos > > > > > > + > > > +required: > > > + - compatible > > > + - reg > > > + - ranges > > > + - "#interrupt-cells" > > > + - interrupt-map-mask > > > + - interrupt-map > > > + - resets > > > + - reset-names > > > + - clocks > > > + - clock-names > > > + - phys > > > + - phy-names > > > + - reset-gpios > > > + > > > +unevaluatedProperties: false > > > + > > > +examples: > > > + - | > > > + #include <dt-bindings/gpio/gpio.h> > > > + #include <dt-bindings/interrupt-controller/mips-gic.h> > > > + > > > + pcie: pcie@1e140000 { > > > + compatible = "mediatek,mt7621-pci"; > > > + reg = <0x1e140000 0x100>, > > > + <0x1e142000 0x100>, > > > + <0x1e143000 0x100>, > > > + <0x1e144000 0x100>; > > > + > > > + #address-cells = <3>; > > > + #size-cells = <2>; > > > + pinctrl-names = "default"; > > > + pinctrl-0 = <&pcie_pins>; > > > + device_type = "pci"; > > > + ranges = <0x02000000 0 0x00000000 0x60000000 0 0x10000000>, /* pci memory */ > > > + <0x01000000 0 0x00000000 0x1e160000 0 0x00010000>; /* io space */ > > > + #interrupt-cells = <1>; > > > + interrupt-map-mask = <0xF800 0 0 0>; > > > + interrupt-map = <0x0000 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>, > > > + <0x0800 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>, > > > + <0x1000 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > > > + resets = <&rstctrl 24>, <&rstctrl 25>, <&rstctrl 26>; > > > + reset-names = "pcie0", "pcie1", "pcie2"; > > > + clocks = <&clkctrl 24>, <&clkctrl 25>, <&clkctrl 26>; > > > + clock-names = "pcie0", "pcie1", "pcie2"; > > > + phys = <&pcie0_phy 1>, <&pcie2_phy 0>; > > > + phy-names = "pcie-phy0", "pcie-phy2"; > > > + reset-gpios = <&gpio 19 GPIO_ACTIVE_LOW>; > > > + > > > + pcie@0,0 { > > > + reg = <0x0000 0 0 0 0>; > > > + #address-cells = <3>; > > > + #size-cells = <2>; > > > + device_type = "pci"; > > > + #interrupt-cells = <1>; > > > + interrupt-map-mask = <0 0 0 0>; > > > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 4 IRQ_TYPE_LEVEL_HIGH>; > > > + ranges; > > > + }; > > > + > > > + pcie@1,0 { > > > + reg = <0x0800 0 0 0 0>; > > > + #address-cells = <3>; > > > + #size-cells = <2>; > > > + device_type = "pci"; > > > + #interrupt-cells = <1>; > > > + interrupt-map-mask = <0 0 0 0>; > > > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 24 IRQ_TYPE_LEVEL_HIGH>; > > > + ranges; > > > + }; > > > + > > > + pcie@2,0 { > > > + reg = <0x1000 0 0 0 0>; > > > + #address-cells = <3>; > > > + #size-cells = <2>; > > > + device_type = "pci"; > > > + #interrupt-cells = <1>; > > > + interrupt-map-mask = <0 0 0 0>; > > > + interrupt-map = <0 0 0 0 &gic GIC_SHARED 25 IRQ_TYPE_LEVEL_HIGH>; > > > + ranges; > > > + }; > > > + }; > > > +... > > > -- > > > 2.25.1
Hi Rob, On Sat, Jun 5, 2021 at 12:25 AM Sergio Paracuellos <sergio.paracuellos@gmail.com> wrote: > > Hi Rob, > > Thanks for the review. > > On Fri, Jun 4, 2021 at 9:30 PM Rob Herring <robh@kernel.org> wrote: > > > > On Sat, May 15, 2021 at 02:40:53PM +0200, Sergio Paracuellos wrote: > > > This patch adds a driver for the PCIe controller of MT7621 SoC. > > > > > > Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com> > > > --- > > > arch/mips/pci/Makefile | 1 + > > > arch/mips/pci/pci-mt7621.c | 624 +++++++++++++++++++++++++++++++++++++ > > > arch/mips/ralink/Kconfig | 9 +- > > > 3 files changed, 633 insertions(+), 1 deletion(-) > > > create mode 100644 arch/mips/pci/pci-mt7621.c > > > > > > diff --git a/arch/mips/pci/Makefile b/arch/mips/pci/Makefile > > > index f3eecc065e5c..178c550739c4 100644 > > > --- a/arch/mips/pci/Makefile > > > +++ b/arch/mips/pci/Makefile > > > @@ -24,6 +24,7 @@ obj-$(CONFIG_PCI_AR2315) += pci-ar2315.o > > > obj-$(CONFIG_SOC_AR71XX) += pci-ar71xx.o > > > obj-$(CONFIG_PCI_AR724X) += pci-ar724x.o > > > obj-$(CONFIG_PCI_XTALK_BRIDGE) += pci-xtalk-bridge.o > > > +obj-$(CONFIG_PCI_MT7621) += pci-mt7621.o > > > # > > > # These are still pretty much in the old state, watch, go blind. > > > # > > > diff --git a/arch/mips/pci/pci-mt7621.c b/arch/mips/pci/pci-mt7621.c > > > new file mode 100644 > > > index 000000000000..fe1945819d25 > > > --- /dev/null > > > +++ b/arch/mips/pci/pci-mt7621.c > > > @@ -0,0 +1,624 @@ > > > +// SPDX-License-Identifier: GPL-2.0+ > > > +/* > > > + * BRIEF MODULE DESCRIPTION > > > + * PCI init for Ralink RT2880 solution > > > + * > > > + * Copyright 2007 Ralink Inc. (bruce_chang@ralinktech.com.tw) > > > + * > > > + * May 2007 Bruce Chang > > > + * Initial Release > > > + * > > > + * May 2009 Bruce Chang > > > + * support RT2880/RT3883 PCIe > > > + * > > > + * May 2011 Bruce Chang > > > + * support RT6855/MT7620 PCIe > > > + */ > > > + > > > +#include <linux/bitops.h> > > > +#include <linux/clk.h> > > > +#include <linux/delay.h> > > > +#include <linux/gpio/consumer.h> > > > +#include <linux/module.h> > > > +#include <linux/of.h> > > > +#include <linux/of_address.h> > > > +#include <linux/of_pci.h> > > > +#include <linux/of_platform.h> > > > +#include <linux/pci.h> > > > +#include <linux/phy/phy.h> > > > +#include <linux/platform_device.h> > > > +#include <linux/reset.h> > > > +#include <linux/sys_soc.h> > > > + > > > +/* MediaTek specific configuration registers */ > > > +#define PCIE_FTS_NUM 0x70c > > > +#define PCIE_FTS_NUM_MASK GENMASK(15, 8) > > > +#define PCIE_FTS_NUM_L0(x) (((x) & 0xff) << 8) > > > + > > > +/* Host-PCI bridge registers */ > > > +#define RALINK_PCI_PCICFG_ADDR 0x0000 > > > +#define RALINK_PCI_PCIMSK_ADDR 0x000C > > > +#define RALINK_PCI_CONFIG_ADDR 0x0020 > > > +#define RALINK_PCI_CONFIG_DATA 0x0024 > > > +#define RALINK_PCI_MEMBASE 0x0028 > > > +#define RALINK_PCI_IOBASE 0x002C > > > + > > > +/* PCIe RC control registers */ > > > +#define MT7621_PCIE_OFFSET 0x2000 > > > +#define MT7621_NEXT_PORT 0x1000 > > > + > > > +#define RALINK_PCI_BAR0SETUP_ADDR 0x0010 > > > > Standard BAR0 register? > > Ok, I will remove this and use 'PCI_BASE_ADDRESS_0' instead. > > > > > > +#define RALINK_PCI_ID 0x0030 > > > +#define RALINK_PCI_CLASS 0x0034 > > > +#define RALINK_PCI_SUBID 0x0038 > > > +#define RALINK_PCI_STATUS 0x0050 > > > + > > > +/* Some definition values */ > > > +#define PCIE_REVISION_ID BIT(0) > > > +#define PCIE_CLASS_CODE (0x60400 << 8) > > > +#define PCIE_BAR_MAP_MAX GENMASK(30, 16) > > > +#define PCIE_BAR_ENABLE BIT(0) > > > +#define PCIE_PORT_INT_EN(x) BIT(20 + (x)) > > > +#define PCIE_PORT_LINKUP BIT(0) > > > + > > > +#define PERST_DELAY_MS 100 > > > + > > > +/** > > > + * struct mt7621_pcie_port - PCIe port information > > > + * @base: I/O mapped register base > > > + * @list: port list > > > + * @pcie: pointer to PCIe host info > > > + * @clk: pointer to the port clock gate > > > + * @phy: pointer to PHY control block > > > + * @pcie_rst: pointer to port reset control > > > + * @gpio_rst: gpio reset > > > + * @slot: port slot > > > + * @enabled: indicates if port is enabled > > > + */ > > > +struct mt7621_pcie_port { > > > + void __iomem *base; > > > + struct list_head list; > > > + struct mt7621_pcie *pcie; > > > + struct clk *clk; > > > + struct phy *phy; > > > + struct reset_control *pcie_rst; > > > + struct gpio_desc *gpio_rst; > > > + u32 slot; > > > + bool enabled; > > > +}; > > > + > > > +/** > > > + * struct mt7621_pcie - PCIe host information > > > + * @base: IO Mapped Register Base > > > + * @io: IO resource > > > + * @mem: pointer to non-prefetchable memory resource > > > + * @dev: Pointer to PCIe device > > > + * @io_map_base: virtual memory base address for io > > > + * @ports: pointer to PCIe port information > > > + * @resets_inverted: depends on chip revision > > > + * reset lines are inverted. > > > + */ > > > +struct mt7621_pcie { > > > + void __iomem *base; > > > + struct device *dev; > > > > > + struct resource io; > > > + struct resource *mem; > > > + unsigned long io_map_base; > > > > These are all stored in the host bridge struct, no need for you to store > > them. > > IO resources must be requested and mapped manually (see my explanation > below) and I use mem resource to also setup mips iocu regions, that is > why I am storing it also here. > > > > > > + struct list_head ports; > > > > A list is kind of an overkill for 3 entries and you know how many ports. > > Just embed an array of struct mt7621_pcie_port. Then you only need 1 > > alloc. > > Since ports are at most three but can be one or two also depending on > the board I ended up using a list instead of a fixed array of three > ports. This list is dynamically updated depending on link status. If > some of the ports are not present in the board nodes initially stored > after device tree parsing are deleted for the list. If it is not a big > problem I prefer to maintain this list as it is. > > > > > > + bool resets_inverted; > > > +}; > > > + > > > +static inline u32 pcie_read(struct mt7621_pcie *pcie, u32 reg) > > > +{ > > > + return readl(pcie->base + reg); > > > > Can use _relaxed variants here and through out. > > Ok will change into _relaxed variants if is preferred. > > > > > > +} > > > + > > > +static inline void pcie_write(struct mt7621_pcie *pcie, u32 val, u32 reg) > > > +{ > > > + writel(val, pcie->base + reg); > > > +} > > > + > > > +static inline void pcie_rmw(struct mt7621_pcie *pcie, u32 reg, u32 clr, u32 set) > > > +{ > > > + u32 val = readl(pcie->base + reg); > > > + > > > + val &= ~clr; > > > + val |= set; > > > + writel(val, pcie->base + reg); > > > +} > > > + > > > +static inline u32 pcie_port_read(struct mt7621_pcie_port *port, u32 reg) > > > +{ > > > + return readl(port->base + reg); > > > +} > > > + > > > +static inline void pcie_port_write(struct mt7621_pcie_port *port, > > > + u32 val, u32 reg) > > > +{ > > > + writel(val, port->base + reg); > > > +} > > > + > > > +static inline u32 mt7621_pci_get_cfgaddr(unsigned int bus, unsigned int slot, > > > + unsigned int func, unsigned int where) > > > +{ > > > + return (((where & 0xF00) >> 8) << 24) | (bus << 16) | (slot << 11) | > > > + (func << 8) | (where & 0xfc) | 0x80000000; > > > +} > > > + > > > +static void __iomem *mt7621_pcie_map_bus(struct pci_bus *bus, > > > + unsigned int devfn, int where) > > > +{ > > > + struct mt7621_pcie *pcie = bus->sysdata; > > > + u32 address = mt7621_pci_get_cfgaddr(bus->number, PCI_SLOT(devfn), > > > + PCI_FUNC(devfn), where); > > > + > > > + writel(address, pcie->base + RALINK_PCI_CONFIG_ADDR); > > > + > > > + return pcie->base + RALINK_PCI_CONFIG_DATA + (where & 3); > > > +} > > > + > > > +struct pci_ops mt7621_pci_ops = { > > > + .map_bus = mt7621_pcie_map_bus, > > > + .read = pci_generic_config_read, > > > + .write = pci_generic_config_write, > > > +}; > > > + > > > +static u32 read_config(struct mt7621_pcie *pcie, unsigned int dev, u32 reg) > > > +{ > > > + u32 address = mt7621_pci_get_cfgaddr(0, dev, 0, reg); > > > + > > > + pcie_write(pcie, address, RALINK_PCI_CONFIG_ADDR); > > > + return pcie_read(pcie, RALINK_PCI_CONFIG_DATA); > > > +} > > > + > > > +static void write_config(struct mt7621_pcie *pcie, unsigned int dev, > > > + u32 reg, u32 val) > > > +{ > > > + u32 address = mt7621_pci_get_cfgaddr(0, dev, 0, reg); > > > + > > > + pcie_write(pcie, address, RALINK_PCI_CONFIG_ADDR); > > > + pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA); > > > +} > > > + > > > +static inline void mt7621_rst_gpio_pcie_assert(struct mt7621_pcie_port *port) > > > +{ > > > + if (port->gpio_rst) > > > > Don't need the if. gpiod_set_value should work with NULL. > > Gpio resets can be optional. Some boards use one gpio reset pin for > each port but others only one for all of them. So if no reset is not > requested for a port I don't want to do anything. Hence, the check. > > > > > > + gpiod_set_value(port->gpio_rst, 1); > > > +} > > > + > > > +static inline void mt7621_rst_gpio_pcie_deassert(struct mt7621_pcie_port *port) > > > +{ > > > + if (port->gpio_rst) > > > + gpiod_set_value(port->gpio_rst, 0); > > > +} > > > + > > > +static inline bool mt7621_pcie_port_is_linkup(struct mt7621_pcie_port *port) > > > +{ > > > + return (pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) != 0; > > > +} > > > + > > > +static inline void mt7621_control_assert(struct mt7621_pcie_port *port) > > > +{ > > > + struct mt7621_pcie *pcie = port->pcie; > > > + > > > + if (pcie->resets_inverted) > > > + reset_control_assert(port->pcie_rst); > > > + else > > > + reset_control_deassert(port->pcie_rst); > > > +} > > > + > > > +static inline void mt7621_control_deassert(struct mt7621_pcie_port *port) > > > +{ > > > + struct mt7621_pcie *pcie = port->pcie; > > > + > > > + if (pcie->resets_inverted) > > > + reset_control_deassert(port->pcie_rst); > > > + else > > > + reset_control_assert(port->pcie_rst); > > > +} > > > + > > > +static void setup_cm_memory_region(struct mt7621_pcie *pcie) > > > +{ > > > + struct resource *mem_resource = pcie->mem; > > > + struct device *dev = pcie->dev; > > > + resource_size_t mask; > > > + > > > + if (mips_cps_numiocu(0)) { > > > + /* > > > + * FIXME: hardware doesn't accept mask values with 1s after > > > + * 0s (e.g. 0xffef), so it would be great to warn if that's > > > + * about to happen > > > + */ > > > + mask = ~(mem_resource->end - mem_resource->start); > > > + > > > + write_gcr_reg1_base(mem_resource->start); > > > + write_gcr_reg1_mask(mask | CM_GCR_REGn_MASK_CMTGT_IOCU0); > > > + dev_info(dev, "PCI coherence region base: 0x%08llx, mask/settings: 0x%08llx\n", > > > + (unsigned long long)read_gcr_reg1_base(), > > > + (unsigned long long)read_gcr_reg1_mask()); > > > + } > > > +} > > > + > > > +static int mt7621_pci_parse_request_of_pci_ranges(struct pci_host_bridge *host) > > > +{ > > > + struct mt7621_pcie *pcie = pci_host_bridge_priv(host); > > > + struct device *dev = pcie->dev; > > > + struct device_node *node = dev->of_node; > > > + struct of_pci_range_parser parser; > > > + struct resource_entry *entry; > > > + struct of_pci_range range; > > > + LIST_HEAD(res); > > > + > > > + if (of_pci_range_parser_init(&parser, node)) { > > > + dev_err(dev, "missing \"ranges\" property\n"); > > > + return -EINVAL; > > > + } > > > + > > > + /* > > > + * IO_SPACE_LIMIT for MIPS is 0xffff but this platform uses IO at > > > + * upper address 0x001e160000. of_pci_range_to_resource does not work > > > > I think that's normal... > > No, pci_address_to_pio will fail and io resources are not properly > assigned at all. > > > > > > + * well for MIPS platforms that don't define PCI_IOBASE, so set the IO > > > + * resource manually instead. > > > > Can't this be fixed? > > This is the way the current PCI architecture is in mips... Maybe > Thomas has a strong opinion on this. > > > > > > + */ > > > + for_each_of_pci_range(&parser, &range) { > > > + switch (range.flags & IORESOURCE_TYPE_BITS) { > > > > The core code already parses ranges for you. Try not to do it again. > > (Use the resource instead) > > See below... > > > > > > + case IORESOURCE_IO: > > > + pcie->io_map_base = > > > + (unsigned long)ioremap(range.cpu_addr, > > > + range.size); > > > + pcie->io.name = node->full_name; > > > + pcie->io.flags = range.flags; > > > + pcie->io.start = range.cpu_addr; > > > + pcie->io.end = range.cpu_addr + range.size - 1; > > > + pcie->io.parent = pcie->io.child = pcie->io.sibling = NULL; > > > + set_io_port_base(pcie->io_map_base); > > > + break; > > > + } > > > + } > > > + > > > + entry = resource_list_first_type(&host->windows, IORESOURCE_MEM); > > > + if (!entry) { > > > + dev_err(dev, "Cannot get memory resource"); > > > + return -EINVAL; > > > + } > > > + > > > + pcie->mem = entry->res; > > > + pci_add_resource(&res, &pcie->io); > > > + pci_add_resource(&res, entry->res); > > > + list_splice_init(&res, &host->windows); > > > > This should already be done for you. > > Most MIPS platforms do not define PCI_IOBASE, nor implement > pci_address_to_pio(). Moreover, IO_SPACE_LIMIT is 0xffff for most MIPS > platforms. of_pci_range_to_resource passes the _start address_ of the > IO range into pci_address_to_pio, which then checks it against > IO_SPACE_LIMIT and fails. So I have to do this manually or nothing > will work properly... > > > > > > + > > > + return 0; > > > +} > > > + > > > +static int mt7621_pcie_parse_port(struct mt7621_pcie *pcie, > > > + int slot) > > > +{ > > > + struct mt7621_pcie_port *port; > > > + struct device *dev = pcie->dev; > > > + struct platform_device *pdev = to_platform_device(dev); > > > + char name[10]; > > > + > > > + port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL); > > > + if (!port) > > > + return -ENOMEM; > > > + > > > + port->base = devm_platform_ioremap_resource(pdev, slot + 1); > > > + if (IS_ERR(port->base)) > > > + return PTR_ERR(port->base); > > > + > > > + snprintf(name, sizeof(name), "pcie%d", slot); > > > + port->clk = devm_clk_get(dev, name); > > > + if (IS_ERR(port->clk)) { > > > + dev_err(dev, "failed to get pcie%d clock\n", slot); > > > + return PTR_ERR(port->clk); > > > + } > > > + > > > + snprintf(name, sizeof(name), "pcie%d", slot); > > > + port->pcie_rst = devm_reset_control_get_exclusive(dev, name); > > > + if (PTR_ERR(port->pcie_rst) == -EPROBE_DEFER) { > > > + dev_err(dev, "failed to get pcie%d reset control\n", slot); > > > + return PTR_ERR(port->pcie_rst); > > > + } > > > + > > > + snprintf(name, sizeof(name), "pcie-phy%d", slot); > > > + port->phy = devm_phy_get(dev, name); > > > + if (IS_ERR(port->phy) && slot != 1) > > > + return PTR_ERR(port->phy); > > > + > > > + port->gpio_rst = devm_gpiod_get_index_optional(dev, "reset", slot, > > > + GPIOD_OUT_LOW); > > > + if (IS_ERR(port->gpio_rst)) { > > > + dev_err(dev, "Failed to get GPIO for PCIe%d\n", slot); > > > + return PTR_ERR(port->gpio_rst); > > > + } > > > + > > > + port->slot = slot; > > > + port->pcie = pcie; > > > + > > > + INIT_LIST_HEAD(&port->list); > > > + list_add_tail(&port->list, &pcie->ports); > > > + > > > + return 0; > > > +} > > > + > > > +static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie) > > > +{ > > > + struct device *dev = pcie->dev; > > > + struct platform_device *pdev = to_platform_device(dev); > > > + struct device_node *node = dev->of_node, *child; > > > + int err; > > > + > > > + pcie->base = devm_platform_ioremap_resource(pdev, 0); > > > + if (IS_ERR(pcie->base)) > > > + return PTR_ERR(pcie->base); > > > + > > > + for_each_available_child_of_node(node, child) { > > > + int slot; > > > + > > > + err = of_pci_get_devfn(child); > > > + if (err < 0) { > > > + of_node_put(child); > > > + dev_err(dev, "failed to parse devfn: %d\n", err); > > > + return err; > > > + } > > > + > > > + slot = PCI_SLOT(err); > > > + > > > + err = mt7621_pcie_parse_port(pcie, slot); > > > + if (err) { > > > + of_node_put(child); > > > + return err; > > > + } > > > + } > > > + > > > + return 0; > > > +} > > > + > > > +static int mt7621_pcie_init_port(struct mt7621_pcie_port *port) > > > +{ > > > + struct mt7621_pcie *pcie = port->pcie; > > > + struct device *dev = pcie->dev; > > > + u32 slot = port->slot; > > > + int err; > > > + > > > + err = phy_init(port->phy); > > > + if (err) { > > > + dev_err(dev, "failed to initialize port%d phy\n", slot); > > > + return err; > > > + } > > > + > > > + err = phy_power_on(port->phy); > > > + if (err) { > > > + dev_err(dev, "failed to power on port%d phy\n", slot); > > > + phy_exit(port->phy); > > > + return err; > > > + } > > > + > > > + port->enabled = true; > > > + > > > + return 0; > > > +} > > > + > > > +static void mt7621_pcie_reset_assert(struct mt7621_pcie *pcie) > > > +{ > > > + struct mt7621_pcie_port *port; > > > + > > > + list_for_each_entry(port, &pcie->ports, list) { > > > + /* PCIe RC reset assert */ > > > + mt7621_control_assert(port); > > > + > > > + /* PCIe EP reset assert */ > > > + mt7621_rst_gpio_pcie_assert(port); > > > + } > > > + > > > + msleep(PERST_DELAY_MS); > > > +} > > > + > > > +static void mt7621_pcie_reset_rc_deassert(struct mt7621_pcie *pcie) > > > +{ > > > + struct mt7621_pcie_port *port; > > > + > > > + list_for_each_entry(port, &pcie->ports, list) > > > + mt7621_control_deassert(port); > > > +} > > > + > > > +static void mt7621_pcie_reset_ep_deassert(struct mt7621_pcie *pcie) > > > +{ > > > + struct mt7621_pcie_port *port; > > > + > > > + list_for_each_entry(port, &pcie->ports, list) > > > + mt7621_rst_gpio_pcie_deassert(port); > > > + > > > + msleep(PERST_DELAY_MS); > > > +} > > > + > > > +static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie) > > > +{ > > > + struct device *dev = pcie->dev; > > > + struct mt7621_pcie_port *port, *tmp; > > > + int err; > > > + > > > + mt7621_pcie_reset_assert(pcie); > > > + mt7621_pcie_reset_rc_deassert(pcie); > > > + > > > + list_for_each_entry_safe(port, tmp, &pcie->ports, list) { > > > + u32 slot = port->slot; > > > + > > > + if (slot == 1) { > > > + port->enabled = true; > > > + continue; > > > + } > > > + > > > + err = mt7621_pcie_init_port(port); > > > + if (err) { > > > + dev_err(dev, "Initiating port %d failed\n", slot); > > > + list_del(&port->list); > > > + } > > > + } > > > + > > > + mt7621_pcie_reset_ep_deassert(pcie); > > > + > > > + tmp = NULL; > > > + list_for_each_entry(port, &pcie->ports, list) { > > > + u32 slot = port->slot; > > > + > > > + if (!mt7621_pcie_port_is_linkup(port)) { > > > + dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", > > > + slot); > > > + mt7621_control_assert(port); > > > + clk_disable_unprepare(port->clk); > > > + port->enabled = false; > > > + > > > + if (slot == 0) { > > > + tmp = port; > > > + continue; > > > + } > > > + > > > + if (slot == 1 && tmp && !tmp->enabled) > > > + phy_power_off(tmp->phy) > > > + } > > > + } > > > +} > > > + > > > +static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port) > > > +{ > > > + struct mt7621_pcie *pcie = port->pcie; > > > + u32 slot = port->slot; > > > + u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT); > > > > I don't see how this works unless the ports happen to be at the same VA > > offset. The writes below are to the 'common' registers which are 0x100 > > bytes long based on the DT example, but the offset lines up with the > > port offsets. > > RC registers for port 0 start at physical address 0x1e142000, for port > 1 0x1e143000 and for port 2 0x1e144000. We are using > 'pcie_write' which internally uses pcie->base which is 0x1e140000, but > all the previous ones have been already requested and mapped for each > port so we calculate the offset from this 0x1e140000 and write there. > > > > > > + u32 val; > > > + > > > + /* enable pcie interrupt */ > > > + val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR); > > > + val |= PCIE_PORT_INT_EN(slot); > > > + pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR); > > > + > > > + /* map 2G DDR region */ > > > + pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE, > > > + offset + RALINK_PCI_BAR0SETUP_ADDR); > > > + > > > + /* configure class code and revision ID */ > > > + pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID, > > > + offset + RALINK_PCI_CLASS); > > > +} > > > + > > > +static int mt7621_pcie_enable_ports(struct mt7621_pcie *pcie) > > > +{ > > > + struct device *dev = pcie->dev; > > > + struct mt7621_pcie_port *port; > > > + u8 num_slots_enabled = 0; > > > + u32 slot; > > > + u32 val; > > > + int err; > > > + > > > + /* Setup MEMWIN and IOWIN */ > > > + pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE); > > > + pcie_write(pcie, pcie->io.start, RALINK_PCI_IOBASE); > > > + > > > + list_for_each_entry(port, &pcie->ports, list) { > > > + if (port->enabled) { > > > + err = clk_prepare_enable(port->clk); > > > + if (err) { > > > + dev_err(dev, "enabling clk pcie%d\n", slot); > > > + return err; > > > + } > > > + > > > + mt7621_pcie_enable_port(port); > > > + dev_info(dev, "PCIE%d enabled\n", port->slot); > > > + num_slots_enabled++; > > > + } > > > + } > > > + > > > + for (slot = 0; slot < num_slots_enabled; slot++) { > > > + val = read_config(pcie, slot, PCI_COMMAND); > > > + val |= PCI_COMMAND_MASTER; > > > + write_config(pcie, slot, PCI_COMMAND, val); > > > + /* configure RC FTS number to 250 when it leaves L0s */ > > > + val = read_config(pcie, slot, PCIE_FTS_NUM); > > > + val &= ~PCIE_FTS_NUM_MASK; > > > + val |= PCIE_FTS_NUM_L0(0x50); > > > + write_config(pcie, slot, PCIE_FTS_NUM, val); > > > + } > > > + > > > + return 0; > > > +} > > > + > > > +static int mt7621_pcie_register_host(struct pci_host_bridge *host) > > > +{ > > > + struct mt7621_pcie *pcie = pci_host_bridge_priv(host); > > > + > > > + host->ops = &mt7621_pci_ops; > > > + host->sysdata = pcie; > > > + return pci_host_probe(host); > > > +} > > > + > > > +static const struct soc_device_attribute mt7621_pci_quirks_match[] = { > > > + { .soc_id = "mt7621", .revision = "E2" } > > > +}; > > > + > > > +static int mt7621_pci_probe(struct platform_device *pdev) > > > +{ > > > + struct device *dev = &pdev->dev; > > > + const struct soc_device_attribute *attr; > > > + struct mt7621_pcie *pcie; > > > + struct pci_host_bridge *bridge; > > > + int err; > > > + > > > + if (!dev->of_node) > > > + return -ENODEV; > > > + > > > + bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie)); > > > + if (!bridge) > > > + return -ENOMEM; > > > + > > > + pcie = pci_host_bridge_priv(bridge); > > > + pcie->dev = dev; > > > + platform_set_drvdata(pdev, pcie); > > > + INIT_LIST_HEAD(&pcie->ports); > > > + > > > + attr = soc_device_match(mt7621_pci_quirks_match); > > > + if (attr) > > > + pcie->resets_inverted = true; > > > + > > > + err = mt7621_pcie_parse_dt(pcie); > > > + if (err) { > > > + dev_err(dev, "Parsing DT failed\n"); > > > + return err; > > > + } > > > + > > > + err = mt7621_pci_parse_request_of_pci_ranges(bridge); > > > + if (err) { > > > + dev_err(dev, "Error requesting pci resources from ranges"); > > > + return err; > > > + } > > > + > > > + /* set resources limits */ > > > + ioport_resource.start = pcie->io.start; > > > + ioport_resource.end = pcie->io.end; > > > + > > > + mt7621_pcie_init_ports(pcie); > > > + > > > + err = mt7621_pcie_enable_ports(pcie); > > > > Really need 2 functions here? > > mt7621_pcie_init_ports is in charge of power on the phy and checks for > link status to see which ports are in use and mt7621_pcie_enable_ports > just enable all the stuff to be ready to be used. > > > > > > + if (err) { > > > + dev_err(dev, "Error enabling pcie ports\n"); > > > + return err; > > > + } > > > + > > > + setup_cm_memory_region(pcie); > > > + > > > + return mt7621_pcie_register_host(bridge); > > > +} > > > + > > > +static const struct of_device_id mt7621_pci_ids[] = { > > > + { .compatible = "mediatek,mt7621-pci" }, > > > + {}, > > > +}; > > > +MODULE_DEVICE_TABLE(of, mt7621_pci_ids); > > > + > > > +static struct platform_driver mt7621_pci_driver = { > > > + .probe = mt7621_pci_probe, > > > + .driver = { > > > + .name = "mt7621-pci", > > > + .of_match_table = of_match_ptr(mt7621_pci_ids), > > > + }, > > > +}; > > > +builtin_platform_driver(mt7621_pci_driver); > > > diff --git a/arch/mips/ralink/Kconfig b/arch/mips/ralink/Kconfig > > > index ec4daa63c5e3..50e5a54f7d9e 100644 > > > --- a/arch/mips/ralink/Kconfig > > > +++ b/arch/mips/ralink/Kconfig > > > @@ -56,7 +56,7 @@ choice > > > select MIPS_GIC > > > select COMMON_CLK > > > select CLKSRC_MIPS_GIC > > > - select HAVE_PCI if PCI_MT7621 > > > + select HAVE_PCI > > > select SOC_BUS > > > endchoice > > > > > > @@ -101,4 +101,11 @@ choice > > > > > > endchoice > > > > > > +config PCI_MT7621 > > > + bool "MediaTek MT7621 PCI Controller" > > > + depends on RALINK && SOC_MT7621 > > > > Ideally this should also have (|| COMPILE_TEST), but looks like there > > are a few MIPS dependencies in here. So maybe (|| (MIPS && > > COMPILE_TEST)? If it's not built by allmodconfig or defconfig, it's hard > > for people to compile test it. > > Ok, I will add (|| (MIPS && COMPILE_TEST) here. > > Best regards, > Sergio Paracuellos > > > > > > > > + select PCI_DRIVERS_GENERIC > > > + help > > > + This selects a driver for the MediaTek MT7621 PCI Controller. > > > + > > > endif > > > -- > > > 2.25.1 I have already sent to staging this patchset[0] regarding your comments before moving this driver to 'drivers/pci/controller' and sending v2: [0]: https://lore.kernel.org/linux-staging/20210606161358.7114-1-sergio.paracuellos@gmail.com/T/#t Thanks, Sergio Paracuellos
MediaTek MT7621 PCIe subsys supports single Root complex (RC) with 3 Root Ports. Each Root Ports supports a Gen1 1-lane Link. Topology is as follows: MT7621 PCIe HOST Topology .-------. | | | CPU | | | '-------' | | | v .------------------. .-----------| HOST/PCI Bridge |------------. | '------------------' | Type1 BUS0 | | | Access v v v On Bus0 .-------------. .-------------. .-------------. | VIRTUAL P2P | | VIRTUAL P2P | | VIRTUAL P2P | | BUS0 | | BUS0 | | BUS0 | | DEV0 | | DEV1 | | DEV2 | '-------------' '-------------' '-------------' Type0 | Type0 | Type0 | Access BUS1 | Access BUS2| Access BUS3| On Bus1 v On Bus2 v On Bus3 v .----------. .----------. .----------. | Device 0 | | Device 0 | | Device 0 | | Func 0 | | Func 0 | | Func 0 | '----------' '----------' '----------' This driver has been very long time in staging and I have been cleaning it from its first versions where there was code kaos and PCI_LEGACY support. Original code came probably from openWRT based on mediatek's SDK code. There is no documentation at all about the mt7621 PCI subsystem. I have been cleaning it targeting mt7621 SoC which is the one I use in my GNUBee PC1 board and HiLink HLK-MT7621A evaluation board. Now I think is clean enough to be moved into 'arch/mips/pci'. This driver also uses already mainlined pci phy driver located in 'drivers/phy/ralink/phy-mt7621-pci.c'. There are two instances of the phy being the first one dual ported for pci0 and pci1, and the second one not dual ported dedicated to pci2. Because of writing twice some phy registers of the dual-ported one sometimes become in not confident boot cycles we have to take care of this when device link is checked here in controller driver. We power on the dual ported-phy if there is something connected in pcie0 or pcie1. In the same manner we have to properly disable it only if nothing is connected in of both pcie0 and pci1 slots. Another thing that must be mentioned is that this driver uses IO in physical address 0x001e160000. IO_SPACE_LIMIT for MIPS is 0xffff so some generic PCI functions (like of_pci_range_to_resource) won't work and the resource ranges part for IO is set manually. I had already sent binding documentation to be reviewed but I am include also here with the driver itself and this cover letter to make easy review process. Best regards, Sergio Paracuellos Sergio Paracuellos (4): dt-bindings: mt7621-pci: PCIe binding documentation for MT7621 SoCs MIPS: pci: Add driver for MT7621 PCIe controller staging: mt7621-pci: remove driver from staging MAINTAINERS: add myself as maintainer of the MT7621 PCI controller driver .../bindings/pci/mediatek,mt7621-pci.yaml | 149 ++++++++++++++++++ MAINTAINERS | 6 + arch/mips/pci/Makefile | 1 + .../mt7621-pci => arch/mips/pci}/pci-mt7621.c | 0 arch/mips/ralink/Kconfig | 9 +- drivers/staging/Kconfig | 2 - drivers/staging/Makefile | 1 - drivers/staging/mt7621-pci/Kconfig | 8 - drivers/staging/mt7621-pci/Makefile | 2 - drivers/staging/mt7621-pci/TODO | 4 - .../mt7621-pci/mediatek,mt7621-pci.txt | 104 ------------ 11 files changed, 164 insertions(+), 122 deletions(-) create mode 100644 Documentation/devicetree/bindings/pci/mediatek,mt7621-pci.yaml rename {drivers/staging/mt7621-pci => arch/mips/pci}/pci-mt7621.c (100%) delete mode 100644 drivers/staging/mt7621-pci/Kconfig delete mode 100644 drivers/staging/mt7621-pci/Makefile delete mode 100644 drivers/staging/mt7621-pci/TODO delete mode 100644 drivers/staging/mt7621-pci/mediatek,mt7621-pci.txt