[v2] treewide: Fix instantiation of devices in DT overlays

When loading a DT overlay that creates a device, the device is not
instantiated, unless the DT overlay is unloaded and reloaded again.

Saravana explains:
  Basically for all overlays (I hope the function is only used for
  overlays) we assume all nodes are NOT devices until they actually
  get added as a device.

Based on a patch by Saravana Kannan, which covered only platform and spi
devices.

Fixes: 4a032827daa89350 ("of: property: Simplify of_link_to_phandle()")
Link: https://lore.kernel.org/r/CAGETcx_+rhHvaC_HJXGrr5_WAd2+k5f=rWYnkCZ6z5bGX-wj4w@mail.gmail.com
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Acked-by: Mark Brown <broonie@kernel.org>
---
v2:
  - Add Acked-by,
  - Drop RFC.
---
 drivers/bus/imx-weim.c    | 1 +
 drivers/i2c/i2c-core-of.c | 1 +
 drivers/of/dynamic.c      | 1 +
 drivers/of/platform.c     | 1 +
 drivers/spi/spi.c         | 1 +
 5 files changed, 5 insertions(+)

On 3/24/23 04:30, Geert Uytterhoeven wrote:
> When loading a DT overlay that creates a device, the device is not
> instantiated, unless the DT overlay is unloaded and reloaded again.

Can you elaborate on why this is?  What the relevant code paths are?

> 
> Saravana explains:
>   Basically for all overlays (I hope the function is only used for
>   overlays) we assume all nodes are NOT devices until they actually
>   get added as a device.
> 
> Based on a patch by Saravana Kannan, which covered only platform and spi
> devices.


> 
> Fixes: 4a032827daa89350 ("of: property: Simplify of_link_to_phandle()")

Hi Frank,

On Wed, Mar 29, 2023 at 6:06 PM Frank Rowand <frowand.list@gmail.com> wrote:
> On 3/24/23 04:30, Geert Uytterhoeven wrote:
> > When loading a DT overlay that creates a device, the device is not
> > instantiated, unless the DT overlay is unloaded and reloaded again.
>
> Can you elaborate on why this is?  What the relevant code paths are?

Honestly, I don't know.

On Fri, Mar 24, 2023 at 2:30 AM Geert Uytterhoeven
<geert+renesas@glider.be> wrote:
>

I saw all the other replies. Let me give more details on the commit
text and inline comments so it's clearer and not a "he said he said"

> When loading a DT overlay that creates a device, the device is not
> instantiated,

IIRC I think the devices are instantiated, but not probed.

> unless the DT overlay is unloaded and reloaded again.
>
> Saravana explains:
>   Basically for all overlays (I hope the function is only used for
>   overlays) we assume all nodes are NOT devices until they actually
>   get added as a device.

This comment was about what the patch is doing. I think a better
commit text would be something like below. Feel free to reword as you
see fit.

After the recent refactor to improve fw_devlink[1], it no longer depends on
"compatible" property to identify which device tree nodes will
become struct device. fw_devlink now picks up dangling consumers (consumers
pointing to descendent device tree nodes of a device that aren't converted
to child devices) when a device is successfully bound to a driver. See
__fw_devlink_pickup_dangling_consumers().

However, during DT overlay, a device's device tree node can have sub-nodes
added/removed without unbinding/rebinding the driver. This difference in
behavior between the normal device instantiation and probing flow vs the DT
overlay flow has a bunch of implications that are pointed out elsewhere[2].
One of them is that the fw_devlink logic to pick up dangling consumers is
never exercised.

This patch solves the fw_devlink issue by marking all DT nodes added by DT
overlay with FWNODE_FLAG_NOT_DEVICE (fwnode that won't become device) and
then clearing the flag when a struct device is actually created for the DT
node. This way, fw_devlink knows not to have consumers waiting on these
newly added DT nodes and to propagate the dependency to an ancestor DT node
that has corresponding struct device.

[1] - https://lore.kernel.org/lkml/20230207014207.1678715-1-saravanak@google.com/
[2] - https://lore.kernel.org/lkml/CAGETcx_bkuFaLCiPrAWCPQz+w79ccDp6=9e881qmK=vx3hBMyg@mail.gmail.com/#t


> Based on a patch by Saravana Kannan, which covered only platform and spi
> devices.

You can keep this in the commit text.

> Fixes: 4a032827daa89350 ("of: property: Simplify of_link_to_phandle()")
> Link: https://lore.kernel.org/r/CAGETcx_+rhHvaC_HJXGrr5_WAd2+k5f=rWYnkCZ6z5bGX-wj4w@mail.gmail.com
> Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
> Acked-by: Mark Brown <broonie@kernel.org>
> ---
> v2:
>   - Add Acked-by,
>   - Drop RFC.
> ---
>  drivers/bus/imx-weim.c    | 1 +
>  drivers/i2c/i2c-core-of.c | 1 +
>  drivers/of/dynamic.c      | 1 +
>  drivers/of/platform.c     | 1 +
>  drivers/spi/spi.c         | 1 +
>  5 files changed, 5 insertions(+)
>
> diff --git a/drivers/bus/imx-weim.c b/drivers/bus/imx-weim.c
> index 36d42484142aede2..898e23a4231400fa 100644
> --- a/drivers/bus/imx-weim.c
> +++ b/drivers/bus/imx-weim.c
> @@ -329,6 +329,7 @@ static int of_weim_notify(struct notifier_block *nb, unsigned long action,
>                                  "Failed to setup timing for '%pOF'\n", rd->dn);
>
>                 if (!of_node_check_flag(rd->dn, OF_POPULATED)) {

It's important this flag clearing is done before the device is added.
So, can you please add a comment before all these clear flag lines
that's something like:

/* Clear the flag before adding the device so that fw_devlink doesn't
skip adding consumers to this device. */

Thanks,
Saravana

> +                       rd->dn->fwnode.flags &= ~FWNODE_FLAG_NOT_DEVICE;
>                         if (!of_platform_device_create(rd->dn, NULL, &pdev->dev)) {
>                                 dev_err(&pdev->dev,
>                                         "Failed to create child device '%pOF'\n",
> diff --git a/drivers/i2c/i2c-core-of.c b/drivers/i2c/i2c-core-of.c
> index aa93467784c29c89..303f9003562eed3d 100644
> --- a/drivers/i2c/i2c-core-of.c
> +++ b/drivers/i2c/i2c-core-of.c
> @@ -178,6 +178,7 @@ static int of_i2c_notify(struct notifier_block *nb, unsigned long action,
>                         return NOTIFY_OK;
>                 }
>
> +               rd->dn->fwnode.flags &= ~FWNODE_FLAG_NOT_DEVICE;
>                 client = of_i2c_register_device(adap, rd->dn);
>                 if (IS_ERR(client)) {
>                         dev_err(&adap->dev, "failed to create client for '%pOF'\n",
> diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
> index 07d93753b12f5f4d..e311d406b1705306 100644
> --- a/drivers/of/dynamic.c
> +++ b/drivers/of/dynamic.c
> @@ -226,6 +226,7 @@ static void __of_attach_node(struct device_node *np)
>         np->sibling = np->parent->child;
>         np->parent->child = np;
>         of_node_clear_flag(np, OF_DETACHED);
> +       np->fwnode.flags |= FWNODE_FLAG_NOT_DEVICE;
>  }
>
>  /**
> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> index b2bd2e783445dd78..17c92cbfb62ee3ef 100644
> --- a/drivers/of/platform.c
> +++ b/drivers/of/platform.c
> @@ -737,6 +737,7 @@ static int of_platform_notify(struct notifier_block *nb,
>                 if (of_node_check_flag(rd->dn, OF_POPULATED))
>                         return NOTIFY_OK;
>
> +               rd->dn->fwnode.flags &= ~FWNODE_FLAG_NOT_DEVICE;
>                 /* pdev_parent may be NULL when no bus platform device */
>                 pdev_parent = of_find_device_by_node(rd->dn->parent);
>                 pdev = of_platform_device_create(rd->dn, NULL,
> diff --git a/drivers/spi/spi.c b/drivers/spi/spi.c
> index 8e8af148b1dc371e..66ac67580d2a473b 100644
> --- a/drivers/spi/spi.c
> +++ b/drivers/spi/spi.c
> @@ -4527,6 +4527,7 @@ static int of_spi_notify(struct notifier_block *nb, unsigned long action,
>                         return NOTIFY_OK;
>                 }
>
> +               rd->dn->fwnode.flags &= ~FWNODE_FLAG_NOT_DEVICE;
>                 spi = of_register_spi_device(ctlr, rd->dn);
>                 put_device(&ctlr->dev);
>
> --
> 2.34.1
>