Message ID | 20201121020232.908850-1-saravanak@google.com |
---|---|
Headers | show |
Series | Refactor fw_devlink to significantly improve boot time | expand |
Hi, On 21/11/2020 04:02, Saravana Kannan wrote: > The current implementation of fw_devlink is very inefficient because it > tries to get away without creating fwnode links in the name of saving > memory usage. Past attempts to optimize runtime at the cost of memory > usage were blocked with request for data showing that the optimization > made significant improvement for real world scenarios. > > We have those scenarios now. There have been several reports of boot > time increase in the order of seconds in this thread [1]. Several OEMs > and SoC manufacturers have also privately reported significant > (350-400ms) increase in boot time due to all the parsing done by > fw_devlink. > > So this patch series refactors fw_devlink to be more efficient. The key > difference now is the addition of support for fwnode links -- just a few > simple APIs. This also allows most of the code to be moved out of > firmware specific (DT mostly) code into driver core. > > This brings the following benefits: > - Instead of parsing the device tree multiple times (complexity was > close to O(N^3) where N in the number of properties) during bootup, > fw_devlink parses each fwnode node/property only once and creates > fwnode links. The rest of the fw_devlink code then just looks at these > fwnode links to do rest of the work. > > - Makes it much easier to debug probe issue due to fw_devlink in the > future. fw_devlink=on blocks the probing of devices if they depend on > a device that hasn't been added yet. With this refactor, it'll be very > easy to tell what that device is because we now have a reference to > the fwnode of the device. > > - Much easier to add fw_devlink support to ACPI and other firmware > types. A refactor to move the common bits from DT specific code to > driver core was in my TODO list as a prerequisite to adding ACPI > support to fw_devlink. This series gets that done. > > Laurent and Grygorii tested the v1 series and they saw boot time > improvment of about 12 seconds and 3 seconds, respectively. Tested v2 on OMAP4 SDP. With my particular config, boot time to starting init went from 18.5 seconds to 12.5 seconds. Tomi
On Tue, Nov 24, 2020 at 12:29 AM 'Tomi Valkeinen' via kernel-team <kernel-team@android.com> wrote: > > Hi, > > On 21/11/2020 04:02, Saravana Kannan wrote: > > The current implementation of fw_devlink is very inefficient because it > > tries to get away without creating fwnode links in the name of saving > > memory usage. Past attempts to optimize runtime at the cost of memory > > usage were blocked with request for data showing that the optimization > > made significant improvement for real world scenarios. > > > > We have those scenarios now. There have been several reports of boot > > time increase in the order of seconds in this thread [1]. Several OEMs > > and SoC manufacturers have also privately reported significant > > (350-400ms) increase in boot time due to all the parsing done by > > fw_devlink. > > > > So this patch series refactors fw_devlink to be more efficient. The key > > difference now is the addition of support for fwnode links -- just a few > > simple APIs. This also allows most of the code to be moved out of > > firmware specific (DT mostly) code into driver core. > > > > This brings the following benefits: > > - Instead of parsing the device tree multiple times (complexity was > > close to O(N^3) where N in the number of properties) during bootup, > > fw_devlink parses each fwnode node/property only once and creates > > fwnode links. The rest of the fw_devlink code then just looks at these > > fwnode links to do rest of the work. > > > > - Makes it much easier to debug probe issue due to fw_devlink in the > > future. fw_devlink=on blocks the probing of devices if they depend on > > a device that hasn't been added yet. With this refactor, it'll be very > > easy to tell what that device is because we now have a reference to > > the fwnode of the device. > > > > - Much easier to add fw_devlink support to ACPI and other firmware > > types. A refactor to move the common bits from DT specific code to > > driver core was in my TODO list as a prerequisite to adding ACPI > > support to fw_devlink. This series gets that done. > > > > Laurent and Grygorii tested the v1 series and they saw boot time > > improvment of about 12 seconds and 3 seconds, respectively. > > Tested v2 on OMAP4 SDP. With my particular config, boot time to starting init went from 18.5 seconds > to 12.5 seconds. Thanks for testing Tomi! -Saravana
On Tue, Nov 24, 2020 at 12:29 AM 'Tomi Valkeinen' via kernel-team <kernel-team@android.com> wrote: > > Hi, > > On 21/11/2020 04:02, Saravana Kannan wrote: > > The current implementation of fw_devlink is very inefficient because it > > tries to get away without creating fwnode links in the name of saving > > memory usage. Past attempts to optimize runtime at the cost of memory > > usage were blocked with request for data showing that the optimization > > made significant improvement for real world scenarios. > > > > We have those scenarios now. There have been several reports of boot > > time increase in the order of seconds in this thread [1]. Several OEMs > > and SoC manufacturers have also privately reported significant > > (350-400ms) increase in boot time due to all the parsing done by > > fw_devlink. > > > > So this patch series refactors fw_devlink to be more efficient. The key > > difference now is the addition of support for fwnode links -- just a few > > simple APIs. This also allows most of the code to be moved out of > > firmware specific (DT mostly) code into driver core. > > > > This brings the following benefits: > > - Instead of parsing the device tree multiple times (complexity was > > close to O(N^3) where N in the number of properties) during bootup, > > fw_devlink parses each fwnode node/property only once and creates > > fwnode links. The rest of the fw_devlink code then just looks at these > > fwnode links to do rest of the work. > > > > - Makes it much easier to debug probe issue due to fw_devlink in the > > future. fw_devlink=on blocks the probing of devices if they depend on > > a device that hasn't been added yet. With this refactor, it'll be very > > easy to tell what that device is because we now have a reference to > > the fwnode of the device. > > > > - Much easier to add fw_devlink support to ACPI and other firmware > > types. A refactor to move the common bits from DT specific code to > > driver core was in my TODO list as a prerequisite to adding ACPI > > support to fw_devlink. This series gets that done. > > > > Laurent and Grygorii tested the v1 series and they saw boot time > > improvment of about 12 seconds and 3 seconds, respectively. > > Tested v2 on OMAP4 SDP. With my particular config, boot time to starting init went from 18.5 seconds > to 12.5 seconds. > > Tomi Rafael, Friendly reminder for a review. -Saravana
On Fri, Nov 20, 2020 at 06:02:23PM -0800, Saravana Kannan wrote: > Add support for creating supplier-consumer links between fwnodes. It is > intended for internal use the driver core and generic firmware support > code (eg. Device Tree, ACPI), so it is simple by design and the API > provided is limited. > > Signed-off-by: Saravana Kannan <saravanak@google.com> > --- > drivers/base/core.c | 98 ++++++++++++++++++++++++++++++++++++++++++ > drivers/of/dynamic.c | 1 + > include/linux/fwnode.h | 14 ++++++ > 3 files changed, 113 insertions(+) > > diff --git a/drivers/base/core.c b/drivers/base/core.c > index 401fa7e3505c..e2b246a44d1a 100644 > --- a/drivers/base/core.c > +++ b/drivers/base/core.c > @@ -50,6 +50,104 @@ static LIST_HEAD(wait_for_suppliers); > static DEFINE_MUTEX(wfs_lock); > static LIST_HEAD(deferred_sync); > static unsigned int defer_sync_state_count = 1; > +static DEFINE_MUTEX(fwnode_link_lock); > + > +/** > + * fwnode_link_add - Create a link between two fwnode_handles. > + * @con: Consumer end of the link. > + * @sup: Supplier end of the link. > + * > + * Create a fwnode link between fwnode handles @con and @sup. The fwnode link > + * represents the detail that the firmware lists @sup fwnode as supplying a > + * resource to @con. > + * > + * The driver core will use the fwnode link to create a device link between the > + * two device objects corresponding to @con and @sup when they are created. The > + * driver core will automatically delete the fwnode link between @con and @sup > + * after doing that. > + * > + * Attempts to create duplicate links between the same pair of fwnode handles > + * are ignored and there is no reference counting. Sorry to ask, but why is that? Isn't this a programmer error? Thanks
On Sat, Dec 5, 2020 at 11:48 PM Leon Romanovsky <leon@kernel.org> wrote: > > On Fri, Nov 20, 2020 at 06:02:23PM -0800, Saravana Kannan wrote: > > Add support for creating supplier-consumer links between fwnodes. It is > > intended for internal use the driver core and generic firmware support > > code (eg. Device Tree, ACPI), so it is simple by design and the API > > provided is limited. > > > > Signed-off-by: Saravana Kannan <saravanak@google.com> > > --- > > drivers/base/core.c | 98 ++++++++++++++++++++++++++++++++++++++++++ > > drivers/of/dynamic.c | 1 + > > include/linux/fwnode.h | 14 ++++++ > > 3 files changed, 113 insertions(+) > > > > diff --git a/drivers/base/core.c b/drivers/base/core.c > > index 401fa7e3505c..e2b246a44d1a 100644 > > --- a/drivers/base/core.c > > +++ b/drivers/base/core.c > > @@ -50,6 +50,104 @@ static LIST_HEAD(wait_for_suppliers); > > static DEFINE_MUTEX(wfs_lock); > > static LIST_HEAD(deferred_sync); > > static unsigned int defer_sync_state_count = 1; > > +static DEFINE_MUTEX(fwnode_link_lock); > > + > > +/** > > + * fwnode_link_add - Create a link between two fwnode_handles. > > + * @con: Consumer end of the link. > > + * @sup: Supplier end of the link. > > + * > > + * Create a fwnode link between fwnode handles @con and @sup. The fwnode link > > + * represents the detail that the firmware lists @sup fwnode as supplying a > > + * resource to @con. > > + * > > + * The driver core will use the fwnode link to create a device link between the > > + * two device objects corresponding to @con and @sup when they are created. The > > + * driver core will automatically delete the fwnode link between @con and @sup > > + * after doing that. > > + * > > + * Attempts to create duplicate links between the same pair of fwnode handles > > + * are ignored and there is no reference counting. > > Sorry to ask, but why is that? > Isn't this a programmer error? No, not a programmer error. One firmware node can point to the same supplier many times. For example, the consumer can be using multiple clocks from the same supplier clock controller. In the context of fw_devlink, there's no reason to keep track of each clock dependency separately because we'll be creating only one device link from fwnode link. So multiple fwnode link attempts between the same two devices are just treated as one instance of dependency. I hope that clarifies things. -Saravana
On Mon, Dec 07, 2020 at 11:25:03AM -0800, Saravana Kannan wrote: > On Sat, Dec 5, 2020 at 11:48 PM Leon Romanovsky <leon@kernel.org> wrote: > > > > On Fri, Nov 20, 2020 at 06:02:23PM -0800, Saravana Kannan wrote: > > > Add support for creating supplier-consumer links between fwnodes. It is > > > intended for internal use the driver core and generic firmware support > > > code (eg. Device Tree, ACPI), so it is simple by design and the API > > > provided is limited. > > > > > > Signed-off-by: Saravana Kannan <saravanak@google.com> > > > --- > > > drivers/base/core.c | 98 ++++++++++++++++++++++++++++++++++++++++++ > > > drivers/of/dynamic.c | 1 + > > > include/linux/fwnode.h | 14 ++++++ > > > 3 files changed, 113 insertions(+) > > > > > > diff --git a/drivers/base/core.c b/drivers/base/core.c > > > index 401fa7e3505c..e2b246a44d1a 100644 > > > --- a/drivers/base/core.c > > > +++ b/drivers/base/core.c > > > @@ -50,6 +50,104 @@ static LIST_HEAD(wait_for_suppliers); > > > static DEFINE_MUTEX(wfs_lock); > > > static LIST_HEAD(deferred_sync); > > > static unsigned int defer_sync_state_count = 1; > > > +static DEFINE_MUTEX(fwnode_link_lock); > > > + > > > +/** > > > + * fwnode_link_add - Create a link between two fwnode_handles. > > > + * @con: Consumer end of the link. > > > + * @sup: Supplier end of the link. > > > + * > > > + * Create a fwnode link between fwnode handles @con and @sup. The fwnode link > > > + * represents the detail that the firmware lists @sup fwnode as supplying a > > > + * resource to @con. > > > + * > > > + * The driver core will use the fwnode link to create a device link between the > > > + * two device objects corresponding to @con and @sup when they are created. The > > > + * driver core will automatically delete the fwnode link between @con and @sup > > > + * after doing that. > > > + * > > > + * Attempts to create duplicate links between the same pair of fwnode handles > > > + * are ignored and there is no reference counting. > > > > Sorry to ask, but why is that? > > Isn't this a programmer error? > > No, not a programmer error. > > One firmware node can point to the same supplier many times. For > example, the consumer can be using multiple clocks from the same > supplier clock controller. In the context of fw_devlink, there's no > reason to keep track of each clock dependency separately because we'll > be creating only one device link from fwnode link. So multiple fwnode > link attempts between the same two devices are just treated as one > instance of dependency. I hope that clarifies things. Yes, thanks. > > -Saravana
On Fri, 20 Nov 2020 18:02:23 -0800, Saravana Kannan wrote: > Add support for creating supplier-consumer links between fwnodes. It is > intended for internal use the driver core and generic firmware support > code (eg. Device Tree, ACPI), so it is simple by design and the API > provided is limited. > > Signed-off-by: Saravana Kannan <saravanak@google.com> > --- > drivers/base/core.c | 98 ++++++++++++++++++++++++++++++++++++++++++ > drivers/of/dynamic.c | 1 + > include/linux/fwnode.h | 14 ++++++ > 3 files changed, 113 insertions(+) > Acked-by: Rob Herring <robh@kernel.org>
On Fri, 20 Nov 2020 18:02:29 -0800, Saravana Kannan wrote: > The semantics of add_links() has changed from creating device link > between devices to creating fwnode links between fwnodes. So, update the > implementation of add_links() to match the new semantics. > > Signed-off-by: Saravana Kannan <saravanak@google.com> > --- > drivers/of/property.c | 150 ++++++++++++------------------------------ > 1 file changed, 41 insertions(+), 109 deletions(-) > Acked-by: Rob Herring <robh@kernel.org>
On Fri, Nov 20, 2020 at 06:02:15PM -0800, Saravana Kannan wrote: > The current implementation of fw_devlink is very inefficient because it > tries to get away without creating fwnode links in the name of saving > memory usage. Past attempts to optimize runtime at the cost of memory > usage were blocked with request for data showing that the optimization > made significant improvement for real world scenarios. > > We have those scenarios now. There have been several reports of boot > time increase in the order of seconds in this thread [1]. Several OEMs > and SoC manufacturers have also privately reported significant > (350-400ms) increase in boot time due to all the parsing done by > fw_devlink. > > So this patch series refactors fw_devlink to be more efficient. The key > difference now is the addition of support for fwnode links -- just a few > simple APIs. This also allows most of the code to be moved out of > firmware specific (DT mostly) code into driver core. > > This brings the following benefits: > - Instead of parsing the device tree multiple times (complexity was > close to O(N^3) where N in the number of properties) during bootup, > fw_devlink parses each fwnode node/property only once and creates > fwnode links. The rest of the fw_devlink code then just looks at these > fwnode links to do rest of the work. > > - Makes it much easier to debug probe issue due to fw_devlink in the > future. fw_devlink=on blocks the probing of devices if they depend on > a device that hasn't been added yet. With this refactor, it'll be very > easy to tell what that device is because we now have a reference to > the fwnode of the device. > > - Much easier to add fw_devlink support to ACPI and other firmware > types. A refactor to move the common bits from DT specific code to > driver core was in my TODO list as a prerequisite to adding ACPI > support to fw_devlink. This series gets that done. > > Laurent and Grygorii tested the v1 series and they saw boot time > improvment of about 12 seconds and 3 seconds, respectively. Now queued up to my tree. Note, I had to hand-apply patches 13 and 16 due to some reason (for 13, I have no idea, for 16 it was due to a previous patch applied to my tree that I cc:ed you on.) Verifying I got it all correct would be great :) thanks, greg k-h
On Wed, Dec 9, 2020 at 10:15 AM Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote: > > On Fri, Nov 20, 2020 at 06:02:15PM -0800, Saravana Kannan wrote: > > The current implementation of fw_devlink is very inefficient because it > > tries to get away without creating fwnode links in the name of saving > > memory usage. Past attempts to optimize runtime at the cost of memory > > usage were blocked with request for data showing that the optimization > > made significant improvement for real world scenarios. > > > > We have those scenarios now. There have been several reports of boot > > time increase in the order of seconds in this thread [1]. Several OEMs > > and SoC manufacturers have also privately reported significant > > (350-400ms) increase in boot time due to all the parsing done by > > fw_devlink. > > > > So this patch series refactors fw_devlink to be more efficient. The key > > difference now is the addition of support for fwnode links -- just a few > > simple APIs. This also allows most of the code to be moved out of > > firmware specific (DT mostly) code into driver core. > > > > This brings the following benefits: > > - Instead of parsing the device tree multiple times (complexity was > > close to O(N^3) where N in the number of properties) during bootup, > > fw_devlink parses each fwnode node/property only once and creates > > fwnode links. The rest of the fw_devlink code then just looks at these > > fwnode links to do rest of the work. > > > > - Makes it much easier to debug probe issue due to fw_devlink in the > > future. fw_devlink=on blocks the probing of devices if they depend on > > a device that hasn't been added yet. With this refactor, it'll be very > > easy to tell what that device is because we now have a reference to > > the fwnode of the device. > > > > - Much easier to add fw_devlink support to ACPI and other firmware > > types. A refactor to move the common bits from DT specific code to > > driver core was in my TODO list as a prerequisite to adding ACPI > > support to fw_devlink. This series gets that done. > > > > Laurent and Grygorii tested the v1 series and they saw boot time > > improvment of about 12 seconds and 3 seconds, respectively. > > Now queued up to my tree. Note, I had to hand-apply patches 13 and 16 > due to some reason (for 13, I have no idea, for 16 it was due to a > previous patch applied to my tree that I cc:ed you on.) > > Verifying I got it all correct would be great :) A quick diff of drivers/base/core.c between driver-core-testing and my local tree doesn't show any major diff (only some unrelated comment fixes). So, it looks fine. The patch 13 conflict is probably due to having to rebase the v2 series on top of this: https://lore.kernel.org/lkml/20201104205431.3795207-1-saravanak@google.com/ And looks like Patch 16 was handled fine. Thanks for applying the series. -Saravana
On Wed, Dec 09, 2020 at 12:24:32PM -0800, Saravana Kannan wrote: > On Wed, Dec 9, 2020 at 10:15 AM Greg Kroah-Hartman > <gregkh@linuxfoundation.org> wrote: > > > > On Fri, Nov 20, 2020 at 06:02:15PM -0800, Saravana Kannan wrote: > > > The current implementation of fw_devlink is very inefficient because it > > > tries to get away without creating fwnode links in the name of saving > > > memory usage. Past attempts to optimize runtime at the cost of memory > > > usage were blocked with request for data showing that the optimization > > > made significant improvement for real world scenarios. > > > > > > We have those scenarios now. There have been several reports of boot > > > time increase in the order of seconds in this thread [1]. Several OEMs > > > and SoC manufacturers have also privately reported significant > > > (350-400ms) increase in boot time due to all the parsing done by > > > fw_devlink. > > > > > > So this patch series refactors fw_devlink to be more efficient. The key > > > difference now is the addition of support for fwnode links -- just a few > > > simple APIs. This also allows most of the code to be moved out of > > > firmware specific (DT mostly) code into driver core. > > > > > > This brings the following benefits: > > > - Instead of parsing the device tree multiple times (complexity was > > > close to O(N^3) where N in the number of properties) during bootup, > > > fw_devlink parses each fwnode node/property only once and creates > > > fwnode links. The rest of the fw_devlink code then just looks at these > > > fwnode links to do rest of the work. > > > > > > - Makes it much easier to debug probe issue due to fw_devlink in the > > > future. fw_devlink=on blocks the probing of devices if they depend on > > > a device that hasn't been added yet. With this refactor, it'll be very > > > easy to tell what that device is because we now have a reference to > > > the fwnode of the device. > > > > > > - Much easier to add fw_devlink support to ACPI and other firmware > > > types. A refactor to move the common bits from DT specific code to > > > driver core was in my TODO list as a prerequisite to adding ACPI > > > support to fw_devlink. This series gets that done. > > > > > > Laurent and Grygorii tested the v1 series and they saw boot time > > > improvment of about 12 seconds and 3 seconds, respectively. > > > > Now queued up to my tree. Note, I had to hand-apply patches 13 and 16 > > due to some reason (for 13, I have no idea, for 16 it was due to a > > previous patch applied to my tree that I cc:ed you on.) > > > > Verifying I got it all correct would be great :) > > A quick diff of drivers/base/core.c between driver-core-testing and my > local tree doesn't show any major diff (only some unrelated comment > fixes). So, it looks fine. > > The patch 13 conflict is probably due to having to rebase the v2 > series on top of this: > https://lore.kernel.org/lkml/20201104205431.3795207-1-saravanak@google.com/ > > And looks like Patch 16 was handled fine. Great, thanks for verifying! greg k-h