| Message ID | 20220511112438.1251024-1-javierm@redhat.com |
|---|---|
| Headers | show |
| Series | Fix some races between sysfb device registration and drivers probe | expand |
Hi Javier. On Wed, May 11, 2022 at 01:32:30PM +0200, Javier Martinez Canillas wrote: > From: Daniel Vetter <daniel.vetter@ffwll.ch> > > Well except when the olpc dcon fbdev driver is enabled, that thing > digs around in there in rather unfixable ways. > > Cc oldc_dcon maintainers as fyi. Another way to fix this is to mark FB_OLPC_DCON and add a TODO entry to fix this. We are really not supposed to carry such special code around to keep staging working. I know this may not be a popular viewpoint, but just look at the ugly workarounds required here. Sam > > v2: I typoed the config name (0day) > > Cc: kernel test robot <lkp@intel.com> > Cc: Jens Frederich <jfrederich@gmail.com> > Cc: Jon Nettleton <jon.nettleton@gmail.com> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > Cc: linux-staging@lists.linux.dev > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> > Reviewed-by: Javier Martinez Canillas <javierm@redhat.com> > Cc: Daniel Vetter <daniel@ffwll.ch> > Cc: Helge Deller <deller@gmx.de> > Cc: Matthew Wilcox <willy@infradead.org> > Cc: Sam Ravnborg <sam@ravnborg.org> > Cc: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> > Cc: Zhen Lei <thunder.leizhen@huawei.com> > Cc: Alex Deucher <alexander.deucher@amd.com> > Cc: Xiyu Yang <xiyuyang19@fudan.edu.cn> > Cc: linux-fbdev@vger.kernel.org > Cc: Zheyu Ma <zheyuma97@gmail.com> > Cc: Guenter Roeck <linux@roeck-us.net> > Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> > --- > > (no changes since v1) > > drivers/video/fbdev/core/fbmem.c | 8 ++++++-- > include/linux/fb.h | 7 +++---- > 2 files changed, 9 insertions(+), 6 deletions(-) > > diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c > index 265efa189bcc..6cab5f4c1fb3 100644 > --- a/drivers/video/fbdev/core/fbmem.c > +++ b/drivers/video/fbdev/core/fbmem.c > @@ -50,10 +50,14 @@ > static DEFINE_MUTEX(registration_lock); > > struct fb_info *registered_fb[FB_MAX] __read_mostly; > -EXPORT_SYMBOL(registered_fb); > - > int num_registered_fb __read_mostly; > +#if IS_ENABLED(CONFIG_FB_OLPC_DCON) > +EXPORT_SYMBOL(registered_fb); > EXPORT_SYMBOL(num_registered_fb); > +#endif It is stuff like this I refer to as "ugly" in the comment above. Sam
Hello Guenter, On 5/11/22 19:17, Guenter Roeck wrote: > On 5/11/22 10:00, Sam Ravnborg wrote: [snip] >>> struct fb_info *registered_fb[FB_MAX] __read_mostly; >>> -EXPORT_SYMBOL(registered_fb); >>> - >>> int num_registered_fb __read_mostly; >>> +#if IS_ENABLED(CONFIG_FB_OLPC_DCON) >>> +EXPORT_SYMBOL(registered_fb); >>> EXPORT_SYMBOL(num_registered_fb); >>> +#endif >> >> It is stuff like this I refer to as "ugly" in the comment above. >> > > My "solution" for that kind of thing is to use a namespace, > such as > > EXPORT_SYMBOL_NS(registered_fb, FB_OLPC_DCON); > EXPORT_SYMBOL_NS(num_registered_fb, FB_OLPC_DCON); > Using a namespace in this case is indeed a great idea I think. I've used in the past to limit the export of a symbol for within a driver that could be scattered across different compilations units, but it never occurred to me using it to limit symbols exported by core code. > and import it from the offending code. That avoids ifdefs > while at the same time limiting the use of the symbols > to the expected scope. Of course that could be abused but > that abuse would be obvious. > Agreed. For the next revision, besides using an namespaced export symbol as you suggested, I'll include a comment to make clear that it shouldn't by any other driver and FB_OLPC_DCON fixed instead.
On Wed, May 11, 2022 at 07:34:38PM +0200, Javier Martinez Canillas wrote: > Hello Guenter, > > On 5/11/22 19:17, Guenter Roeck wrote: > > On 5/11/22 10:00, Sam Ravnborg wrote: > > [snip] > > >>> struct fb_info *registered_fb[FB_MAX] __read_mostly; > >>> -EXPORT_SYMBOL(registered_fb); > >>> - > >>> int num_registered_fb __read_mostly; > >>> +#if IS_ENABLED(CONFIG_FB_OLPC_DCON) > >>> +EXPORT_SYMBOL(registered_fb); > >>> EXPORT_SYMBOL(num_registered_fb); > >>> +#endif > >> > >> It is stuff like this I refer to as "ugly" in the comment above. > >> > > > > My "solution" for that kind of thing is to use a namespace, > > such as > > > > EXPORT_SYMBOL_NS(registered_fb, FB_OLPC_DCON); > > EXPORT_SYMBOL_NS(num_registered_fb, FB_OLPC_DCON); > > > > Using a namespace in this case is indeed a great idea I think. > > I've used in the past to limit the export of a symbol for within a driver > that could be scattered across different compilations units, but it never > occurred to me using it to limit symbols exported by core code. > > > and import it from the offending code. That avoids ifdefs > > while at the same time limiting the use of the symbols > > to the expected scope. Of course that could be abused but > > that abuse would be obvious. > > > > Agreed. For the next revision, besides using an namespaced export symbol > as you suggested, I'll include a comment to make clear that it shouldn't > by any other driver and FB_OLPC_DCON fixed instead. A very nice compromise, thanks Guenter and Javier. Sam
Hi Javier, thanks again for providing the examples. I think I now better get what you're trying to solve. First of all let's merge patch 3, as it seems unrelated. For the other patches, I'd like to take a step back and try to solve the broader problem. IIRC we talked about this briefly already. From my understanding, the problem of the current code is that removal of the firmware device is build around drivers (either DRM or fbdev). Everything works fine if a driver is bound to the firmware device and the native driver can remove it. In other case, we have to manually disable sysfb and force-remove unbound firmware devices. And the patchset doesn't even cover firmware devices that come from DT nodes. But what we really want is to track resource ownership based on devices. When we add a firmware device (via sysfb or DT), we immediately add it to a list of firmware devices. When the native driver arrives, it can remove the firmware device even if no driver has been bound. We can also operate in the other way if the native drivers implicitly reserves the framebuffer range. If sysfb would try to register a firmware device in that range, he can let it fail. No more need to fully disable sysfb on the first arriving native device. We already track the memory ranges in drm aperture helpers. We'd need to move the code to a more prominent location (e.g., <linux/aperture.h>) and change fbdev to use it. Sysfb and DT code needs to insert platform devices upon creation. We can then implement the more fancy stuff, such as tracking native-device memory. (And if we ever get to fix all usage of Linux' request-mem-region, we can even move all the functionality there). Best regards Thomas Am 11.05.22 um 13:24 schrieb Javier Martinez Canillas: > Hello, > > The patches in this series contain mostly changes suggested by Daniel Vetter > Thomas Zimmermann. They aim to fix existing races between the Generic System > Framebuffer (sysfb) infrastructure and the fbdev and DRM device registration. > > For example, it is currently possible for sysfb to register a platform > device after a real DRM driver was registered and requested to remove the > conflicting framebuffers. Or is possible for a simple{fb,drm} to match with > a device previously registered by sysfb, even after a real driver is present. > > A symptom of this issue, was worked around with the commit fb561bf9abde > ("fbdev: Prevent probing generic drivers if a FB is already registered") > but that's really a hack and should be reverted instead. > > This series attempt to fix it more correctly and revert the mentioned hack. > That will also allow to make the num_registered_fb variable not visible to > drivers anymore, since that's internal to fbdev core. > > Pach 1 is just a simple cleanup in preparation for later patches. > > Patch 2 add a sysfb_disable() and sysfb_try_unregister() helpers to allow > disabling sysfb and attempt to unregister registered devices respectively. > > Patch 3 changes how is dealt with conflicting framebuffers unregistering, > rather than having a variable to determine if a lock should be take, it > just drops the lock before unregistering the platform device. > > Patch 4 changes the fbdev core to not attempt to unregister devices that > were registered by sysfb, let the same code doing the registration to also > handle the unregistration. > > Patch 5 fixes the race that exists between sysfb devices registration and > fbdev framebuffer devices registration, by disabling the sysfb when a DRM > or fbdev driver requests to remove conflicting framebuffers. > > Patch 6 is the revert patch that was posted by Daniel before but dropped > from his set and finally patch 7 is the one that makes num_registered_fb > private to fbmem.c, to not allow drivers to use it anymore. > > The patches were tested on a rpi4 with the vc4, simpledrm and simplefb > drivers, using different combinations of built-in and as a module. > > For example, having simpledrm as a module and loading it after the vc4 > driver probed would not register a DRM device, which happens now without > the patches from this series. > > Best regards, > Javier > > Changes in v5: > - Move the sysfb_disable() call at conflicting framebuffers again to > avoid the need of a DRIVER_FIRMWARE capability flag. > - Add Daniel Vetter's Reviewed-by tag again since reverted to the old > patch that he already reviewed in v2. > - Drop patches that added a DRM_FIRMWARE capability and use them > since the case those prevented could be ignored (Daniel Vetter). > > Changes in v4: > - Make sysfb_disable() to also attempt to unregister a device. > - Drop call to sysfb_disable() in fbmem since is done in other places now. > - Add patch to make registered_fb[] private. > - Add patches that introduce the DRM_FIRMWARE capability and usage. > > Changes in v3: > - Add Thomas Zimmermann's Reviewed-by tag to patch #1. > - Call sysfb_disable() when a DRM dev and a fbdev are registered rather > than when conflicting framebuffers are removed (Thomas Zimmermann). > - Call sysfb_disable() when a fbdev framebuffer is registered rather > than when conflicting framebuffers are removed (Thomas Zimmermann). > - Drop Daniel Vetter's Reviewed-by tag since patch changed a lot. > - Rebase on top of latest drm-misc-next branch. > > Changes in v2: > - Rebase on top of latest drm-misc-next and fix conflicts (Daniel Vetter). > - Add kernel-doc comments and include in other_interfaces.rst (Daniel Vetter). > - Explain in the commit message that fbmem has to unregister the device > as fallback if a driver registered the device itself (Daniel Vetter). > - Also explain that fallback in a comment in the code (Daniel Vetter). > - Don't encode in fbmem the assumption that sysfb will always register > platform devices (Daniel Vetter). > - Add a FIXME comment about drivers registering devices (Daniel Vetter). > - Explain in the commit message that fbmem has to unregister the device > as fallback if a driver registered the device itself (Daniel Vetter). > - Also explain that fallback in a comment in the code (Daniel Vetter). > - Don't encode in fbmem the assumption that sysfb will always register > platform devices (Daniel Vetter). > - Add a FIXME comment about drivers registering devices (Daniel Vetter). > - Drop RFC prefix since patches were already reviewed by Daniel Vetter. > - Add Daniel Reviewed-by tags to the patches. > > Daniel Vetter (2): > Revert "fbdev: Prevent probing generic drivers if a FB is already > registered" > fbdev: Make registered_fb[] private to fbmem.c > > Javier Martinez Canillas (5): > firmware: sysfb: Make sysfb_create_simplefb() return a pdev pointer > firmware: sysfb: Add helpers to unregister a pdev and disable > registration > fbdev: Restart conflicting fb removal loop when unregistering devices > fbdev: Make sysfb to unregister its own registered devices > fbdev: Disable sysfb device registration when removing conflicting FBs > > .../driver-api/firmware/other_interfaces.rst | 6 ++ > drivers/firmware/sysfb.c | 91 +++++++++++++++++-- > drivers/firmware/sysfb_simplefb.c | 16 ++-- > drivers/video/fbdev/core/fbmem.c | 67 +++++++++++--- > drivers/video/fbdev/efifb.c | 11 --- > drivers/video/fbdev/simplefb.c | 11 --- > include/linux/fb.h | 8 +- > include/linux/sysfb.h | 29 +++++- > 8 files changed, 178 insertions(+), 61 deletions(-) >
Hello Thomas, Thanks for your feedback and comments. On 5/13/22 12:48, Thomas Zimmermann wrote: > Hi Javier, > > thanks again for providing the examples. I think I now better get what > you're trying to solve. > You are welcome. > First of all let's merge patch 3, as it seems unrelated. > Agreed. I can push it to drm-misc-next. > For the other patches, I'd like to take a step back and try to solve the > broader problem. IIRC we talked about this briefly already. > Yes, that's what we discussed on IRC some time ago. > From my understanding, the problem of the current code is that removal > of the firmware device is build around drivers (either DRM or fbdev). > Everything works fine if a driver is bound to the firmware device and > the native driver can remove it. In other case, we have to manually And this is the common case, since most kernels will have some driver that binds to a platform device registered to provide the firmware FB. > disable sysfb and force-remove unbound firmware devices. And the > patchset doesn't even cover firmware devices that come from DT nodes. > Indeed. > But what we really want is to track resource ownership based on devices. > When we add a firmware device (via sysfb or DT), we immediately add it > to a list of firmware devices. When the native driver arrives, it can > remove the firmware device even if no driver has been bound. > > We can also operate in the other way if the native drivers implicitly > reserves the framebuffer range. If sysfb would try to register a > firmware device in that range, he can let it fail. No more need to fully > disable sysfb on the first arriving native device. > > We already track the memory ranges in drm aperture helpers. We'd need to > move the code to a more prominent location (e.g., <linux/aperture.h>) > and change fbdev to use it. Sysfb and DT code needs to insert platform > devices upon creation. We can then implement the more fancy stuff, such > as tracking native-device memory. (And if we ever get to fix all usage > of Linux' request-mem-region, we can even move all the functionality there). > Agreed. And as mentioned, the race that these patches attempt to fix are for the less common case when a native driver probes but either no generic driver registered a framebuffer yet or the platform device wasn't registered yet. But this can only happen if for example a native driver is built-in but the generic driver is build as a module, which is not the common configuration. What most distros do is the opposite, to have {simple,of,efi,vesa}fb or simpledrm built-in and the native drivers built as modules. So there's no rush to fix this by piling more hacks on top of the ones we already have and instead try to fix it more properly as you suggested.
Hi Javier Am 13.05.22 um 13:10 schrieb Javier Martinez Canillas: ... >> We already track the memory ranges in drm aperture helpers. We'd need to >> move the code to a more prominent location (e.g., <linux/aperture.h>) >> and change fbdev to use it. Sysfb and DT code needs to insert platform >> devices upon creation. We can then implement the more fancy stuff, such >> as tracking native-device memory. (And if we ever get to fix all usage >> of Linux' request-mem-region, we can even move all the functionality there). >> > > Agreed. And as mentioned, the race that these patches attempt to fix are for > the less common case when a native driver probes but either no generic driver > registered a framebuffer yet or the platform device wasn't registered yet. > > But this can only happen if for example a native driver is built-in but the > generic driver is build as a module, which is not the common configuration. > > What most distros do is the opposite, to have {simple,of,efi,vesa}fb or > simpledrm built-in and the native drivers built as modules. > > So there's no rush to fix this by piling more hacks on top of the ones we > already have and instead try to fix it more properly as you suggested. > A first step would be to use DRM's aperture helpers in fbdev. That would be a good idea anyway, as it would simplify both. You already mentioned some API changes to make aperture helpers DRM-independent. The affected fbdev drivers use platform devices, so this should be easy. Aperture helpers have something like the registration_lock. [1] I don't know if we need to recreate patch 3 for this as well. If we absolutely need some special detachment handling for fbdev, we can make devm_aperture_aquire() a public interface. The detach helper is provided by the caller. Best regards Thomas [1] https://elixir.bootlin.com/linux/v5.17.6/source/drivers/gpu/drm/drm_aperture.c#L254 [2] https://elixir.bootlin.com/linux/v5.17.6/source/drivers/gpu/drm/drm_aperture.c#L159
On 5/11/22 13:30, Javier Martinez Canillas wrote: > Drivers that want to remove registered conflicting framebuffers prior to > register their own framebuffer, calls remove_conflicting_framebuffers(). > > This function takes the registration_lock mutex, to prevent a races when > drivers register framebuffer devices. But if a conflicting framebuffer > device is found, the underlaying platform device is unregistered and this > will lead to the platform driver .remove callback to be called, which in > turn will call to the unregister_framebuffer() that takes the same lock. > > To prevent this, a struct fb_info.forced_out field was used as indication > to unregister_framebuffer() whether the mutex has to be grabbed or not. > > A cleaner solution is to drop the lock before platform_device_unregister() > so unregister_framebuffer() can take it when called from the fbdev driver, > and just grab the lock again after the device has been registered and do > a removal loop restart. > > Since the framebuffer devices will already be removed, the loop would just > finish when no more conflicting framebuffers are found. > > Suggested-by: Daniel Vetter <daniel.vetter@ffwll.ch> > Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> > --- Pushed this to drm-misc (drm-misc-next). Thanks all!
Hello Daniel, On 6/7/22 17:01, Daniel Vetter wrote: [snip] >> >> drivers/video/fbdev/core/fbmem.c | 11 +++++++++++ >> 1 file changed, 11 insertions(+) >> >> diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c >> index 9b035ef4d552..265efa189bcc 100644 >> --- a/drivers/video/fbdev/core/fbmem.c >> +++ b/drivers/video/fbdev/core/fbmem.c >> @@ -1789,6 +1789,17 @@ int remove_conflicting_framebuffers(struct apertures_struct *a, >> if (do_free) >> kfree(a); >> >> + /* >> + * If a driver asked to unregister a platform device registered by >> + * sysfb, then can be assumed that this is a driver for a display >> + * that is set up by the system firmware and has a generic driver. >> + * >> + * Drivers for devices that don't have a generic driver will never >> + * ask for this, so let's assume that a real driver for the display >> + * was already probed and prevent sysfb to register devices later. >> + */ >> + sysfb_disable(); > > So the og version had (or should have had at least) the sysfb_disable() > call before we go through the loop and try to unregister stuff. I think > this needs to be done before we call do_remove_conflicting_framebuffer() > instead. With that: > Yes, the original version did that but also the original version didn't attempt to remove the devices registered by sysfb on sysfb_disable(). I was going to answer that this has to be done after the loop because that way fbmem could first ask sysfb to remove the devices, but then I realized that you are correct That this wouldn't be needed if sysfb does the disable (and unregistration) before the loop. So by doing it before the loop, we should be able to drop [PATCH v5 4/7] fbdev: Make sysfb to unregister its own registered devices: https://lists.freedesktop.org/archives/dri-devel/2022-May/355201.html Since by the time the loop is executed, no registered_fb associated with a device that was registered by sysfb should be present in that array.
Hello, The patches in this series contain mostly changes suggested by Daniel Vetter Thomas Zimmermann. They aim to fix existing races between the Generic System Framebuffer (sysfb) infrastructure and the fbdev and DRM device registration. For example, it is currently possible for sysfb to register a platform device after a real DRM driver was registered and requested to remove the conflicting framebuffers. Or is possible for a simple{fb,drm} to match with a device previously registered by sysfb, even after a real driver is present. A symptom of this issue, was worked around with the commit fb561bf9abde ("fbdev: Prevent probing generic drivers if a FB is already registered") but that's really a hack and should be reverted instead. This series attempt to fix it more correctly and revert the mentioned hack. That will also allow to make the num_registered_fb variable not visible to drivers anymore, since that's internal to fbdev core. Pach 1 is just a simple cleanup in preparation for later patches. Patch 2 add a sysfb_disable() and sysfb_try_unregister() helpers to allow disabling sysfb and attempt to unregister registered devices respectively. Patch 3 changes how is dealt with conflicting framebuffers unregistering, rather than having a variable to determine if a lock should be take, it just drops the lock before unregistering the platform device. Patch 4 changes the fbdev core to not attempt to unregister devices that were registered by sysfb, let the same code doing the registration to also handle the unregistration. Patch 5 fixes the race that exists between sysfb devices registration and fbdev framebuffer devices registration, by disabling the sysfb when a DRM or fbdev driver requests to remove conflicting framebuffers. Patch 6 is the revert patch that was posted by Daniel before but dropped from his set and finally patch 7 is the one that makes num_registered_fb private to fbmem.c, to not allow drivers to use it anymore. The patches were tested on a rpi4 with the vc4, simpledrm and simplefb drivers, using different combinations of built-in and as a module. For example, having simpledrm as a module and loading it after the vc4 driver probed would not register a DRM device, which happens now without the patches from this series. Best regards, Javier Changes in v5: - Move the sysfb_disable() call at conflicting framebuffers again to avoid the need of a DRIVER_FIRMWARE capability flag. - Add Daniel Vetter's Reviewed-by tag again since reverted to the old patch that he already reviewed in v2. - Drop patches that added a DRM_FIRMWARE capability and use them since the case those prevented could be ignored (Daniel Vetter). Changes in v4: - Make sysfb_disable() to also attempt to unregister a device. - Drop call to sysfb_disable() in fbmem since is done in other places now. - Add patch to make registered_fb[] private. - Add patches that introduce the DRM_FIRMWARE capability and usage. Changes in v3: - Add Thomas Zimmermann's Reviewed-by tag to patch #1. - Call sysfb_disable() when a DRM dev and a fbdev are registered rather than when conflicting framebuffers are removed (Thomas Zimmermann). - Call sysfb_disable() when a fbdev framebuffer is registered rather than when conflicting framebuffers are removed (Thomas Zimmermann). - Drop Daniel Vetter's Reviewed-by tag since patch changed a lot. - Rebase on top of latest drm-misc-next branch. Changes in v2: - Rebase on top of latest drm-misc-next and fix conflicts (Daniel Vetter). - Add kernel-doc comments and include in other_interfaces.rst (Daniel Vetter). - Explain in the commit message that fbmem has to unregister the device as fallback if a driver registered the device itself (Daniel Vetter). - Also explain that fallback in a comment in the code (Daniel Vetter). - Don't encode in fbmem the assumption that sysfb will always register platform devices (Daniel Vetter). - Add a FIXME comment about drivers registering devices (Daniel Vetter). - Explain in the commit message that fbmem has to unregister the device as fallback if a driver registered the device itself (Daniel Vetter). - Also explain that fallback in a comment in the code (Daniel Vetter). - Don't encode in fbmem the assumption that sysfb will always register platform devices (Daniel Vetter). - Add a FIXME comment about drivers registering devices (Daniel Vetter). - Drop RFC prefix since patches were already reviewed by Daniel Vetter. - Add Daniel Reviewed-by tags to the patches. Daniel Vetter (2): Revert "fbdev: Prevent probing generic drivers if a FB is already registered" fbdev: Make registered_fb[] private to fbmem.c Javier Martinez Canillas (5): firmware: sysfb: Make sysfb_create_simplefb() return a pdev pointer firmware: sysfb: Add helpers to unregister a pdev and disable registration fbdev: Restart conflicting fb removal loop when unregistering devices fbdev: Make sysfb to unregister its own registered devices fbdev: Disable sysfb device registration when removing conflicting FBs .../driver-api/firmware/other_interfaces.rst | 6 ++ drivers/firmware/sysfb.c | 91 +++++++++++++++++-- drivers/firmware/sysfb_simplefb.c | 16 ++-- drivers/video/fbdev/core/fbmem.c | 67 +++++++++++--- drivers/video/fbdev/efifb.c | 11 --- drivers/video/fbdev/simplefb.c | 11 --- include/linux/fb.h | 8 +- include/linux/sysfb.h | 29 +++++- 8 files changed, 178 insertions(+), 61 deletions(-)