| Message ID | 20240815-preemption-a750-t-v1-0-7bda26c34037@gmail.com |
|---|---|
| Headers | show |
| Series | Preemption support for A7XX | expand |
On Thu, Aug 15, 2024 at 11:27 AM Antonino Maniscalco <antomani103@gmail.com> wrote: > > This series implements preemption for A7XX targets, which allows the GPU to > switch to an higher priority ring when work is pushed to it, reducing latency > for high priority submissions. > > This series enables L1 preemption with skip_save_restore which requires > the following userspace patches to function: > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544 > > A flag is added to `msm_gem_submit` to only allow submissions from compatible > userspace to be preempted, therefore maintaining compatibility. I guess this last para is from an earlier iteration of this series? Looks like instead you are making this a submitqueue flag (which is an approach that I prefer) BR, -R > Some commits from this series are based on a previous series to enable > preemption on A6XX targets: > > https://lkml.kernel.org/1520489185-21828-1-git-send-email-smasetty@codeaurora.org > > Signed-off-by: Antonino Maniscalco <antomani103@gmail.com> > --- > Antonino Maniscalco (7): > drm/msm: Fix bv_fence being used as bv_rptr > drm/msm: Add submitqueue setup and close > drm/msm: Add a `preempt_record_size` field > drm/msm/A6xx: Implement preemption for A7XX targets > drm/msm/A6xx: Add traces for preemption > drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create > drm/msm/A6xx: Enable preemption for A7xx targets > > drivers/gpu/drm/msm/Makefile | 1 + > drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 3 + > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 339 ++++++++++++++++++++++- > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 168 ++++++++++++ > drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 441 ++++++++++++++++++++++++++++++ > drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + > drivers/gpu/drm/msm/msm_gpu.h | 7 + > drivers/gpu/drm/msm/msm_gpu_trace.h | 28 ++ > drivers/gpu/drm/msm/msm_ringbuffer.h | 8 + > drivers/gpu/drm/msm/msm_submitqueue.c | 10 + > include/uapi/drm/msm_drm.h | 5 +- > 11 files changed, 995 insertions(+), 16 deletions(-) > --- > base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba > change-id: 20240815-preemption-a750-t-fcee9a844b39 > > Best regards, > -- > Antonino Maniscalco <antomani103@gmail.com> >
On 8/16/24 7:47 PM, Rob Clark wrote: > On Thu, Aug 15, 2024 at 11:27 AM Antonino Maniscalco > <antomani103@gmail.com> wrote: >> >> This series implements preemption for A7XX targets, which allows the GPU to >> switch to an higher priority ring when work is pushed to it, reducing latency >> for high priority submissions. >> >> This series enables L1 preemption with skip_save_restore which requires >> the following userspace patches to function: >> >> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544 >> >> A flag is added to `msm_gem_submit` to only allow submissions from compatible >> userspace to be preempted, therefore maintaining compatibility. > > I guess this last para is from an earlier iteration of this series? > Looks like instead you are making this a submitqueue flag (which is an > approach that I prefer) > > BR, > -R > That is correct, I got confused on which one I ended up going with when I wrote that. Thanks for spotting it! >> Some commits from this series are based on a previous series to enable >> preemption on A6XX targets: >> >> https://lkml.kernel.org/1520489185-21828-1-git-send-email-smasetty@codeaurora.org >> >> Signed-off-by: Antonino Maniscalco <antomani103@gmail.com> >> --- >> Antonino Maniscalco (7): >> drm/msm: Fix bv_fence being used as bv_rptr >> drm/msm: Add submitqueue setup and close >> drm/msm: Add a `preempt_record_size` field >> drm/msm/A6xx: Implement preemption for A7XX targets >> drm/msm/A6xx: Add traces for preemption >> drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create >> drm/msm/A6xx: Enable preemption for A7xx targets >> >> drivers/gpu/drm/msm/Makefile | 1 + >> drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 3 + >> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 339 ++++++++++++++++++++++- >> drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 168 ++++++++++++ >> drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 441 ++++++++++++++++++++++++++++++ >> drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + >> drivers/gpu/drm/msm/msm_gpu.h | 7 + >> drivers/gpu/drm/msm/msm_gpu_trace.h | 28 ++ >> drivers/gpu/drm/msm/msm_ringbuffer.h | 8 + >> drivers/gpu/drm/msm/msm_submitqueue.c | 10 + >> include/uapi/drm/msm_drm.h | 5 +- >> 11 files changed, 995 insertions(+), 16 deletions(-) >> --- >> base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba >> change-id: 20240815-preemption-a750-t-fcee9a844b39 >> >> Best regards, >> -- >> Antonino Maniscalco <antomani103@gmail.com> >> Best regards,
On 15/08/2024 20:26, Antonino Maniscalco wrote: > This series implements preemption for A7XX targets, which allows the GPU to > switch to an higher priority ring when work is pushed to it, reducing latency > for high priority submissions. > > This series enables L1 preemption with skip_save_restore which requires > the following userspace patches to function: > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544 > > A flag is added to `msm_gem_submit` to only allow submissions from compatible > userspace to be preempted, therefore maintaining compatibility. > > Some commits from this series are based on a previous series to enable > preemption on A6XX targets: > > https://lkml.kernel.org/1520489185-21828-1-git-send-email-smasetty@codeaurora.org > > Signed-off-by: Antonino Maniscalco <antomani103@gmail.com> > --- > Antonino Maniscalco (7): > drm/msm: Fix bv_fence being used as bv_rptr > drm/msm: Add submitqueue setup and close > drm/msm: Add a `preempt_record_size` field > drm/msm/A6xx: Implement preemption for A7XX targets > drm/msm/A6xx: Add traces for preemption > drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create > drm/msm/A6xx: Enable preemption for A7xx targets > > drivers/gpu/drm/msm/Makefile | 1 + > drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 3 + > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 339 ++++++++++++++++++++++- > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 168 ++++++++++++ > drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 441 ++++++++++++++++++++++++++++++ > drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + > drivers/gpu/drm/msm/msm_gpu.h | 7 + > drivers/gpu/drm/msm/msm_gpu_trace.h | 28 ++ > drivers/gpu/drm/msm/msm_ringbuffer.h | 8 + > drivers/gpu/drm/msm/msm_submitqueue.c | 10 + > include/uapi/drm/msm_drm.h | 5 +- > 11 files changed, 995 insertions(+), 16 deletions(-) > --- > base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba > change-id: 20240815-preemption-a750-t-fcee9a844b39 > > Best regards, For what is worth, I've tested it on the SM8650 QRD with the Mesa 30544 MR & vkcube Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-QRD If you think of more tests to run, please tell me. Neil
On Fri, Aug 23, 2024 at 9:30 AM <neil.armstrong@linaro.org> wrote: > > On 15/08/2024 20:26, Antonino Maniscalco wrote: > > This series implements preemption for A7XX targets, which allows the GPU to > > switch to an higher priority ring when work is pushed to it, reducing latency > > for high priority submissions. > > > > This series enables L1 preemption with skip_save_restore which requires > > the following userspace patches to function: > > > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544 > > > > A flag is added to `msm_gem_submit` to only allow submissions from compatible > > userspace to be preempted, therefore maintaining compatibility. > > > > Some commits from this series are based on a previous series to enable > > preemption on A6XX targets: > > > > https://lkml.kernel.org/1520489185-21828-1-git-send-email-smasetty@codeaurora.org > > > > Signed-off-by: Antonino Maniscalco <antomani103@gmail.com> > > --- > > Antonino Maniscalco (7): > > drm/msm: Fix bv_fence being used as bv_rptr > > drm/msm: Add submitqueue setup and close > > drm/msm: Add a `preempt_record_size` field > > drm/msm/A6xx: Implement preemption for A7XX targets > > drm/msm/A6xx: Add traces for preemption > > drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create > > drm/msm/A6xx: Enable preemption for A7xx targets > > > > drivers/gpu/drm/msm/Makefile | 1 + > > drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 3 + > > drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 339 ++++++++++++++++++++++- > > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 168 ++++++++++++ > > drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 441 ++++++++++++++++++++++++++++++ > > drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + > > drivers/gpu/drm/msm/msm_gpu.h | 7 + > > drivers/gpu/drm/msm/msm_gpu_trace.h | 28 ++ > > drivers/gpu/drm/msm/msm_ringbuffer.h | 8 + > > drivers/gpu/drm/msm/msm_submitqueue.c | 10 + > > include/uapi/drm/msm_drm.h | 5 +- > > 11 files changed, 995 insertions(+), 16 deletions(-) > > --- > > base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba > > change-id: 20240815-preemption-a750-t-fcee9a844b39 > > > > Best regards, > > For what is worth, I've tested it on the SM8650 QRD with the Mesa 30544 MR & vkcube > > Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-QRD > > If you think of more tests to run, please tell me. > > Neil Hi Neil, I think it would help to test this on SM8550 and SM8450 too. I don't have SM8450 to test with. Maybe also worth mentioning that there are now vulkan CTS tests that try to test L1 preemption: https://github.com/KhronosGroup/VK-GL-CTS/commit/7e0e4a000f34e748bb527b39f730f78b595140b9 although it's not in a released version yet. Connor
On 23/08/2024 11:54, Connor Abbott wrote: > On Fri, Aug 23, 2024 at 9:30 AM <neil.armstrong@linaro.org> wrote: >> >> On 15/08/2024 20:26, Antonino Maniscalco wrote: >>> This series implements preemption for A7XX targets, which allows the GPU to >>> switch to an higher priority ring when work is pushed to it, reducing latency >>> for high priority submissions. >>> >>> This series enables L1 preemption with skip_save_restore which requires >>> the following userspace patches to function: >>> >>> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544 >>> >>> A flag is added to `msm_gem_submit` to only allow submissions from compatible >>> userspace to be preempted, therefore maintaining compatibility. >>> >>> Some commits from this series are based on a previous series to enable >>> preemption on A6XX targets: >>> >>> https://lkml.kernel.org/1520489185-21828-1-git-send-email-smasetty@codeaurora.org >>> >>> Signed-off-by: Antonino Maniscalco <antomani103@gmail.com> >>> --- >>> Antonino Maniscalco (7): >>> drm/msm: Fix bv_fence being used as bv_rptr >>> drm/msm: Add submitqueue setup and close >>> drm/msm: Add a `preempt_record_size` field >>> drm/msm/A6xx: Implement preemption for A7XX targets >>> drm/msm/A6xx: Add traces for preemption >>> drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create >>> drm/msm/A6xx: Enable preemption for A7xx targets >>> >>> drivers/gpu/drm/msm/Makefile | 1 + >>> drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 3 + >>> drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 339 ++++++++++++++++++++++- >>> drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 168 ++++++++++++ >>> drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 441 ++++++++++++++++++++++++++++++ >>> drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + >>> drivers/gpu/drm/msm/msm_gpu.h | 7 + >>> drivers/gpu/drm/msm/msm_gpu_trace.h | 28 ++ >>> drivers/gpu/drm/msm/msm_ringbuffer.h | 8 + >>> drivers/gpu/drm/msm/msm_submitqueue.c | 10 + >>> include/uapi/drm/msm_drm.h | 5 +- >>> 11 files changed, 995 insertions(+), 16 deletions(-) >>> --- >>> base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba >>> change-id: 20240815-preemption-a750-t-fcee9a844b39 >>> >>> Best regards, >> >> For what is worth, I've tested it on the SM8650 QRD with the Mesa 30544 MR & vkcube >> >> Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-QRD >> >> If you think of more tests to run, please tell me. >> >> Neil > > Hi Neil, > > I think it would help to test this on SM8550 and SM8450 too. I don't > have SM8450 to test with. Maybe also worth mentioning that there are > now vulkan CTS tests that try to test L1 preemption: > https://github.com/KhronosGroup/VK-GL-CTS/commit/7e0e4a000f34e748bb527b39f730f78b595140b9 > although it's not in a released version yet. Sure, It's in my plans, I'm currently testing it on SM8550. Thanks, Neil > > Connor
This series implements preemption for A7XX targets, which allows the GPU to switch to an higher priority ring when work is pushed to it, reducing latency for high priority submissions. This series enables L1 preemption with skip_save_restore which requires the following userspace patches to function: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30544 A flag is added to `msm_gem_submit` to only allow submissions from compatible userspace to be preempted, therefore maintaining compatibility. Some commits from this series are based on a previous series to enable preemption on A6XX targets: https://lkml.kernel.org/1520489185-21828-1-git-send-email-smasetty@codeaurora.org Signed-off-by: Antonino Maniscalco <antomani103@gmail.com> --- Antonino Maniscalco (7): drm/msm: Fix bv_fence being used as bv_rptr drm/msm: Add submitqueue setup and close drm/msm: Add a `preempt_record_size` field drm/msm/A6xx: Implement preemption for A7XX targets drm/msm/A6xx: Add traces for preemption drm/msm/A6XX: Add a flag to allow preemption to submitqueue_create drm/msm/A6xx: Enable preemption for A7xx targets drivers/gpu/drm/msm/Makefile | 1 + drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 3 + drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 339 ++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 168 ++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 441 ++++++++++++++++++++++++++++++ drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + drivers/gpu/drm/msm/msm_gpu.h | 7 + drivers/gpu/drm/msm/msm_gpu_trace.h | 28 ++ drivers/gpu/drm/msm/msm_ringbuffer.h | 8 + drivers/gpu/drm/msm/msm_submitqueue.c | 10 + include/uapi/drm/msm_drm.h | 5 +- 11 files changed, 995 insertions(+), 16 deletions(-) --- base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba change-id: 20240815-preemption-a750-t-fcee9a844b39 Best regards,