mbox series

[v2,00/11] arm64: qcom: add and enable SHM Bridge support

Message ID 20230928092040.9420-1-brgl@bgdev.pl
Headers show
Series arm64: qcom: add and enable SHM Bridge support | expand

Message

Bartosz Golaszewski Sept. 28, 2023, 9:20 a.m. UTC
From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>

This is technically the second iteration of the SHM Bridge enablement on
QCom platforms but in practice it's a complete rewrite.

During the internal discussion with QCom the requirement has been established
as an SHM Bridge pool manager with the assumption that there will be multiple
users, each with their own pool. The problem with this is that we don't have
many potential users of SHM pools upstream at the moment which was rightfully
pointed out in the reviews under v1 (which even had some unused symbols etc.).

Moreover: after some investigating, it turns out that there simply isn't any
need for multiple pools as the core SCM only allocates a buffer if given call
requires more than 4 arguments and there are only a handful of SCM calls that
need to pass a physical address to a buffer as argument to the trustzone.

Additionally all but one SCM call allocate buffers passed to the TZ only for
the duration of the call and then free it right aftr it returns. The one
exception is called once and the buffer it uses can remain in memory until
released by the user.

This all makes using multiple pools wasteful as most of that memory will be
reserved but remain unused 99% of the time. What we need is a single pool of
SCM memory that deals out chunks of suitable format (coherent and
page-aligned) that fulfills the requirements of all calls.

As not all architectures support SHM bridge, it makes sense to first unify the
memory operations in SCM calls. All users do some kind of DMA mapping to obtain
buffers, most using dma_alloc_coherent() which impacts performance.

Genalloc pools are very fast so let's use them instead. Create a custom
allocator that also deals with the mapping and use it across all SCM calls.

Then once this is done, let's extend the allocator to use the SHM bridge
functionality if available on the given platform.

While at it: let's create a qcom specific directory in drivers/firmware/ and
move relevant code in there.

I couldn't test all SCM calls but tested with the inline crypto engine on
sm8550 and sa8775p that runs most of new code paths (with and without SHM
bridge). At least qseecom would need some Tested-by.

v1 -> v2:
- too many changes to list, it's a complete rewrite as explained above

Bartosz Golaszewski (11):
  firmware: qcom: move Qualcomm code into its own directory
  firmware: qcom: scm: add a dedicated SCM memory allocator
  firmware: qcom: scm: switch to using the SCM allocator
  firmware: qcom: scm: make qcom_scm_assign_mem() use the SCM allocator
  firmware: qcom: scm: make qcom_scm_ice_set_key() use the SCM allocator
  firmware: qcom: scm: make qcom_scm_pas_init_image() use the SCM
    allocator
  firmware: qcom: scm: make qcom_scm_lmh_dcvsh() use the SCM allocator
  firmware: qcom: scm: make qcom_scm_qseecom_app_get_id() use the SCM
    allocator
  firmware: qcom: qseecom: convert to using the SCM allocator
  firmware: qcom-scm: add support for SHM bridge operations
  firmware: qcom: scm: enable SHM bridge

 MAINTAINERS                                   |   4 +-
 drivers/firmware/Kconfig                      |  48 +---
 drivers/firmware/Makefile                     |   5 +-
 drivers/firmware/qcom/Kconfig                 |  56 ++++
 drivers/firmware/qcom/Makefile                |   9 +
 drivers/firmware/{ => qcom}/qcom_qseecom.c    |   0
 .../{ => qcom}/qcom_qseecom_uefisecapp.c      | 251 ++++++------------
 drivers/firmware/{ => qcom}/qcom_scm-legacy.c |   0
 drivers/firmware/qcom/qcom_scm-mem.c          | 170 ++++++++++++
 drivers/firmware/{ => qcom}/qcom_scm-smc.c    |  21 +-
 drivers/firmware/{ => qcom}/qcom_scm.c        | 149 ++++++-----
 drivers/firmware/{ => qcom}/qcom_scm.h        |   9 +
 include/linux/firmware/qcom/qcom_qseecom.h    |   4 +-
 include/linux/firmware/qcom/qcom_scm.h        |  13 +
 14 files changed, 427 insertions(+), 312 deletions(-)
 create mode 100644 drivers/firmware/qcom/Kconfig
 create mode 100644 drivers/firmware/qcom/Makefile
 rename drivers/firmware/{ => qcom}/qcom_qseecom.c (100%)
 rename drivers/firmware/{ => qcom}/qcom_qseecom_uefisecapp.c (84%)
 rename drivers/firmware/{ => qcom}/qcom_scm-legacy.c (100%)
 create mode 100644 drivers/firmware/qcom/qcom_scm-mem.c
 rename drivers/firmware/{ => qcom}/qcom_scm-smc.c (92%)
 rename drivers/firmware/{ => qcom}/qcom_scm.c (94%)
 rename drivers/firmware/{ => qcom}/qcom_scm.h (95%)

Comments

Elliot Berman Sept. 28, 2023, 5:08 p.m. UTC | #1
On 9/28/2023 2:20 AM, Bartosz Golaszewski wrote:
> From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
> 
> We're getting more and more qcom specific .c files in drivers/firmware/
> and about to get even more. Create a separate directory for Qualcomm
> firmware drivers and move existing sources in there.
> 
> Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>

Acked-by: Elliot Berman <quic_eberman@quicinc.com>

> ---
>  MAINTAINERS                                   |  4 +-
>  drivers/firmware/Kconfig                      | 48 +---------------
>  drivers/firmware/Makefile                     |  5 +-
>  drivers/firmware/qcom/Kconfig                 | 56 +++++++++++++++++++
>  drivers/firmware/qcom/Makefile                |  9 +++
>  drivers/firmware/{ => qcom}/qcom_qseecom.c    |  0
>  .../{ => qcom}/qcom_qseecom_uefisecapp.c      |  0
>  drivers/firmware/{ => qcom}/qcom_scm-legacy.c |  0
>  drivers/firmware/{ => qcom}/qcom_scm-smc.c    |  0
>  drivers/firmware/{ => qcom}/qcom_scm.c        |  0
>  drivers/firmware/{ => qcom}/qcom_scm.h        |  0
>  11 files changed, 69 insertions(+), 53 deletions(-)
>  create mode 100644 drivers/firmware/qcom/Kconfig
>  create mode 100644 drivers/firmware/qcom/Makefile
>  rename drivers/firmware/{ => qcom}/qcom_qseecom.c (100%)
>  rename drivers/firmware/{ => qcom}/qcom_qseecom_uefisecapp.c (100%)
>  rename drivers/firmware/{ => qcom}/qcom_scm-legacy.c (100%)
>  rename drivers/firmware/{ => qcom}/qcom_scm-smc.c (100%)
>  rename drivers/firmware/{ => qcom}/qcom_scm.c (100%)
>  rename drivers/firmware/{ => qcom}/qcom_scm.h (100%)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 861a16b4586c..88c2186b4975 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -17887,13 +17887,13 @@ QUALCOMM QSEECOM DRIVER
>  M:	Maximilian Luz <luzmaximilian@gmail.com>
>  L:	linux-arm-msm@vger.kernel.org
>  S:	Maintained
> -F:	drivers/firmware/qcom_qseecom.c
> +F:	drivers/firmware/qcom/qcom_qseecom.c
>  
>  QUALCOMM QSEECOM UEFISECAPP DRIVER
>  M:	Maximilian Luz <luzmaximilian@gmail.com>
>  L:	linux-arm-msm@vger.kernel.org
>  S:	Maintained
> -F:	drivers/firmware/qcom_qseecom_uefisecapp.c
> +F:	drivers/firmware/qcom/qcom_qseecom_uefisecapp.c
>  
>  QUALCOMM RMNET DRIVER
>  M:	Subash Abhinov Kasiviswanathan <quic_subashab@quicinc.com>
> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
> index 817e011a8945..74d00b0c83fe 100644
> --- a/drivers/firmware/Kconfig
> +++ b/drivers/firmware/Kconfig
> @@ -188,53 +188,6 @@ config MTK_ADSP_IPC
>  	  ADSP exists on some mtk processors.
>  	  Client might use shared memory to exchange information with ADSP.
>  
> -config QCOM_SCM
> -	tristate
> -
> -config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
> -	bool "Qualcomm download mode enabled by default"
> -	depends on QCOM_SCM
> -	help
> -	  A device with "download mode" enabled will upon an unexpected
> -	  warm-restart enter a special debug mode that allows the user to
> -	  "download" memory content over USB for offline postmortem analysis.
> -	  The feature can be enabled/disabled on the kernel command line.
> -
> -	  Say Y here to enable "download mode" by default.
> -
> -config QCOM_QSEECOM
> -	bool "Qualcomm QSEECOM interface driver"
> -	depends on QCOM_SCM=y
> -	select AUXILIARY_BUS
> -	help
> -	  Various Qualcomm SoCs have a Secure Execution Environment (SEE) running
> -	  in the Trust Zone. This module provides an interface to that via the
> -	  QSEECOM mechanism, using SCM calls.
> -
> -	  The QSEECOM interface allows, among other things, access to applications
> -	  running in the SEE. An example of such an application is 'uefisecapp',
> -	  which is required to access UEFI variables on certain systems. If
> -	  selected, the interface will also attempt to detect and register client
> -	  devices for supported applications.
> -
> -	  Select Y here to enable the QSEECOM interface driver.
> -
> -config QCOM_QSEECOM_UEFISECAPP
> -	bool "Qualcomm SEE UEFI Secure App client driver"
> -	depends on QCOM_QSEECOM
> -	depends on EFI
> -	help
> -	  Various Qualcomm SoCs do not allow direct access to EFI variables.
> -	  Instead, these need to be accessed via the UEFI Secure Application
> -	  (uefisecapp), residing in the Secure Execution Environment (SEE).
> -
> -	  This module provides a client driver for uefisecapp, installing efivar
> -	  operations to allow the kernel accessing EFI variables, and via that also
> -	  provide user-space with access to EFI variables via efivarfs.
> -
> -	  Select Y here to provide access to EFI variables on the aforementioned
> -	  platforms.
> -
>  config SYSFB
>  	bool
>  	select BOOT_VESA_SUPPORT
> @@ -320,6 +273,7 @@ source "drivers/firmware/efi/Kconfig"
>  source "drivers/firmware/imx/Kconfig"
>  source "drivers/firmware/meson/Kconfig"
>  source "drivers/firmware/psci/Kconfig"
> +source "drivers/firmware/qcom/Kconfig"
>  source "drivers/firmware/smccc/Kconfig"
>  source "drivers/firmware/tegra/Kconfig"
>  source "drivers/firmware/xilinx/Kconfig"
> diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
> index cb18fd8882dc..5f9dab82e1a0 100644
> --- a/drivers/firmware/Makefile
> +++ b/drivers/firmware/Makefile
> @@ -17,10 +17,6 @@ obj-$(CONFIG_FIRMWARE_MEMMAP)	+= memmap.o
>  obj-$(CONFIG_MTK_ADSP_IPC)	+= mtk-adsp-ipc.o
>  obj-$(CONFIG_RASPBERRYPI_FIRMWARE) += raspberrypi.o
>  obj-$(CONFIG_FW_CFG_SYSFS)	+= qemu_fw_cfg.o
> -obj-$(CONFIG_QCOM_SCM)		+= qcom-scm.o
> -qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
> -obj-$(CONFIG_QCOM_QSEECOM)	+= qcom_qseecom.o
> -obj-$(CONFIG_QCOM_QSEECOM_UEFISECAPP) += qcom_qseecom_uefisecapp.o
>  obj-$(CONFIG_SYSFB)		+= sysfb.o
>  obj-$(CONFIG_SYSFB_SIMPLEFB)	+= sysfb_simplefb.o
>  obj-$(CONFIG_TI_SCI_PROTOCOL)	+= ti_sci.o
> @@ -36,6 +32,7 @@ obj-$(CONFIG_GOOGLE_FIRMWARE)	+= google/
>  obj-y				+= efi/
>  obj-y				+= imx/
>  obj-y				+= psci/
> +obj-y				+= qcom/
>  obj-y				+= smccc/
>  obj-y				+= tegra/
>  obj-y				+= xilinx/
> diff --git a/drivers/firmware/qcom/Kconfig b/drivers/firmware/qcom/Kconfig
> new file mode 100644
> index 000000000000..3f05d9854ddf
> --- /dev/null
> +++ b/drivers/firmware/qcom/Kconfig
> @@ -0,0 +1,56 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +# For a description of the syntax of this configuration file,
> +# see Documentation/kbuild/kconfig-language.rst.
> +#
> +
> +menu "Qualcomm firmware drivers"
> +
> +config QCOM_SCM
> +	tristate
> +
> +config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
> +	bool "Qualcomm download mode enabled by default"
> +	depends on QCOM_SCM
> +	help
> +	  A device with "download mode" enabled will upon an unexpected
> +	  warm-restart enter a special debug mode that allows the user to
> +	  "download" memory content over USB for offline postmortem analysis.
> +	  The feature can be enabled/disabled on the kernel command line.
> +
> +	  Say Y here to enable "download mode" by default.
> +
> +config QCOM_QSEECOM
> +	bool "Qualcomm QSEECOM interface driver"
> +	depends on QCOM_SCM=y
> +	select AUXILIARY_BUS
> +	help
> +	  Various Qualcomm SoCs have a Secure Execution Environment (SEE) running
> +	  in the Trust Zone. This module provides an interface to that via the
> +	  QSEECOM mechanism, using SCM calls.
> +
> +	  The QSEECOM interface allows, among other things, access to applications
> +	  running in the SEE. An example of such an application is 'uefisecapp',
> +	  which is required to access UEFI variables on certain systems. If
> +	  selected, the interface will also attempt to detect and register client
> +	  devices for supported applications.
> +
> +	  Select Y here to enable the QSEECOM interface driver.
> +
> +config QCOM_QSEECOM_UEFISECAPP
> +	bool "Qualcomm SEE UEFI Secure App client driver"
> +	depends on QCOM_QSEECOM
> +	depends on EFI
> +	help
> +	  Various Qualcomm SoCs do not allow direct access to EFI variables.
> +	  Instead, these need to be accessed via the UEFI Secure Application
> +	  (uefisecapp), residing in the Secure Execution Environment (SEE).
> +
> +	  This module provides a client driver for uefisecapp, installing efivar
> +	  operations to allow the kernel accessing EFI variables, and via that also
> +	  provide user-space with access to EFI variables via efivarfs.
> +
> +	  Select Y here to provide access to EFI variables on the aforementioned
> +	  platforms.
> +
> +endmenu
> diff --git a/drivers/firmware/qcom/Makefile b/drivers/firmware/qcom/Makefile
> new file mode 100644
> index 000000000000..c9f12ee8224a
> --- /dev/null
> +++ b/drivers/firmware/qcom/Makefile
> @@ -0,0 +1,9 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for the linux kernel.
> +#
> +
> +obj-$(CONFIG_QCOM_SCM)		+= qcom-scm.o
> +qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
> +obj-$(CONFIG_QCOM_QSEECOM)	+= qcom_qseecom.o
> +obj-$(CONFIG_QCOM_QSEECOM_UEFISECAPP) += qcom_qseecom_uefisecapp.o
> diff --git a/drivers/firmware/qcom_qseecom.c b/drivers/firmware/qcom/qcom_qseecom.c
> similarity index 100%
> rename from drivers/firmware/qcom_qseecom.c
> rename to drivers/firmware/qcom/qcom_qseecom.c
> diff --git a/drivers/firmware/qcom_qseecom_uefisecapp.c b/drivers/firmware/qcom/qcom_qseecom_uefisecapp.c
> similarity index 100%
> rename from drivers/firmware/qcom_qseecom_uefisecapp.c
> rename to drivers/firmware/qcom/qcom_qseecom_uefisecapp.c
> diff --git a/drivers/firmware/qcom_scm-legacy.c b/drivers/firmware/qcom/qcom_scm-legacy.c
> similarity index 100%
> rename from drivers/firmware/qcom_scm-legacy.c
> rename to drivers/firmware/qcom/qcom_scm-legacy.c
> diff --git a/drivers/firmware/qcom_scm-smc.c b/drivers/firmware/qcom/qcom_scm-smc.c
> similarity index 100%
> rename from drivers/firmware/qcom_scm-smc.c
> rename to drivers/firmware/qcom/qcom_scm-smc.c
> diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom/qcom_scm.c
> similarity index 100%
> rename from drivers/firmware/qcom_scm.c
> rename to drivers/firmware/qcom/qcom_scm.c
> diff --git a/drivers/firmware/qcom_scm.h b/drivers/firmware/qcom/qcom_scm.h
> similarity index 100%
> rename from drivers/firmware/qcom_scm.h
> rename to drivers/firmware/qcom/qcom_scm.h
Andrew Halaney Sept. 29, 2023, 3:29 p.m. UTC | #2
On Thu, Sep 28, 2023 at 11:20:29AM +0200, Bartosz Golaszewski wrote:
> From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
>
> This is technically the second iteration of the SHM Bridge enablement on
> QCom platforms but in practice it's a complete rewrite.
>
> During the internal discussion with QCom the requirement has been established
> as an SHM Bridge pool manager with the assumption that there will be multiple
> users, each with their own pool. The problem with this is that we don't have
> many potential users of SHM pools upstream at the moment which was rightfully
> pointed out in the reviews under v1 (which even had some unused symbols etc.).
>
> Moreover: after some investigating, it turns out that there simply isn't any
> need for multiple pools as the core SCM only allocates a buffer if given call
> requires more than 4 arguments and there are only a handful of SCM calls that
> need to pass a physical address to a buffer as argument to the trustzone.
>
> Additionally all but one SCM call allocate buffers passed to the TZ only for
> the duration of the call and then free it right aftr it returns. The one
> exception is called once and the buffer it uses can remain in memory until
> released by the user.
>
> This all makes using multiple pools wasteful as most of that memory will be
> reserved but remain unused 99% of the time. What we need is a single pool of
> SCM memory that deals out chunks of suitable format (coherent and
> page-aligned) that fulfills the requirements of all calls.
>
> As not all architectures support SHM bridge, it makes sense to first unify the
> memory operations in SCM calls. All users do some kind of DMA mapping to obtain
> buffers, most using dma_alloc_coherent() which impacts performance.
>
> Genalloc pools are very fast so let's use them instead. Create a custom
> allocator that also deals with the mapping and use it across all SCM calls.
>
> Then once this is done, let's extend the allocator to use the SHM bridge
> functionality if available on the given platform.
>
> While at it: let's create a qcom specific directory in drivers/firmware/ and
> move relevant code in there.
>
> I couldn't test all SCM calls but tested with the inline crypto engine on
> sm8550 and sa8775p that runs most of new code paths (with and without SHM
> bridge). At least qseecom would need some Tested-by.

I have been playing around with this on my x13s (sc8280xp). It seems
that efivars works ok (and therefore the qseecom stuff I believe), but
with these patches applied I'm getting -22 when loading any firmware.

I'll try to dig a little more, but thought I'd report that before I
context switch for a little bit.

    dmesg | grep "firmware\|-22"
    [    0.000000] psci: PSCIv1.1 detected in firmware.
    [    2.351999] qcom_scm firmware:scm: SHM Bridge enabled
    [    2.353002] qcom_scm firmware:scm: qseecom: found qseecom with version 0x1402000
    [    6.727604] systemd[1]: systemd-pcrmachine.service - TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
    [    8.198381] qcom_q6v5_pas 1b300000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qccdsp8280.mbn
    [    8.198418] remoteproc remoteproc1: can't start rproc 1b300000.remoteproc: -22
    [    8.407641] qcom_q6v5_pas 3000000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcadsp8280.mbn
    [    8.407742] remoteproc remoteproc0: can't start rproc 3000000.remoteproc: -22
    [    8.588496] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [    8.588509] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [    9.392185] ath11k_pci 0006:01:00.0: fw_version 0x110b196e fw_build_timestamp 2022-12-22 12:54 fw_build_id WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.23
    [   10.229367] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   10.229383] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.041385] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.041399] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.070144] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.070160] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.321999] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.322015] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.340989] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.341002] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.576180] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.576198] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.593647] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.593661] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.854212] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.854226] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.879925] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.879937] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   12.157090] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   12.157106] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   12.183074] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   12.183088] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22

Thanks,
Andrew
>
> v1 -> v2:
> - too many changes to list, it's a complete rewrite as explained above
>
> Bartosz Golaszewski (11):
>   firmware: qcom: move Qualcomm code into its own directory
>   firmware: qcom: scm: add a dedicated SCM memory allocator
>   firmware: qcom: scm: switch to using the SCM allocator
>   firmware: qcom: scm: make qcom_scm_assign_mem() use the SCM allocator
>   firmware: qcom: scm: make qcom_scm_ice_set_key() use the SCM allocator
>   firmware: qcom: scm: make qcom_scm_pas_init_image() use the SCM
>     allocator
>   firmware: qcom: scm: make qcom_scm_lmh_dcvsh() use the SCM allocator
>   firmware: qcom: scm: make qcom_scm_qseecom_app_get_id() use the SCM
>     allocator
>   firmware: qcom: qseecom: convert to using the SCM allocator
>   firmware: qcom-scm: add support for SHM bridge operations
>   firmware: qcom: scm: enable SHM bridge
>
>  MAINTAINERS                                   |   4 +-
>  drivers/firmware/Kconfig                      |  48 +---
>  drivers/firmware/Makefile                     |   5 +-
>  drivers/firmware/qcom/Kconfig                 |  56 ++++
>  drivers/firmware/qcom/Makefile                |   9 +
>  drivers/firmware/{ => qcom}/qcom_qseecom.c    |   0
>  .../{ => qcom}/qcom_qseecom_uefisecapp.c      | 251 ++++++------------
>  drivers/firmware/{ => qcom}/qcom_scm-legacy.c |   0
>  drivers/firmware/qcom/qcom_scm-mem.c          | 170 ++++++++++++
>  drivers/firmware/{ => qcom}/qcom_scm-smc.c    |  21 +-
>  drivers/firmware/{ => qcom}/qcom_scm.c        | 149 ++++++-----
>  drivers/firmware/{ => qcom}/qcom_scm.h        |   9 +
>  include/linux/firmware/qcom/qcom_qseecom.h    |   4 +-
>  include/linux/firmware/qcom/qcom_scm.h        |  13 +
>  14 files changed, 427 insertions(+), 312 deletions(-)
>  create mode 100644 drivers/firmware/qcom/Kconfig
>  create mode 100644 drivers/firmware/qcom/Makefile
>  rename drivers/firmware/{ => qcom}/qcom_qseecom.c (100%)
>  rename drivers/firmware/{ => qcom}/qcom_qseecom_uefisecapp.c (84%)
>  rename drivers/firmware/{ => qcom}/qcom_scm-legacy.c (100%)
>  create mode 100644 drivers/firmware/qcom/qcom_scm-mem.c
>  rename drivers/firmware/{ => qcom}/qcom_scm-smc.c (92%)
>  rename drivers/firmware/{ => qcom}/qcom_scm.c (94%)
>  rename drivers/firmware/{ => qcom}/qcom_scm.h (95%)
>
> --
> 2.39.2
>
Andrew Halaney Sept. 29, 2023, 7:18 p.m. UTC | #3
On Fri, Sep 29, 2023 at 08:56:57PM +0200, Bartosz Golaszewski wrote:
> On Fri, Sep 29, 2023 at 5:29 PM Andrew Halaney <ahalaney@redhat.com> wrote:
> >
> > On Thu, Sep 28, 2023 at 11:20:29AM +0200, Bartosz Golaszewski wrote:
> > > From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
> > >
> > > This is technically the second iteration of the SHM Bridge enablement on
> > > QCom platforms but in practice it's a complete rewrite.
> > >
> > > During the internal discussion with QCom the requirement has been established
> > > as an SHM Bridge pool manager with the assumption that there will be multiple
> > > users, each with their own pool. The problem with this is that we don't have
> > > many potential users of SHM pools upstream at the moment which was rightfully
> > > pointed out in the reviews under v1 (which even had some unused symbols etc.).
> > >
> > > Moreover: after some investigating, it turns out that there simply isn't any
> > > need for multiple pools as the core SCM only allocates a buffer if given call
> > > requires more than 4 arguments and there are only a handful of SCM calls that
> > > need to pass a physical address to a buffer as argument to the trustzone.
> > >
> > > Additionally all but one SCM call allocate buffers passed to the TZ only for
> > > the duration of the call and then free it right aftr it returns. The one
> > > exception is called once and the buffer it uses can remain in memory until
> > > released by the user.
> > >
> > > This all makes using multiple pools wasteful as most of that memory will be
> > > reserved but remain unused 99% of the time. What we need is a single pool of
> > > SCM memory that deals out chunks of suitable format (coherent and
> > > page-aligned) that fulfills the requirements of all calls.
> > >
> > > As not all architectures support SHM bridge, it makes sense to first unify the
> > > memory operations in SCM calls. All users do some kind of DMA mapping to obtain
> > > buffers, most using dma_alloc_coherent() which impacts performance.
> > >
> > > Genalloc pools are very fast so let's use them instead. Create a custom
> > > allocator that also deals with the mapping and use it across all SCM calls.
> > >
> > > Then once this is done, let's extend the allocator to use the SHM bridge
> > > functionality if available on the given platform.
> > >
> > > While at it: let's create a qcom specific directory in drivers/firmware/ and
> > > move relevant code in there.
> > >
> > > I couldn't test all SCM calls but tested with the inline crypto engine on
> > > sm8550 and sa8775p that runs most of new code paths (with and without SHM
> > > bridge). At least qseecom would need some Tested-by.
> >
> > I have been playing around with this on my x13s (sc8280xp). It seems
> > that efivars works ok (and therefore the qseecom stuff I believe), but
> > with these patches applied I'm getting -22 when loading any firmware.
> >
> > I'll try to dig a little more, but thought I'd report that before I
> > context switch for a little bit.
> >
> >     dmesg | grep "firmware\|-22"
> >     [    0.000000] psci: PSCIv1.1 detected in firmware.
> >     [    2.351999] qcom_scm firmware:scm: SHM Bridge enabled
> >     [    2.353002] qcom_scm firmware:scm: qseecom: found qseecom with version 0x1402000
> >     [    6.727604] systemd[1]: systemd-pcrmachine.service - TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
> >     [    8.198381] qcom_q6v5_pas 1b300000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qccdsp8280.mbn
> >     [    8.198418] remoteproc remoteproc1: can't start rproc 1b300000.remoteproc: -22
> >     [    8.407641] qcom_q6v5_pas 3000000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcadsp8280.mbn
> >     [    8.407742] remoteproc remoteproc0: can't start rproc 3000000.remoteproc: -22
> >     [    8.588496] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [    8.588509] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [    9.392185] ath11k_pci 0006:01:00.0: fw_version 0x110b196e fw_build_timestamp 2022-12-22 12:54 fw_build_id WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.23
> >     [   10.229367] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   10.229383] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.041385] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.041399] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.070144] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.070160] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.321999] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.322015] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.340989] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.341002] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.576180] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.576198] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.593647] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.593661] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.854212] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.854226] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.879925] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.879937] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   12.157090] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   12.157106] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   12.183074] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   12.183088] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >
> > Thanks,
> > Andrew
> 
> Huh remoteproc seems to work fine on sm8550. Can you post the full
> kernel log? Do you know which SCM calls fails?
> 
> Bart

See my response on patch 6, and please let me know if you I messed up
the logic there and need to dig a little more. The log didn't have
anything useful outside what is shown here unfortunately, but I can
gather more if you refute that comment on patch 6.

Thanks,
Andrew