From patchwork Mon Dec 18 11:31:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757245 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F5D51BDE1; Mon, 18 Dec 2023 11:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="FrZmTNPP" Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsPSk000317; Mon, 18 Dec 2023 11:33:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=uemSpET3sD+B6z4KBIkwMzg9w2z6sbdfcQ+wwx6z580=; b=Fr ZmTNPPqEeQ0fk1Dx9CSOl3m7BN4zZy5fEqiTGZN0Z/0uBjd48/qUEmDJYG6V8q6K 9mqoHJ75IQCz/QQX+ju4IQ68OrnonYzp9cBZRJSXSjJ6w8tdFmsRvAB/c3bZWrYp H75Yz6C2xCVLNtyLaWrxrQXirD3YW4guBQwIOnN0oH1NkOEt3/V2+70CS6+raXvg CUOi5rPom2vCFqfdD8yhM1Alxy7g1DWLDqVPL7t1uoObiThFg1DBcLaeJIh/LKW9 RCixp3UXAk6NdY82FQZ0voYpzH+A7qvNB+HxoGgNUOEOxDNCLkTu3fhObW+zUxOG Wm6/piFOe2OFmVtxE1Sg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v14xy443u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:09 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5MB029943; Mon, 18 Dec 2023 11:33:05 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyg9-1; Mon, 18 Dec 2023 11:33:05 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Gj029914; Mon, 18 Dec 2023 11:33:05 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX53B029905; Mon, 18 Dec 2023 11:33:05 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 211F0A34; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 01/34] media: introduce common helpers for video firmware handling Date: Mon, 18 Dec 2023 17:01:56 +0530 Message-Id: <1702899149-21321-2-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: x3Oy-UIFpQzs5vsfJ35JZG1ksAM8in_R X-Proofpoint-ORIG-GUID: x3Oy-UIFpQzs5vsfJ35JZG1ksAM8in_R X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Re-organize the video driver code by introducing a new folder 'vcodec' and placing 'venus' driver code inside that. Introduce common helpers for trustzone based firmware load/unload etc. which are placed in common folder i.e. 'vcodec'. Use these helpers in 'venus' driver. These helpers will be used by 'iris' driver as well which is introduced later in this patch series. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/Kconfig | 2 +- drivers/media/platform/qcom/Makefile | 2 +- drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++ drivers/media/platform/qcom/vcodec/firmware.h | 21 ++ .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0 .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +- .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++- .../media/platform/qcom/{ => vcodec}/venus/core.h | 0 .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0 .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0 .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++ .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++ .../platform/qcom/{ => vcodec}/venus/helpers.c | 0 .../platform/qcom/{ => vcodec}/venus/helpers.h | 0 .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0 .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0 .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0 .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0 .../qcom/{ => vcodec}/venus/hfi_platform.c | 0 .../qcom/{ => vcodec}/venus/hfi_platform.h | 0 .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0 .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +- .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0 .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0 .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0 .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0 .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0 .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0 .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0 .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0 .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0 .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0 drivers/media/platform/qcom/venus/firmware.c | 363 --------------------- drivers/media/platform/qcom/venus/firmware.h | 26 -- 42 files changed, 492 insertions(+), 409 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%) rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%) rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%) create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%) delete mode 100644 drivers/media/platform/qcom/venus/firmware.c delete mode 100644 drivers/media/platform/qcom/venus/firmware.h diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig index cc5799b..e94142f 100644 --- a/drivers/media/platform/qcom/Kconfig +++ b/drivers/media/platform/qcom/Kconfig @@ -3,4 +3,4 @@ comment "Qualcomm media platform drivers" source "drivers/media/platform/qcom/camss/Kconfig" -source "drivers/media/platform/qcom/venus/Kconfig" +source "drivers/media/platform/qcom/vcodec/venus/Kconfig" diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile index 4f055c3..3d2d82b 100644 --- a/drivers/media/platform/qcom/Makefile +++ b/drivers/media/platform/qcom/Makefile @@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += camss/ -obj-y += venus/ +obj-y += vcodec/venus/ diff --git a/drivers/media/platform/qcom/vcodec/firmware.c b/drivers/media/platform/qcom/vcodec/firmware.c new file mode 100644 index 0000000..dbc220a --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/firmware.c @@ -0,0 +1,147 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "firmware.h" + +bool use_tz(struct device *core_dev) +{ + struct device_node *np; + + np = of_get_child_by_name(core_dev->of_node, "video-firmware"); + if (!np) + return true; + + return false; +} + +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start, + u32 cp_nonpixel_size, u32 pas_id) +{ + int ret; + /* + * Clues for porting using downstream data: + * cp_start = 0 + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size! + * This works, as the non-secure context bank is placed + * contiguously right after the Content Protection region. + * + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0] + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1] + */ + ret = qcom_scm_mem_protect_video_var(cp_start, + cp_size, + cp_nonpixel_start, + cp_nonpixel_size); + if (ret) + qcom_scm_pas_shutdown(pas_id); + + return ret; +} + +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys, + size_t *mem_size, u32 pas_id, bool use_tz) +{ + const struct firmware *firmware = NULL; + struct reserved_mem *rmem; + struct device_node *node; + void *mem_virt = NULL; + ssize_t fw_size = 0; + int ret; + + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) || + (use_tz && !qcom_scm_is_available())) + return -EPROBE_DEFER; + + if (!fw_name || !(*fw_name)) + return -EINVAL; + + *mem_phys = 0; + *mem_size = 0; + + node = of_parse_phandle(dev->of_node, "memory-region", 0); + if (!node) { + dev_err(dev, "no memory-region specified\n"); + return -EINVAL; + } + + rmem = of_reserved_mem_lookup(node); + of_node_put(node); + if (!rmem) { + dev_err(dev, "failed to lookup reserved memory-region\n"); + return -EINVAL; + } + + ret = request_firmware(&firmware, fw_name, dev); + if (ret) { + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n", + __func__, fw_name, ret); + return ret; + } + + fw_size = qcom_mdt_get_size(firmware); + if (fw_size < 0) { + ret = fw_size; + dev_err(dev, "%s: out of bound fw image fw size: %ld\n", + __func__, fw_size); + goto err_release_fw; + } + + *mem_phys = rmem->base; + *mem_size = rmem->size; + + if (*mem_size < fw_size) { + ret = -EINVAL; + goto err_release_fw; + } + + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC); + if (!mem_virt) { + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n", + mem_phys, *mem_size); + goto err_release_fw; + } + + if (use_tz) + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt, + *mem_phys, *mem_size, NULL); + else + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id, mem_virt, + *mem_phys, *mem_size, NULL); + if (ret) { + dev_err(dev, "%s: error %d loading fw \"%s\"\n", + __func__, ret, fw_name); + } + + memunmap(mem_virt); +err_release_fw: + release_firmware(firmware); + return ret; +} + +int auth_reset_fw(u32 pas_id) +{ + return qcom_scm_pas_auth_and_reset(pas_id); +} + +void unload_fw(u32 pas_id) +{ + qcom_scm_pas_shutdown(pas_id); +} + +int set_hw_state(bool resume) +{ + return qcom_scm_set_remote_state(resume, 0); +} diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h new file mode 100644 index 0000000..7d410a8 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/firmware.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _FIRMWARE_H_ +#define _FIRMWARE_H_ + +#include +#include + +bool use_tz(struct device *core_dev); +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys, + size_t *mem_size, u32 pas_id, bool use_tz); +int auth_reset_fw(u32 pas_id); +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start, + u32 cp_nonpixel_size, u32 pas_id); +void unload_fw(u32 pas_id); +int set_hw_state(bool resume); + +#endif diff --git a/drivers/media/platform/qcom/venus/Kconfig b/drivers/media/platform/qcom/vcodec/venus/Kconfig similarity index 100% rename from drivers/media/platform/qcom/venus/Kconfig rename to drivers/media/platform/qcom/vcodec/venus/Kconfig diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile similarity index 83% rename from drivers/media/platform/qcom/venus/Makefile rename to drivers/media/platform/qcom/vcodec/venus/Makefile index 91ee6be..f6f3a88 100644 --- a/drivers/media/platform/qcom/venus/Makefile +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile @@ -1,7 +1,9 @@ # SPDX-License-Identifier: GPL-2.0 # Makefile for Qualcomm Venus driver -venus-core-objs += core.o helpers.o firmware.o \ +venus-core-objs += ../firmware.o + +venus-core-objs += core.o helpers.o firmware_no_tz.o \ hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \ hfi_parser.o pm_helpers.o dbgfs.o \ hfi_platform.o hfi_platform_v4.o \ diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/vcodec/venus/core.c similarity index 91% rename from drivers/media/platform/qcom/venus/core.c rename to drivers/media/platform/qcom/vcodec/venus/core.c index 9cffe97..56d9a53 100644 --- a/drivers/media/platform/qcom/venus/core.c +++ b/drivers/media/platform/qcom/vcodec/venus/core.c @@ -22,7 +22,8 @@ #include #include "core.h" -#include "firmware.h" +#include "../firmware.h" +#include "firmware_no_tz.h" #include "pm_helpers.h" #include "hfi_venus_io.h" @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct work_struct *work) struct venus_core *core = container_of(work, struct venus_core, work.work); int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS; + const struct venus_resources *res = core->res; + const char *fwpath = NULL; const char *err_msg = ""; bool failed = false; @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct work_struct *work) mutex_lock(&core->lock); - venus_shutdown(core); + if (core->use_tz) + unload_fw(VENUS_PAS_ID); + else + unload_fw_no_tz(core); venus_coredump(core); @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct work_struct *work) failed = true; } - ret = venus_boot(core); + ret = of_property_read_string_index(core->dev->of_node, "firmware-name", 0, + &fwpath); + if (ret) + fwpath = core->res->fwname; + + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size, + VENUS_PAS_ID, core->use_tz); if (ret && !failed) { - err_msg = "boot Venus"; + err_msg = "load FW"; failed = true; } + if (core->use_tz) + ret = auth_reset_fw(VENUS_PAS_ID); + else + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size); + if (ret && !failed) { + err_msg = "Auth and Reset"; + failed = true; + } + + if (core->use_tz && res->cp_size) { + ret = protect_secure_region(res->cp_start, + res->cp_size, + res->cp_nonpixel_start, + res->cp_nonpixel_size, + VENUS_PAS_ID); + if (ret && !failed) { + err_msg = "Protect CP Mem"; + failed = true; + } + } + ret = hfi_core_resume(core, true); if (ret && !failed) { err_msg = "resume HFI"; @@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void *dev_id) static int venus_probe(struct platform_device *pdev) { + const struct venus_resources *res; struct device *dev = &pdev->dev; + const char *fwpath = NULL; struct venus_core *core; int ret; @@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev) if (ret) goto err_runtime_disable; - ret = venus_firmware_init(core); + core->use_tz = use_tz(core->dev); + + if (!core->use_tz) { + ret = init_fw_no_tz(core); + if (ret) + goto err_of_depopulate; + } + + ret = of_property_read_string_index(dev->of_node, "firmware-name", 0, + &fwpath); if (ret) - goto err_of_depopulate; + fwpath = core->res->fwname; - ret = venus_boot(core); + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size, + VENUS_PAS_ID, core->use_tz); if (ret) goto err_firmware_deinit; + if (core->use_tz) + ret = auth_reset_fw(VENUS_PAS_ID); + else + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size); + if (ret) + goto err_firmware_deinit; + + res = core->res; + if (core->use_tz && res->cp_size) { + ret = protect_secure_region(res->cp_start, + res->cp_size, + res->cp_nonpixel_start, + res->cp_nonpixel_size, + VENUS_PAS_ID); + if (ret) + goto err_firmware_deinit; + } + ret = hfi_core_resume(core, true); if (ret) goto err_venus_shutdown; @@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev) err_dev_unregister: v4l2_device_unregister(&core->v4l2_dev); err_venus_shutdown: - venus_shutdown(core); + if (core->use_tz) + unload_fw(VENUS_PAS_ID); + else + unload_fw_no_tz(core); err_firmware_deinit: - venus_firmware_deinit(core); + if (!core->use_tz) + deinit_fw_no_tz(core); err_of_depopulate: of_platform_depopulate(dev); err_runtime_disable: @@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev) ret = hfi_core_deinit(core, true); WARN_ON(ret); - venus_shutdown(core); + if (core->use_tz) + unload_fw(VENUS_PAS_ID); + else + unload_fw_no_tz(core); + of_platform_depopulate(dev); - venus_firmware_deinit(core); + if (!core->use_tz) + deinit_fw_no_tz(core); pm_runtime_put_sync(dev); pm_runtime_disable(dev); @@ -455,8 +527,12 @@ static void venus_core_shutdown(struct platform_device *pdev) struct venus_core *core = platform_get_drvdata(pdev); pm_runtime_get_sync(core->dev); - venus_shutdown(core); - venus_firmware_deinit(core); + if (core->use_tz) { + unload_fw(VENUS_PAS_ID); + } else { + unload_fw_no_tz(core); + deinit_fw_no_tz(core); + } pm_runtime_put_sync(core->dev); } diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/vcodec/venus/core.h similarity index 100% rename from drivers/media/platform/qcom/venus/core.h rename to drivers/media/platform/qcom/vcodec/venus/core.h diff --git a/drivers/media/platform/qcom/venus/dbgfs.c b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c similarity index 100% rename from drivers/media/platform/qcom/venus/dbgfs.c rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c diff --git a/drivers/media/platform/qcom/venus/dbgfs.h b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h similarity index 100% rename from drivers/media/platform/qcom/venus/dbgfs.h rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c new file mode 100644 index 0000000..9dca6e23 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c @@ -0,0 +1,194 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017 Linaro Ltd. + */ + +#include +#include +#include "core.h" +#include "firmware_no_tz.h" +#include "hfi_venus_io.h" + +#define VENUS_FW_MEM_SIZE (6 * SZ_1M) +#define VENUS_FW_START_ADDR 0x0 + +int init_fw_no_tz(struct venus_core *core) +{ + struct platform_device_info info; + struct iommu_domain *iommu_dom; + struct platform_device *pdev; + struct device_node *np; + int ret; + + np = of_get_child_by_name(core->dev->of_node, "video-firmware"); + + memset(&info, 0, sizeof(info)); + info.fwnode = &np->fwnode; + info.parent = core->dev; + info.name = np->name; + info.dma_mask = DMA_BIT_MASK(32); + + pdev = platform_device_register_full(&info); + if (IS_ERR(pdev)) { + of_node_put(np); + return PTR_ERR(pdev); + } + + pdev->dev.of_node = np; + + ret = of_dma_configure(&pdev->dev, np, true); + if (ret) { + dev_err(core->dev, "dma configure fail\n"); + goto err_unregister; + } + + core->fw.dev = &pdev->dev; + + iommu_dom = iommu_domain_alloc(&platform_bus_type); + if (!iommu_dom) { + dev_err(core->fw.dev, "Failed to allocate iommu domain\n"); + ret = -ENOMEM; + goto err_unregister; + } + + ret = iommu_attach_device(iommu_dom, core->fw.dev); + if (ret) { + dev_err(core->fw.dev, "could not attach device\n"); + goto err_iommu_free; + } + + core->fw.iommu_domain = iommu_dom; + + of_node_put(np); + + return 0; + +err_iommu_free: + iommu_domain_free(iommu_dom); +err_unregister: + platform_device_unregister(pdev); + of_node_put(np); + return ret; +} + +void deinit_fw_no_tz(struct venus_core *core) +{ + struct iommu_domain *iommu; + + if (!core->fw.dev) + return; + + iommu = core->fw.iommu_domain; + + iommu_detach_device(iommu, core->fw.dev); + + if (iommu) { + iommu_domain_free(iommu); + iommu = NULL; + } + + platform_device_unregister(to_platform_device(core->fw.dev)); +} + +static void reset_cpu_no_tz(struct venus_core *core) +{ + u32 fw_size = core->fw.mapped_mem_size; + void __iomem *wrapper_base; + + if (IS_IRIS2_1(core)) + wrapper_base = core->wrapper_tz_base; + else + wrapper_base = core->wrapper_base; + + writel(0, wrapper_base + WRAPPER_FW_START_ADDR); + writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR); + writel(0, wrapper_base + WRAPPER_CPA_START_ADDR); + writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR); + writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR); + writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR); + + if (IS_IRIS2_1(core)) { + /* Bring XTSS out of reset */ + writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET); + } else { + writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS); + writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG); + + /* Bring ARM9 out of reset */ + writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET); + } +} + +void set_hw_state_no_tz(struct venus_core *core, bool resume) +{ + if (resume) { + reset_cpu_no_tz(core); + } else { + if (IS_IRIS2_1(core)) + writel(WRAPPER_XTSS_SW_RESET_BIT, + core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); + else + writel(WRAPPER_A9SS_SW_RESET_BIT, + core->wrapper_base + WRAPPER_A9SS_SW_RESET); + } +} + +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys, + size_t mem_size) +{ + struct iommu_domain *iommu; + struct device *dev; + int ret; + + dev = core->fw.dev; + if (!dev) + return -EPROBE_DEFER; + + iommu = core->fw.iommu_domain; + core->fw.mapped_mem_size = mem_size; + + ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size, + IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL); + if (ret) { + dev_err(dev, "could not map video firmware region\n"); + return ret; + } + + reset_cpu_no_tz(core); + + return 0; +} + +void unload_fw_no_tz(struct venus_core *core) +{ + const size_t mapped = core->fw.mapped_mem_size; + struct iommu_domain *iommu; + size_t unmapped; + u32 reg; + struct device *dev = core->fw.dev; + void __iomem *wrapper_base = core->wrapper_base; + void __iomem *wrapper_tz_base = core->wrapper_tz_base; + + if (IS_IRIS2_1(core)) { + /* Assert the reset to XTSS */ + reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); + reg |= WRAPPER_XTSS_SW_RESET_BIT; + writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); + } else { + /* Assert the reset to ARM9 */ + reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET); + reg |= WRAPPER_A9SS_SW_RESET_BIT; + writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET); + } + + iommu = core->fw.iommu_domain; + + if (core->fw.mapped_mem_size && iommu) { + unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped); + + if (unmapped != mapped) + dev_err(dev, "failed to unmap firmware\n"); + else + core->fw.mapped_mem_size = 0; + } +} diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h new file mode 100644 index 0000000..5f008ef --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017 Linaro Ltd. + */ +#ifndef __FIRMWARE_NO_TZ_H__ +#define __FIRMWARE_NO_TZ_H__ + +struct device; + +#define VENUS_PAS_ID 9 + +int init_fw_no_tz(struct venus_core *core); +void deinit_fw_no_tz(struct venus_core *core); +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys, + size_t mem_size); +void unload_fw_no_tz(struct venus_core *core); +void set_hw_state_no_tz(struct venus_core *core, bool resume); + +#endif diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c similarity index 100% rename from drivers/media/platform/qcom/venus/helpers.c rename to drivers/media/platform/qcom/vcodec/venus/helpers.c diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h similarity index 100% rename from drivers/media/platform/qcom/venus/helpers.h rename to drivers/media/platform/qcom/vcodec/venus/helpers.h diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/vcodec/venus/hfi.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi.c rename to drivers/media/platform/qcom/vcodec/venus/hfi.c diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/vcodec/venus/hfi.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi.h rename to drivers/media/platform/qcom/vcodec/venus/hfi.h diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_cmds.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_cmds.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_helper.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_msgs.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_msgs.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_parser.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_parser.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c similarity index 99% rename from drivers/media/platform/qcom/venus/hfi_venus.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c index f9437b6..5a68db9 100644 --- a/drivers/media/platform/qcom/venus/hfi_venus.c +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c @@ -13,11 +13,12 @@ #include #include "core.h" +#include "../firmware.h" #include "hfi_cmds.h" #include "hfi_msgs.h" #include "hfi_venus.h" #include "hfi_venus_io.h" -#include "firmware.h" +#include "firmware_no_tz.h" #define HFI_MASK_QHDR_TX_TYPE 0xff000000 #define HFI_MASK_QHDR_RX_TYPE 0x00ff0000 @@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device *hdev) if (!hdev->power_enabled) return 0; - ret = venus_set_hw_state_suspend(hdev->core); + if (hdev->core->use_tz) + ret = set_hw_state(false); + else + set_hw_state_no_tz(hdev->core, false); if (ret) return ret; @@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device *hdev) if (hdev->power_enabled) return 0; - ret = venus_set_hw_state_resume(hdev->core); + if (hdev->core->use_tz) { + ret = set_hw_state(true); + if (ret == -EINVAL) + ret = 0; + } else { + set_hw_state_no_tz(hdev->core, true); + } if (ret) goto err; @@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device *hdev) return 0; err_suspend: - venus_set_hw_state_suspend(hdev->core); + if (hdev->core->use_tz) + set_hw_state(false); + else + set_hw_state_no_tz(hdev->core, false); err: hdev->power_enabled = false; return ret; diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_venus.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_venus_io.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c similarity index 100% rename from drivers/media/platform/qcom/venus/pm_helpers.c rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h similarity index 100% rename from drivers/media/platform/qcom/venus/pm_helpers.h rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c similarity index 100% rename from drivers/media/platform/qcom/venus/vdec.c rename to drivers/media/platform/qcom/vcodec/venus/vdec.c diff --git a/drivers/media/platform/qcom/venus/vdec.h b/drivers/media/platform/qcom/vcodec/venus/vdec.h similarity index 100% rename from drivers/media/platform/qcom/venus/vdec.h rename to drivers/media/platform/qcom/vcodec/venus/vdec.h diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c similarity index 100% rename from drivers/media/platform/qcom/venus/vdec_ctrls.c rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/vcodec/venus/venc.c similarity index 100% rename from drivers/media/platform/qcom/venus/venc.c rename to drivers/media/platform/qcom/vcodec/venus/venc.c diff --git a/drivers/media/platform/qcom/venus/venc.h b/drivers/media/platform/qcom/vcodec/venus/venc.h similarity index 100% rename from drivers/media/platform/qcom/venus/venc.h rename to drivers/media/platform/qcom/vcodec/venus/venc.h diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c similarity index 100% rename from drivers/media/platform/qcom/venus/venc_ctrls.c rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c diff --git a/drivers/media/platform/qcom/venus/firmware.c b/drivers/media/platform/qcom/venus/firmware.c deleted file mode 100644 index fe7da2b..0000000 --- a/drivers/media/platform/qcom/venus/firmware.c +++ /dev/null @@ -1,363 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Copyright (C) 2017 Linaro Ltd. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "core.h" -#include "firmware.h" -#include "hfi_venus_io.h" - -#define VENUS_PAS_ID 9 -#define VENUS_FW_MEM_SIZE (6 * SZ_1M) -#define VENUS_FW_START_ADDR 0x0 - -static void venus_reset_cpu(struct venus_core *core) -{ - u32 fw_size = core->fw.mapped_mem_size; - void __iomem *wrapper_base; - - if (IS_IRIS2_1(core)) - wrapper_base = core->wrapper_tz_base; - else - wrapper_base = core->wrapper_base; - - writel(0, wrapper_base + WRAPPER_FW_START_ADDR); - writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR); - writel(0, wrapper_base + WRAPPER_CPA_START_ADDR); - writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR); - writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR); - writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR); - - if (IS_IRIS2_1(core)) { - /* Bring XTSS out of reset */ - writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET); - } else { - writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS); - writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG); - - /* Bring ARM9 out of reset */ - writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET); - } -} - -int venus_set_hw_state(struct venus_core *core, bool resume) -{ - int ret; - - if (core->use_tz) { - ret = qcom_scm_set_remote_state(resume, 0); - if (resume && ret == -EINVAL) - ret = 0; - return ret; - } - - if (resume) { - venus_reset_cpu(core); - } else { - if (IS_IRIS2_1(core)) - writel(WRAPPER_XTSS_SW_RESET_BIT, - core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); - else - writel(WRAPPER_A9SS_SW_RESET_BIT, - core->wrapper_base + WRAPPER_A9SS_SW_RESET); - } - - return 0; -} - -static int venus_load_fw(struct venus_core *core, const char *fwname, - phys_addr_t *mem_phys, size_t *mem_size) -{ - const struct firmware *mdt; - struct reserved_mem *rmem; - struct device_node *node; - struct device *dev; - ssize_t fw_size; - void *mem_va; - int ret; - - *mem_phys = 0; - *mem_size = 0; - - dev = core->dev; - node = of_parse_phandle(dev->of_node, "memory-region", 0); - if (!node) { - dev_err(dev, "no memory-region specified\n"); - return -EINVAL; - } - - rmem = of_reserved_mem_lookup(node); - of_node_put(node); - if (!rmem) { - dev_err(dev, "failed to lookup reserved memory-region\n"); - return -EINVAL; - } - - ret = request_firmware(&mdt, fwname, dev); - if (ret < 0) - return ret; - - fw_size = qcom_mdt_get_size(mdt); - if (fw_size < 0) { - ret = fw_size; - goto err_release_fw; - } - - *mem_phys = rmem->base; - *mem_size = rmem->size; - - if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) { - ret = -EINVAL; - goto err_release_fw; - } - - mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC); - if (!mem_va) { - dev_err(dev, "unable to map memory region %pa size %#zx\n", mem_phys, *mem_size); - ret = -ENOMEM; - goto err_release_fw; - } - - if (core->use_tz) - ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID, - mem_va, *mem_phys, *mem_size, NULL); - else - ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID, - mem_va, *mem_phys, *mem_size, NULL); - - memunmap(mem_va); -err_release_fw: - release_firmware(mdt); - return ret; -} - -static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys, - size_t mem_size) -{ - struct iommu_domain *iommu; - struct device *dev; - int ret; - - dev = core->fw.dev; - if (!dev) - return -EPROBE_DEFER; - - iommu = core->fw.iommu_domain; - core->fw.mapped_mem_size = mem_size; - - ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size, - IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL); - if (ret) { - dev_err(dev, "could not map video firmware region\n"); - return ret; - } - - venus_reset_cpu(core); - - return 0; -} - -static int venus_shutdown_no_tz(struct venus_core *core) -{ - const size_t mapped = core->fw.mapped_mem_size; - struct iommu_domain *iommu; - size_t unmapped; - u32 reg; - struct device *dev = core->fw.dev; - void __iomem *wrapper_base = core->wrapper_base; - void __iomem *wrapper_tz_base = core->wrapper_tz_base; - - if (IS_IRIS2_1(core)) { - /* Assert the reset to XTSS */ - reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); - reg |= WRAPPER_XTSS_SW_RESET_BIT; - writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); - } else { - /* Assert the reset to ARM9 */ - reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET); - reg |= WRAPPER_A9SS_SW_RESET_BIT; - writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET); - } - - iommu = core->fw.iommu_domain; - - if (core->fw.mapped_mem_size && iommu) { - unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped); - - if (unmapped != mapped) - dev_err(dev, "failed to unmap firmware\n"); - else - core->fw.mapped_mem_size = 0; - } - - return 0; -} - -int venus_boot(struct venus_core *core) -{ - struct device *dev = core->dev; - const struct venus_resources *res = core->res; - const char *fwpath = NULL; - phys_addr_t mem_phys; - size_t mem_size; - int ret; - - if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) || - (core->use_tz && !qcom_scm_is_available())) - return -EPROBE_DEFER; - - ret = of_property_read_string_index(dev->of_node, "firmware-name", 0, - &fwpath); - if (ret) - fwpath = core->res->fwname; - - ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size); - if (ret) { - dev_err(dev, "fail to load video firmware\n"); - return -EINVAL; - } - - core->fw.mem_size = mem_size; - core->fw.mem_phys = mem_phys; - - if (core->use_tz) - ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID); - else - ret = venus_boot_no_tz(core, mem_phys, mem_size); - - if (ret) - return ret; - - if (core->use_tz && res->cp_size) { - /* - * Clues for porting using downstream data: - * cp_start = 0 - * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size! - * This works, as the non-secure context bank is placed - * contiguously right after the Content Protection region. - * - * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0] - * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1] - */ - ret = qcom_scm_mem_protect_video_var(res->cp_start, - res->cp_size, - res->cp_nonpixel_start, - res->cp_nonpixel_size); - if (ret) { - qcom_scm_pas_shutdown(VENUS_PAS_ID); - dev_err(dev, "set virtual address ranges fail (%d)\n", - ret); - return ret; - } - } - - return 0; -} - -int venus_shutdown(struct venus_core *core) -{ - int ret; - - if (core->use_tz) - ret = qcom_scm_pas_shutdown(VENUS_PAS_ID); - else - ret = venus_shutdown_no_tz(core); - - return ret; -} - -int venus_firmware_init(struct venus_core *core) -{ - struct platform_device_info info; - struct iommu_domain *iommu_dom; - struct platform_device *pdev; - struct device_node *np; - int ret; - - np = of_get_child_by_name(core->dev->of_node, "video-firmware"); - if (!np) { - core->use_tz = true; - return 0; - } - - memset(&info, 0, sizeof(info)); - info.fwnode = &np->fwnode; - info.parent = core->dev; - info.name = np->name; - info.dma_mask = DMA_BIT_MASK(32); - - pdev = platform_device_register_full(&info); - if (IS_ERR(pdev)) { - of_node_put(np); - return PTR_ERR(pdev); - } - - pdev->dev.of_node = np; - - ret = of_dma_configure(&pdev->dev, np, true); - if (ret) { - dev_err(core->dev, "dma configure fail\n"); - goto err_unregister; - } - - core->fw.dev = &pdev->dev; - - iommu_dom = iommu_domain_alloc(&platform_bus_type); - if (!iommu_dom) { - dev_err(core->fw.dev, "Failed to allocate iommu domain\n"); - ret = -ENOMEM; - goto err_unregister; - } - - ret = iommu_attach_device(iommu_dom, core->fw.dev); - if (ret) { - dev_err(core->fw.dev, "could not attach device\n"); - goto err_iommu_free; - } - - core->fw.iommu_domain = iommu_dom; - - of_node_put(np); - - return 0; - -err_iommu_free: - iommu_domain_free(iommu_dom); -err_unregister: - platform_device_unregister(pdev); - of_node_put(np); - return ret; -} - -void venus_firmware_deinit(struct venus_core *core) -{ - struct iommu_domain *iommu; - - if (!core->fw.dev) - return; - - iommu = core->fw.iommu_domain; - - iommu_detach_device(iommu, core->fw.dev); - - if (core->fw.iommu_domain) { - iommu_domain_free(iommu); - core->fw.iommu_domain = NULL; - } - - platform_device_unregister(to_platform_device(core->fw.dev)); -} diff --git a/drivers/media/platform/qcom/venus/firmware.h b/drivers/media/platform/qcom/venus/firmware.h deleted file mode 100644 index aaccd84..0000000 --- a/drivers/media/platform/qcom/venus/firmware.h +++ /dev/null @@ -1,26 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (C) 2017 Linaro Ltd. - */ -#ifndef __VENUS_FIRMWARE_H__ -#define __VENUS_FIRMWARE_H__ - -struct device; - -int venus_firmware_init(struct venus_core *core); -void venus_firmware_deinit(struct venus_core *core); -int venus_boot(struct venus_core *core); -int venus_shutdown(struct venus_core *core); -int venus_set_hw_state(struct venus_core *core, bool suspend); - -static inline int venus_set_hw_state_suspend(struct venus_core *core) -{ - return venus_set_hw_state(core, false); -} - -static inline int venus_set_hw_state_resume(struct venus_core *core) -{ - return venus_set_hw_state(core, true); -} - -#endif From patchwork Mon Dec 18 11:31:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757249 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1922F1A714; Mon, 18 Dec 2023 11:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="L1ZCkcZ+" Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAscO4000368; Mon, 18 Dec 2023 11:33:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=bQdmZd+idGctlHl2svbf9r/7EV7zOUrFdIGTxbDrfGw=; b=L1 ZCkcZ+FADLcg9PK5YrH7PTFfzrjtQFTS0kPCHxnZeZzQEdPVI3Ag0cbfYKL2iN5X za8ydKYd9/c3ap0GkXNr7/YQm9Q92jTL5SXY+909LBhrOtrGRqdfBSJKLMoJm72m /l3JM8lljqOhwPgV6LoHG95065UBmdZGFJJsGRBNcMeJufgq2P8Y6Ner7EG/iKG5 AVfswmbtXcCequIUJPvhamvvCAnW6bt+iFNj6iXosu5w3ZxJFIorMUi0hwsy8785 YoThpU2R8KRXu3lmIfvlK3hbg3jUm8X64SaP9x6hFNqORcC+VJkLmnFWkPJlv6ge CAuufTZVq/eBWo755Lwg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v14xy443w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:09 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Xe029942; Mon, 18 Dec 2023 11:33:05 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyg6-1; Mon, 18 Dec 2023 11:33:05 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX50h029913; Mon, 18 Dec 2023 11:33:05 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX530029906; Mon, 18 Dec 2023 11:33:05 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 23B6722BC; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 02/34] media: introduce common helpers for queues handling Date: Mon, 18 Dec 2023 17:01:57 +0530 Message-Id: <1702899149-21321-3-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 4G0iKqfqT_8ZcAehWwbFf1zCDz1xgW1F X-Proofpoint-ORIG-GUID: 4G0iKqfqT_8ZcAehWwbFf1zCDz1xgW1F X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Introduce common helpers for shared queues init, deinit and read/write queue operations and use them in venus driver. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++++++++++ drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++++++ drivers/media/platform/qcom/vcodec/venus/Makefile | 3 +- .../media/platform/qcom/vcodec/venus/hfi_venus.c | 500 ++++----------------- 4 files changed, 465 insertions(+), 425 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.c b/drivers/media/platform/qcom/vcodec/hfi_queue.c new file mode 100644 index 0000000..e3e470b --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.c @@ -0,0 +1,258 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "hfi_queue.h" + +int write_queue(struct iface_q_info *qinfo, void *packet, + u32 packet_size, u32 *rx_req) +{ + u32 empty_space, read_idx, write_idx, new_write_idx; + struct hfi_queue_header *queue; + u32 *write_ptr, residue; + + queue = qinfo->qhdr; + if (!queue) + return -EINVAL; + + if (!packet_size || packet_size > qinfo->q_array.size) + return -EINVAL; + + read_idx = queue->qhdr_read_idx * sizeof(u32); + write_idx = queue->qhdr_write_idx * sizeof(u32); + + empty_space = (write_idx >= read_idx) ? + (qinfo->q_array.size - (write_idx - read_idx)) : + (read_idx - write_idx); + if (empty_space <= packet_size) { + queue->qhdr_tx_req = 1; + return -ENOSPC; + } + + queue->qhdr_tx_req = 0; + + new_write_idx = write_idx + packet_size; + write_ptr = (u32 *)((u8 *)qinfo->q_array.kernel_vaddr + write_idx); + + if (write_ptr < (u32 *)qinfo->q_array.kernel_vaddr || + write_ptr > (u32 *)(qinfo->q_array.kernel_vaddr + + qinfo->q_array.size)) + return -EINVAL; + + if (new_write_idx < qinfo->q_array.size) { + memcpy(write_ptr, packet, packet_size); + } else { + residue = new_write_idx - qinfo->q_array.size; + memcpy(write_ptr, packet, (packet_size - residue)); + memcpy(qinfo->q_array.kernel_vaddr, + packet + (packet_size - residue), residue); + new_write_idx = residue; + } + + /* Make sure packet is written before updating the write index */ + mb(); + queue->qhdr_write_idx = new_write_idx / sizeof(u32); + *rx_req = queue->qhdr_rx_req ? 1 : 0; + + /* Make sure write index is updated before an interrupt is raised */ + mb(); + + return 0; +} + +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req) +{ + u32 read_idx, write_idx, new_read_idx; + struct hfi_queue_header *queue; + u32 receive_request = 0; + u32 packet_size, residue; + u32 *read_ptr; + int ret = 0; + + /* Make sure data is valid before reading it */ + mb(); + queue = qinfo->qhdr; + if (!queue) + return -EINVAL; + + if (queue->qhdr_type & IFACEQ_MSGQ_ID) + receive_request = 1; + + read_idx = queue->qhdr_read_idx * sizeof(u32); + write_idx = queue->qhdr_write_idx * sizeof(u32); + + if (read_idx == write_idx) { + queue->qhdr_rx_req = receive_request; + /* Ensure qhdr is updated in main memory */ + mb(); + return -ENODATA; + } + + read_ptr = (u32 *)(qinfo->q_array.kernel_vaddr + read_idx); + if (read_ptr < (u32 *)qinfo->q_array.kernel_vaddr || + read_ptr > (u32 *)(qinfo->q_array.kernel_vaddr + + qinfo->q_array.size - sizeof(*read_ptr))) + return -ENODATA; + + packet_size = *read_ptr; + if (!packet_size) + return -EINVAL; + + new_read_idx = read_idx + packet_size; + if (packet_size <= IFACEQ_CORE_PKT_SIZE && + read_idx <= qinfo->q_array.size) { + if (new_read_idx < qinfo->q_array.size) { + memcpy(packet, read_ptr, packet_size); + } else { + residue = new_read_idx - qinfo->q_array.size; + memcpy(packet, read_ptr, (packet_size - residue)); + memcpy((packet + (packet_size - residue)), + qinfo->q_array.kernel_vaddr, residue); + new_read_idx = residue; + } + } else { + new_read_idx = write_idx; + ret = -EBADMSG; + } + + queue->qhdr_rx_req = receive_request; + + queue->qhdr_read_idx = new_read_idx / sizeof(u32); + + *tx_req = queue->qhdr_tx_req ? 1 : 0; + /* Ensure qhdr is updated in main memory */ + mb(); + + return ret; +} + +static void __set_queue_hdr_defaults(struct hfi_queue_header *q_hdr) +{ + q_hdr->qhdr_status = 0x1; + q_hdr->qhdr_type = IFACEQ_DFLT_QHDR; + q_hdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4; + q_hdr->qhdr_pkt_size = 0; + q_hdr->qhdr_rx_wm = 0x1; + q_hdr->qhdr_tx_wm = 0x1; + q_hdr->qhdr_rx_req = 0x1; + q_hdr->qhdr_tx_req = 0x0; + q_hdr->qhdr_rx_irq_status = 0x0; + q_hdr->qhdr_tx_irq_status = 0x0; + q_hdr->qhdr_read_idx = 0x0; + q_hdr->qhdr_write_idx = 0x0; +} + +static void hfi_set_queue_header(u32 queue_id, struct iface_q_info *iface_q) +{ + __set_queue_hdr_defaults(iface_q->qhdr); + iface_q->qhdr->qhdr_start_addr = iface_q->q_array.device_addr; + iface_q->qhdr->qhdr_type |= queue_id; + + /* + * Set receive request to zero on debug queue as there is no + * need of interrupt from video hardware for debug messages + */ + if (queue_id == IFACEQ_DBGQ_ID) + iface_q->qhdr->qhdr_rx_req = 0; +} + +static void __queue_init(struct mem_desc *q_table, u32 queue_id, struct iface_q_info *iface_q) +{ + unsigned int offset = 0; + + offset = q_table->size + (queue_id * IFACEQ_QUEUE_SIZE); + iface_q->q_array.device_addr = q_table->device_addr + offset; + iface_q->q_array.kernel_vaddr = + (void *)((char *)q_table->kernel_vaddr + offset); + iface_q->q_array.size = IFACEQ_QUEUE_SIZE; + iface_q->qhdr = + IFACEQ_GET_QHDR_START_ADDR(q_table->kernel_vaddr, queue_id); + + memset(iface_q->qhdr, 0, sizeof(struct hfi_queue_header)); + + hfi_set_queue_header(queue_id, iface_q); +} + +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q, void *daddr) +{ + struct hfi_queue_table_header *q_tbl_hdr; + + if (q_table->kernel_vaddr) { + hfi_set_queue_header(IFACEQ_CMDQ_ID, cmd_q); + hfi_set_queue_header(IFACEQ_MSGQ_ID, msg_q); + hfi_set_queue_header(IFACEQ_DBGQ_ID, dbg_q); + return 0; + } + + q_table->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_QUEUE_SIZE, + &q_table->device_addr, + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE); + if (!q_table->kernel_vaddr) { + dev_err(dev, "%s: queues alloc and map failed\n", __func__); + return -ENOMEM; + } + + sfr->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_SFR_SIZE, + &sfr->device_addr, + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE); + if (!sfr->kernel_vaddr) { + dev_err(dev, "%s: sfr alloc and map failed\n", __func__); + return -ENOMEM; + } + + q_table->size = IFACEQ_TABLE_SIZE; + + __queue_init(q_table, IFACEQ_CMDQ_ID, cmd_q); + __queue_init(q_table, IFACEQ_MSGQ_ID, msg_q); + __queue_init(q_table, IFACEQ_DBGQ_ID, dbg_q); + + q_tbl_hdr = (struct hfi_queue_table_header *)q_table->kernel_vaddr; + q_tbl_hdr->qtbl_version = 0; + q_tbl_hdr->device_addr = (void *)daddr; + strscpy(q_tbl_hdr->name, "hfi-queues", sizeof(q_tbl_hdr->name)); + q_tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE; + q_tbl_hdr->qtbl_qhdr0_offset = sizeof(*q_tbl_hdr); + q_tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header); + q_tbl_hdr->qtbl_num_q = IFACEQ_NUMQ; + q_tbl_hdr->qtbl_num_active_q = IFACEQ_NUMQ; + + sfr->size = ALIGNED_SFR_SIZE; + /* Write sfr size in first word to be used by firmware */ + *((u32 *)sfr->kernel_vaddr) = sfr->size; + + return 0; +} + +static void __queue_deinit(struct iface_q_info *iface_q) +{ + iface_q->qhdr = NULL; + iface_q->q_array.kernel_vaddr = NULL; + iface_q->q_array.device_addr = 0; +} + +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q) +{ + if (!q_table->kernel_vaddr) + return; + + dma_free_attrs(dev, q_table->size, q_table->kernel_vaddr, + q_table->device_addr, q_table->attrs); + + dma_free_attrs(dev, sfr->size, sfr->kernel_vaddr, + sfr->device_addr, sfr->attrs); + + __queue_deinit(cmd_q); + __queue_deinit(msg_q); + __queue_deinit(dbg_q); + + q_table->kernel_vaddr = NULL; + q_table->device_addr = 0; + + sfr->kernel_vaddr = NULL; + sfr->device_addr = 0; +} diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.h b/drivers/media/platform/qcom/vcodec/hfi_queue.h new file mode 100644 index 0000000..8cc9491 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.h @@ -0,0 +1,129 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _HFI_QUEUE_H_ +#define _HFI_QUEUE_H_ + +#include +#include + +/* + * Shared queues are used for communication between driver and firmware. + * There are 3 types of queues: + * Command queue - driver to write any command to firmware. + * Message queue - firmware to send any response to driver. + * Debug queue - firmware to write debug message. + */ + +/* Host-firmware shared queue ids */ +enum iris_iface_queue { + IFACEQ_CMDQ_ID = 0, + IFACEQ_MSGQ_ID, + IFACEQ_DBGQ_ID, + IFACEQ_NUMQ, /* not an index */ +}; + +#define IFACEQ_MAX_BUF_COUNT 50 +/* + * Max session supported are 16. + * this value is used to calculate the size of + * individual shared queue. + */ +#define IFACE_MAX_PARALLEL_CLNTS 16 +#define IFACEQ_DFLT_QHDR 0x01010000 + +#define IFACEQ_MAX_PKT_SIZE 1024 +#define IFACEQ_CORE_PKT_SIZE (1024 * 4) + +#define IFACEQ_TABLE_SIZE (sizeof(struct hfi_queue_table_header) + \ + sizeof(struct hfi_queue_header) * IFACEQ_NUMQ) + +#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \ + IFACEQ_MAX_BUF_COUNT * IFACE_MAX_PARALLEL_CLNTS) + +#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \ + ((void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \ + ((i) * sizeof(struct hfi_queue_header)))) + +#define SFR_SIZE SZ_4K +#define QUEUE_SIZE (IFACEQ_TABLE_SIZE + \ + (IFACEQ_QUEUE_SIZE * IFACEQ_NUMQ)) + +#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K) +#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K) +#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE, SZ_1M) + +struct mem_desc { + dma_addr_t device_addr; + void *kernel_vaddr; + u32 size; + unsigned long attrs; +}; + +struct hfi_queue_table_header { + u32 qtbl_version; + u32 qtbl_size; + u32 qtbl_qhdr0_offset; + u32 qtbl_qhdr_size; + u32 qtbl_num_q; + u32 qtbl_num_active_q; + void *device_addr; + char name[256]; +}; + +/** + * struct hfi_queue_header - queue header definition + * + * @qhdr_status: bits (7:0) 0x1 - active 0x0 - inactive + * @qhdr_start_addr: queue start addr. + * @qhdr_type: queue type. + * @qhdr_q_size: size in number of HFI_QUEUE_PACKET_TYPE packets it can hold. + * @qhdr_pkt_size: size of the Queue packet entries, in bytes. + * @qhdr_rx_wm: the receiver watermark in number of queue packets. + * @qhdr_tx_wm: the transmitter watermark in number of queue packets. + * @qhdr_rx_req: set to request an interrupt from transmitter if qhdr_tx_wm is reached. + * @qhdr_tx_req: set to request an interrupt from receiver if qhdr_rx_wm is reached. + * @qhdr_rx_irq_status: status of receiver irq. + * @qhdr_tx_irq_status: status of transmitter irq. + * @qhdr_read_idx: read index, which starts with a value of 0 and wraps around after + * (QHDR_Q_SIZE - 1). + * @qhdr_write_idx: write index, which starts with a value of 0 and wraps around after + * (QHDR_Q_SIZE - 1). + */ + +struct hfi_queue_header { + u32 qhdr_status; + u32 qhdr_start_addr; + u32 qhdr_type; + u32 qhdr_q_size; + u32 qhdr_pkt_size; + u32 qhdr_pkt_drop_cnt; + u32 qhdr_rx_wm; + u32 qhdr_tx_wm; + u32 qhdr_rx_req; + u32 qhdr_tx_req; + u32 qhdr_rx_irq_status; + u32 qhdr_tx_irq_status; + u32 qhdr_read_idx; + u32 qhdr_write_idx; +}; + +struct iface_q_info { + struct hfi_queue_header *qhdr; + struct mem_desc q_array; +}; + +int write_queue(struct iface_q_info *qinfo, void *packet, + u32 packet_size, u32 *rx_req); +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req); +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q, void *daddr); + +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile index f6f3a88..1941ef4 100644 --- a/drivers/media/platform/qcom/vcodec/venus/Makefile +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile @@ -1,7 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 # Makefile for Qualcomm Venus driver -venus-core-objs += ../firmware.o +venus-core-objs += ../firmware.o \ + ../hfi_queue.o venus-core-objs += core.o helpers.o firmware_no_tz.o \ hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \ diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c index 5a68db9..41e3f27 100644 --- a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c @@ -16,6 +16,7 @@ #include "../firmware.h" #include "hfi_cmds.h" #include "hfi_msgs.h" +#include "../hfi_queue.h" #include "hfi_venus.h" #include "hfi_venus_io.h" #include "firmware_no_tz.h" @@ -34,79 +35,13 @@ #define IFACEQ_CMD_IDX 0 #define IFACEQ_MSG_IDX 1 #define IFACEQ_DBG_IDX 2 -#define IFACEQ_MAX_BUF_COUNT 50 -#define IFACEQ_MAX_PARALLEL_CLNTS 16 -#define IFACEQ_DFLT_QHDR 0x01010000 #define POLL_INTERVAL_US 50 -#define IFACEQ_MAX_PKT_SIZE 1024 -#define IFACEQ_MED_PKT_SIZE 768 -#define IFACEQ_MIN_PKT_SIZE 8 #define IFACEQ_VAR_SMALL_PKT_SIZE 100 #define IFACEQ_VAR_LARGE_PKT_SIZE 512 #define IFACEQ_VAR_HUGE_PKT_SIZE (1024 * 12) -struct hfi_queue_table_header { - u32 version; - u32 size; - u32 qhdr0_offset; - u32 qhdr_size; - u32 num_q; - u32 num_active_q; -}; - -struct hfi_queue_header { - u32 status; - u32 start_addr; - u32 type; - u32 q_size; - u32 pkt_size; - u32 pkt_drop_cnt; - u32 rx_wm; - u32 tx_wm; - u32 rx_req; - u32 tx_req; - u32 rx_irq_status; - u32 tx_irq_status; - u32 read_idx; - u32 write_idx; -}; - -#define IFACEQ_TABLE_SIZE \ - (sizeof(struct hfi_queue_table_header) + \ - sizeof(struct hfi_queue_header) * IFACEQ_NUM) - -#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \ - IFACEQ_MAX_BUF_COUNT * IFACEQ_MAX_PARALLEL_CLNTS) - -#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \ - (void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \ - ((i) * sizeof(struct hfi_queue_header))) - -#define QDSS_SIZE SZ_4K -#define SFR_SIZE SZ_4K -#define QUEUE_SIZE \ - (IFACEQ_TABLE_SIZE + (IFACEQ_QUEUE_SIZE * IFACEQ_NUM)) - -#define ALIGNED_QDSS_SIZE ALIGN(QDSS_SIZE, SZ_4K) -#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K) -#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K) -#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE + \ - ALIGNED_QDSS_SIZE, SZ_1M) - -struct mem_desc { - dma_addr_t da; /* device address */ - void *kva; /* kernel virtual address */ - u32 size; - unsigned long attrs; -}; - -struct iface_queue { - struct hfi_queue_header *qhdr; - struct mem_desc qmem; -}; - enum venus_state { VENUS_STATE_DEINIT = 1, VENUS_STATE_INIT, @@ -125,12 +60,11 @@ struct venus_hfi_device { struct completion release_resource; struct mem_desc ifaceq_table; struct mem_desc sfr; - struct iface_queue queues[IFACEQ_NUM]; + struct iface_q_info queues[IFACEQ_NUM]; u8 pkt_buf[IFACEQ_VAR_HUGE_PKT_SIZE]; u8 dbg_buf[IFACEQ_VAR_HUGE_PKT_SIZE]; }; -static bool venus_pkt_debug; int venus_fw_debug = HFI_DEBUG_MSG_ERROR | HFI_DEBUG_MSG_FATAL; static bool venus_fw_low_power_mode = true; static int venus_hw_rsp_timeout = 1000; @@ -149,212 +83,6 @@ static bool venus_is_valid_state(struct venus_hfi_device *hdev) return hdev->state != VENUS_STATE_DEINIT; } -static void venus_dump_packet(struct venus_hfi_device *hdev, const void *packet) -{ - size_t pkt_size = *(u32 *)packet; - - if (!venus_pkt_debug) - return; - - print_hex_dump(KERN_DEBUG, "", DUMP_PREFIX_OFFSET, 16, 1, packet, - pkt_size, true); -} - -static int venus_write_queue(struct venus_hfi_device *hdev, - struct iface_queue *queue, - void *packet, u32 *rx_req) -{ - struct hfi_queue_header *qhdr; - u32 dwords, new_wr_idx; - u32 empty_space, rd_idx, wr_idx, qsize; - u32 *wr_ptr; - - if (!queue->qmem.kva) - return -EINVAL; - - qhdr = queue->qhdr; - if (!qhdr) - return -EINVAL; - - venus_dump_packet(hdev, packet); - - dwords = (*(u32 *)packet) >> 2; - if (!dwords) - return -EINVAL; - - rd_idx = qhdr->read_idx; - wr_idx = qhdr->write_idx; - qsize = qhdr->q_size; - /* ensure rd/wr indices's are read from memory */ - rmb(); - - if (wr_idx >= rd_idx) - empty_space = qsize - (wr_idx - rd_idx); - else - empty_space = rd_idx - wr_idx; - - if (empty_space <= dwords) { - qhdr->tx_req = 1; - /* ensure tx_req is updated in memory */ - wmb(); - return -ENOSPC; - } - - qhdr->tx_req = 0; - /* ensure tx_req is updated in memory */ - wmb(); - - new_wr_idx = wr_idx + dwords; - wr_ptr = (u32 *)(queue->qmem.kva + (wr_idx << 2)); - - if (wr_ptr < (u32 *)queue->qmem.kva || - wr_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*wr_ptr))) - return -EINVAL; - - if (new_wr_idx < qsize) { - memcpy(wr_ptr, packet, dwords << 2); - } else { - size_t len; - - new_wr_idx -= qsize; - len = (dwords - new_wr_idx) << 2; - memcpy(wr_ptr, packet, len); - memcpy(queue->qmem.kva, packet + len, new_wr_idx << 2); - } - - /* make sure packet is written before updating the write index */ - wmb(); - - qhdr->write_idx = new_wr_idx; - *rx_req = qhdr->rx_req ? 1 : 0; - - /* make sure write index is updated before an interrupt is raised */ - mb(); - - return 0; -} - -static int venus_read_queue(struct venus_hfi_device *hdev, - struct iface_queue *queue, void *pkt, u32 *tx_req) -{ - struct hfi_queue_header *qhdr; - u32 dwords, new_rd_idx; - u32 rd_idx, wr_idx, type, qsize; - u32 *rd_ptr; - u32 recv_request = 0; - int ret = 0; - - if (!queue->qmem.kva) - return -EINVAL; - - qhdr = queue->qhdr; - if (!qhdr) - return -EINVAL; - - type = qhdr->type; - rd_idx = qhdr->read_idx; - wr_idx = qhdr->write_idx; - qsize = qhdr->q_size; - - /* make sure data is valid before using it */ - rmb(); - - /* - * Do not set receive request for debug queue, if set, Venus generates - * interrupt for debug messages even when there is no response message - * available. In general debug queue will not become full as it is being - * emptied out for every interrupt from Venus. Venus will anyway - * generates interrupt if it is full. - */ - if (type & HFI_CTRL_TO_HOST_MSG_Q) - recv_request = 1; - - if (rd_idx == wr_idx) { - qhdr->rx_req = recv_request; - *tx_req = 0; - /* update rx_req field in memory */ - wmb(); - return -ENODATA; - } - - rd_ptr = (u32 *)(queue->qmem.kva + (rd_idx << 2)); - - if (rd_ptr < (u32 *)queue->qmem.kva || - rd_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*rd_ptr))) - return -EINVAL; - - dwords = *rd_ptr >> 2; - if (!dwords) - return -EINVAL; - - new_rd_idx = rd_idx + dwords; - if (((dwords << 2) <= IFACEQ_VAR_HUGE_PKT_SIZE) && rd_idx <= qsize) { - if (new_rd_idx < qsize) { - memcpy(pkt, rd_ptr, dwords << 2); - } else { - size_t len; - - new_rd_idx -= qsize; - len = (dwords - new_rd_idx) << 2; - memcpy(pkt, rd_ptr, len); - memcpy(pkt + len, queue->qmem.kva, new_rd_idx << 2); - } - } else { - /* bad packet received, dropping */ - new_rd_idx = qhdr->write_idx; - ret = -EBADMSG; - } - - /* ensure the packet is read before updating read index */ - rmb(); - - qhdr->read_idx = new_rd_idx; - /* ensure updating read index */ - wmb(); - - rd_idx = qhdr->read_idx; - wr_idx = qhdr->write_idx; - /* ensure rd/wr indices are read from memory */ - rmb(); - - if (rd_idx != wr_idx) - qhdr->rx_req = 0; - else - qhdr->rx_req = recv_request; - - *tx_req = qhdr->tx_req ? 1 : 0; - - /* ensure rx_req is stored to memory and tx_req is loaded from memory */ - mb(); - - venus_dump_packet(hdev, pkt); - - return ret; -} - -static int venus_alloc(struct venus_hfi_device *hdev, struct mem_desc *desc, - u32 size) -{ - struct device *dev = hdev->core->dev; - - desc->attrs = DMA_ATTR_WRITE_COMBINE; - desc->size = ALIGN(size, SZ_4K); - - desc->kva = dma_alloc_attrs(dev, desc->size, &desc->da, GFP_KERNEL, - desc->attrs); - if (!desc->kva) - return -ENOMEM; - - return 0; -} - -static void venus_free(struct venus_hfi_device *hdev, struct mem_desc *mem) -{ - struct device *dev = hdev->core->dev; - - dma_free_attrs(dev, mem->size, mem->kva, mem->da, mem->attrs); -} - static void venus_set_registers(struct venus_hfi_device *hdev) { const struct venus_resources *res = hdev->core->res; @@ -380,11 +108,11 @@ static void venus_soft_int(struct venus_hfi_device *hdev) } static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, - void *pkt, bool sync) + void *pkt, u32 pkt_size, bool sync) { struct device *dev = hdev->core->dev; struct hfi_pkt_hdr *cmd_packet; - struct iface_queue *queue; + struct iface_q_info *queue; u32 rx_req; int ret; @@ -396,7 +124,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, queue = &hdev->queues[IFACEQ_CMD_IDX]; - ret = venus_write_queue(hdev, queue, pkt, &rx_req); + ret = write_queue(queue, pkt, pkt_size, &rx_req); if (ret) { dev_err(dev, "write to iface cmd queue failed (%d)\n", ret); return ret; @@ -408,7 +136,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, * commands */ queue = &hdev->queues[IFACEQ_MSG_IDX]; - queue->qhdr->rx_req = 1; + queue->qhdr->qhdr_rx_req = 1; /* ensure rx_req is updated in memory */ wmb(); } @@ -419,12 +147,12 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, return 0; } -static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, bool sync) +static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, u32 pkt_size, bool sync) { int ret; mutex_lock(&hdev->lock); - ret = venus_iface_cmdq_write_nolock(hdev, pkt, sync); + ret = venus_iface_cmdq_write_nolock(hdev, pkt, pkt_size, sync); mutex_unlock(&hdev->lock); return ret; @@ -447,7 +175,7 @@ static int venus_hfi_core_set_resource(struct venus_core *core, u32 id, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, pkt, false); + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); if (ret) return ret; @@ -531,12 +259,12 @@ static int venus_run(struct venus_hfi_device *hdev) */ venus_set_registers(hdev); - writel(hdev->ifaceq_table.da, cpu_cs_base + UC_REGION_ADDR); + writel(hdev->ifaceq_table.device_addr, cpu_cs_base + UC_REGION_ADDR); writel(SHARED_QSIZE, cpu_cs_base + UC_REGION_SIZE); - writel(hdev->ifaceq_table.da, cpu_cs_base + CPU_CS_SCIACMDARG2); + writel(hdev->ifaceq_table.device_addr, cpu_cs_base + CPU_CS_SCIACMDARG2); writel(0x01, cpu_cs_base + CPU_CS_SCIACMDARG1); - if (hdev->sfr.da) - writel(hdev->sfr.da, cpu_cs_base + SFR_ADDR); + if (hdev->sfr.device_addr) + writel(hdev->sfr.device_addr, cpu_cs_base + SFR_ADDR); ret = venus_boot_core(hdev); if (ret) { @@ -690,7 +418,7 @@ static int venus_power_on(struct venus_hfi_device *hdev) static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev, void *pkt) { - struct iface_queue *queue; + struct iface_q_info *queue; u32 tx_req; int ret; @@ -699,7 +427,7 @@ static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev, queue = &hdev->queues[IFACEQ_MSG_IDX]; - ret = venus_read_queue(hdev, queue, pkt, &tx_req); + ret = read_queue(queue, pkt, &tx_req); if (ret) return ret; @@ -723,7 +451,7 @@ static int venus_iface_msgq_read(struct venus_hfi_device *hdev, void *pkt) static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev, void *pkt) { - struct iface_queue *queue; + struct iface_q_info *queue; u32 tx_req; int ret; @@ -733,7 +461,7 @@ static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev, queue = &hdev->queues[IFACEQ_DBG_IDX]; - ret = venus_read_queue(hdev, queue, pkt, &tx_req); + ret = read_queue(queue, pkt, &tx_req); if (ret) return ret; @@ -759,100 +487,18 @@ static int venus_iface_dbgq_read(struct venus_hfi_device *hdev, void *pkt) static void venus_set_qhdr_defaults(struct hfi_queue_header *qhdr) { - qhdr->status = 1; - qhdr->type = IFACEQ_DFLT_QHDR; - qhdr->q_size = IFACEQ_QUEUE_SIZE / 4; - qhdr->pkt_size = 0; - qhdr->rx_wm = 1; - qhdr->tx_wm = 1; - qhdr->rx_req = 1; - qhdr->tx_req = 0; - qhdr->rx_irq_status = 0; - qhdr->tx_irq_status = 0; - qhdr->read_idx = 0; - qhdr->write_idx = 0; -} - -static void venus_interface_queues_release(struct venus_hfi_device *hdev) -{ - mutex_lock(&hdev->lock); - - venus_free(hdev, &hdev->ifaceq_table); - venus_free(hdev, &hdev->sfr); - - memset(hdev->queues, 0, sizeof(hdev->queues)); - memset(&hdev->ifaceq_table, 0, sizeof(hdev->ifaceq_table)); - memset(&hdev->sfr, 0, sizeof(hdev->sfr)); - - mutex_unlock(&hdev->lock); -} - -static int venus_interface_queues_init(struct venus_hfi_device *hdev) -{ - struct hfi_queue_table_header *tbl_hdr; - struct iface_queue *queue; - struct hfi_sfr *sfr; - struct mem_desc desc = {0}; - unsigned int offset; - unsigned int i; - int ret; - - ret = venus_alloc(hdev, &desc, ALIGNED_QUEUE_SIZE); - if (ret) - return ret; - - hdev->ifaceq_table = desc; - offset = IFACEQ_TABLE_SIZE; - - for (i = 0; i < IFACEQ_NUM; i++) { - queue = &hdev->queues[i]; - queue->qmem.da = desc.da + offset; - queue->qmem.kva = desc.kva + offset; - queue->qmem.size = IFACEQ_QUEUE_SIZE; - offset += queue->qmem.size; - queue->qhdr = - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i); - - venus_set_qhdr_defaults(queue->qhdr); - - queue->qhdr->start_addr = queue->qmem.da; - - if (i == IFACEQ_CMD_IDX) - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q; - else if (i == IFACEQ_MSG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q; - else if (i == IFACEQ_DBG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q; - } - - tbl_hdr = hdev->ifaceq_table.kva; - tbl_hdr->version = 0; - tbl_hdr->size = IFACEQ_TABLE_SIZE; - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header); - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header); - tbl_hdr->num_q = IFACEQ_NUM; - tbl_hdr->num_active_q = IFACEQ_NUM; - - /* - * Set receive request to zero on debug queue as there is no - * need of interrupt from video hardware for debug messages - */ - queue = &hdev->queues[IFACEQ_DBG_IDX]; - queue->qhdr->rx_req = 0; - - ret = venus_alloc(hdev, &desc, ALIGNED_SFR_SIZE); - if (ret) { - hdev->sfr.da = 0; - } else { - hdev->sfr = desc; - sfr = hdev->sfr.kva; - sfr->buf_size = ALIGNED_SFR_SIZE; - } - - /* ensure table and queue header structs are settled in memory */ - wmb(); - - return 0; + qhdr->qhdr_status = 1; + qhdr->qhdr_type = IFACEQ_DFLT_QHDR; + qhdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4; + qhdr->qhdr_pkt_size = 0; + qhdr->qhdr_rx_wm = 1; + qhdr->qhdr_tx_wm = 1; + qhdr->qhdr_rx_req = 1; + qhdr->qhdr_tx_req = 0; + qhdr->qhdr_rx_irq_status = 0; + qhdr->qhdr_tx_irq_status = 0; + qhdr->qhdr_read_idx = 0; + qhdr->qhdr_write_idx = 0; } static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug) @@ -864,7 +510,7 @@ static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug) pkt_sys_debug_config(pkt, HFI_DEBUG_MODE_QUEUE, debug); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode) @@ -876,7 +522,7 @@ static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode) pkt_sys_coverage_config(pkt, mode); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_idle_message(struct venus_hfi_device *hdev, @@ -892,7 +538,7 @@ static int venus_sys_set_idle_message(struct venus_hfi_device *hdev, pkt_sys_idle_indicator(pkt, enable); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_power_control(struct venus_hfi_device *hdev, @@ -905,7 +551,7 @@ static int venus_sys_set_power_control(struct venus_hfi_device *hdev, pkt_sys_power_control(pkt, enable); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev) @@ -919,7 +565,7 @@ static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev) pkt_sys_ubwc_config(pkt, res->ubwc_conf); - ret = venus_iface_cmdq_write(hdev, pkt, false); + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); if (ret) return ret; @@ -938,7 +584,7 @@ static int venus_get_queue_size(struct venus_hfi_device *hdev, if (!qhdr) return -EINVAL; - return abs(qhdr->read_idx - qhdr->write_idx); + return abs(qhdr->qhdr_read_idx - qhdr->qhdr_write_idx); } static int venus_sys_set_default_properties(struct venus_hfi_device *hdev) @@ -980,7 +626,7 @@ static int venus_session_cmd(struct venus_inst *inst, u32 pkt_type, bool sync) pkt_session_cmd(&pkt, pkt_type, inst); - return venus_iface_cmdq_write(hdev, &pkt, sync); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, sync); } static void venus_flush_debug_queue(struct venus_hfi_device *hdev) @@ -1010,7 +656,7 @@ static int venus_prepare_power_collapse(struct venus_hfi_device *hdev, pkt_sys_pc_prep(&pkt); - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); if (ret) return ret; @@ -1047,7 +693,7 @@ static int venus_are_queues_empty(struct venus_hfi_device *hdev) static void venus_sfr_print(struct venus_hfi_device *hdev) { struct device *dev = hdev->core->dev; - struct hfi_sfr *sfr = hdev->sfr.kva; + struct hfi_sfr *sfr = hdev->sfr.kernel_vaddr; void *p; if (!sfr) @@ -1163,13 +809,13 @@ static int venus_core_init(struct venus_core *core) venus_set_state(hdev, VENUS_STATE_INIT); - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); if (ret) return ret; pkt_sys_image_version(&version_pkt); - ret = venus_iface_cmdq_write(hdev, &version_pkt, false); + ret = venus_iface_cmdq_write(hdev, &version_pkt, version_pkt.hdr.size, false); if (ret) dev_warn(dev, "failed to send image version pkt to fw\n"); @@ -1198,7 +844,7 @@ static int venus_core_ping(struct venus_core *core, u32 cookie) pkt_sys_ping(&pkt, cookie); - return venus_iface_cmdq_write(hdev, &pkt, false); + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); } static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type) @@ -1211,7 +857,7 @@ static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type) if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, false); + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); } static int venus_session_init(struct venus_inst *inst, u32 session_type, @@ -1229,7 +875,7 @@ static int venus_session_init(struct venus_inst *inst, u32 session_type, if (ret) goto err; - ret = venus_iface_cmdq_write(hdev, &pkt, true); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true); if (ret) goto err; @@ -1272,7 +918,7 @@ static int venus_session_flush(struct venus_inst *inst, u32 flush_mode) if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, true); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true); } static int venus_session_start(struct venus_inst *inst) @@ -1304,7 +950,7 @@ static int venus_session_etb(struct venus_inst *inst, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false); } else if (session_type == VIDC_SESSION_TYPE_ENC) { struct hfi_session_empty_buffer_uncompressed_plane0_pkt pkt; @@ -1312,7 +958,7 @@ static int venus_session_etb(struct venus_inst *inst, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false); } else { ret = -EINVAL; } @@ -1331,7 +977,7 @@ static int venus_session_ftb(struct venus_inst *inst, if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, false); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false); } static int venus_session_set_buffers(struct venus_inst *inst, @@ -1351,7 +997,7 @@ static int venus_session_set_buffers(struct venus_inst *inst, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); } static int venus_session_unset_buffers(struct venus_inst *inst, @@ -1371,7 +1017,7 @@ static int venus_session_unset_buffers(struct venus_inst *inst, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, true); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, true); } static int venus_session_load_res(struct venus_inst *inst) @@ -1398,7 +1044,7 @@ static int venus_session_parse_seq_hdr(struct venus_inst *inst, u32 seq_hdr, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, pkt, false); + ret = venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); if (ret) return ret; @@ -1419,7 +1065,7 @@ static int venus_session_get_seq_hdr(struct venus_inst *inst, u32 seq_hdr, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); } static int venus_session_set_property(struct venus_inst *inst, u32 ptype, @@ -1438,7 +1084,7 @@ static int venus_session_set_property(struct venus_inst *inst, u32 ptype, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); } static int venus_session_get_property(struct venus_inst *inst, u32 ptype) @@ -1451,7 +1097,7 @@ static int venus_session_get_property(struct venus_inst *inst, u32 ptype) if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, true); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true); } static int venus_resume(struct venus_core *core) @@ -1685,7 +1331,11 @@ void venus_hfi_destroy(struct venus_core *core) struct venus_hfi_device *hdev = to_hfi_priv(core); core->priv = NULL; - venus_interface_queues_release(hdev); + mutex_lock(&hdev->lock); + hfi_queue_deinit(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr, + &hdev->queues[0], &hdev->queues[1], + &hdev->queues[2]); + mutex_unlock(&hdev->lock); mutex_destroy(&hdev->lock); kfree(hdev); core->ops = NULL; @@ -1707,7 +1357,9 @@ int venus_hfi_create(struct venus_core *core) core->priv = hdev; core->ops = &venus_hfi_ops; - ret = venus_interface_queues_init(hdev); + ret = hfi_queue_init(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr, + &hdev->queues[0], &hdev->queues[1], + &hdev->queues[2], &hdev->core); if (ret) goto err_kfree; @@ -1724,7 +1376,7 @@ void venus_hfi_queues_reinit(struct venus_core *core) { struct venus_hfi_device *hdev = to_hfi_priv(core); struct hfi_queue_table_header *tbl_hdr; - struct iface_queue *queue; + struct iface_q_info *queue; struct hfi_sfr *sfr; unsigned int i; @@ -1733,36 +1385,36 @@ void venus_hfi_queues_reinit(struct venus_core *core) for (i = 0; i < IFACEQ_NUM; i++) { queue = &hdev->queues[i]; queue->qhdr = - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i); + IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kernel_vaddr, i); venus_set_qhdr_defaults(queue->qhdr); - queue->qhdr->start_addr = queue->qmem.da; + queue->qhdr->qhdr_start_addr = queue->q_array.device_addr; if (i == IFACEQ_CMD_IDX) - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q; + queue->qhdr->qhdr_type |= HFI_HOST_TO_CTRL_CMD_Q; else if (i == IFACEQ_MSG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q; + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_MSG_Q; else if (i == IFACEQ_DBG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q; + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_DBG_Q; } - tbl_hdr = hdev->ifaceq_table.kva; - tbl_hdr->version = 0; - tbl_hdr->size = IFACEQ_TABLE_SIZE; - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header); - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header); - tbl_hdr->num_q = IFACEQ_NUM; - tbl_hdr->num_active_q = IFACEQ_NUM; + tbl_hdr = hdev->ifaceq_table.kernel_vaddr; + tbl_hdr->qtbl_version = 0; + tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE; + tbl_hdr->qtbl_qhdr0_offset = sizeof(struct hfi_queue_table_header); + tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header); + tbl_hdr->qtbl_num_q = IFACEQ_NUM; + tbl_hdr->qtbl_num_active_q = IFACEQ_NUM; /* * Set receive request to zero on debug queue as there is no * need of interrupt from video hardware for debug messages */ queue = &hdev->queues[IFACEQ_DBG_IDX]; - queue->qhdr->rx_req = 0; + queue->qhdr->qhdr_rx_req = 0; - sfr = hdev->sfr.kva; + sfr = hdev->sfr.kernel_vaddr; sfr->buf_size = ALIGNED_SFR_SIZE; /* ensure table and queue header structs are settled in memory */ From patchwork Mon Dec 18 11:31:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755555 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1AC91A732; Mon, 18 Dec 2023 11:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="LlyFTxRB" Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIB476Z031195; Mon, 18 Dec 2023 11:33:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=HQcOzUhk8zNeXD1GwDBA8FVDCbBz6VQpKL6SA2SwzOY=; b=Ll yFTxRBgaZcpplidbCzjh2ut0w+YX+AToqIH8TOoGO/fZbm1YzgSSAJ1Yf2cn+y0K 4MBs46xG/ynbW+9e5Q8CzCTxmy6Puk842pMxgwdn8Y9ehXGSHU9J4zmwSWavPWQd BUci9q2g5gaKFUn9tu9VRGARqCWqsb0RIZvUvJYyHHDI1XpAy3j4a8NVh7KlNMJF sXcJZq1ZSkT24QZWo7teceC4j7SFKFdkKJ+oziVGqDmVV4fM/tVXhtxHmz6JvjmV 7EeVF1Rg/BgxHu8J08KJIt1fASulsRNS5k1eNOG3+licffEaVOC55jJfyUjx8z9b KuxahDI78AHvJ8z45ijg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2jx0gb98-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:08 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Qx029941; Mon, 18 Dec 2023 11:33:05 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyg8-1; Mon, 18 Dec 2023 11:33:05 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX54W029912; Mon, 18 Dec 2023 11:33:05 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX50W029904; Mon, 18 Dec 2023 11:33:05 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 2691C22BD; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 03/34] media: introduce common helpers for buffer size calculation Date: Mon, 18 Dec 2023 17:01:58 +0530 Message-Id: <1702899149-21321-4-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 2cwTLidpDbCdBte__SHwa2lVRyEoD1tB X-Proofpoint-GUID: 2cwTLidpDbCdBte__SHwa2lVRyEoD1tB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1011 phishscore=0 priorityscore=1501 suspectscore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 mlxscore=0 malwarescore=0 spamscore=0 mlxlogscore=916 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Introduce helper to calculate size of pixel buffer and use in venus driver. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/buffers.c | 103 ++++++++++++ drivers/media/platform/qcom/vcodec/buffers.h | 15 ++ drivers/media/platform/qcom/vcodec/venus/Makefile | 4 +- drivers/media/platform/qcom/vcodec/venus/helpers.c | 172 ++++----------------- drivers/media/platform/qcom/vcodec/venus/helpers.h | 2 +- .../platform/qcom/vcodec/venus/hfi_plat_bufs.h | 4 +- .../platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c | 10 +- drivers/media/platform/qcom/vcodec/venus/vdec.c | 5 +- 8 files changed, 157 insertions(+), 158 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h diff --git a/drivers/media/platform/qcom/vcodec/buffers.c b/drivers/media/platform/qcom/vcodec/buffers.c new file mode 100644 index 0000000..58ab3b0 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/buffers.c @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#include + +#include "buffers.h" + +u32 video_raw_buffer_size(u32 colorformat, + u32 pix_width, + u32 pix_height) +{ + u32 size = 0; + u32 y_plane, uv_plane, y_stride, + uv_stride, y_sclines, uv_sclines; + u32 y_ubwc_plane = 0, uv_ubwc_plane = 0; + u32 y_meta_stride = 0, y_meta_scanlines = 0; + u32 uv_meta_stride = 0, uv_meta_scanlines = 0; + u32 y_meta_plane = 0, uv_meta_plane = 0; + + if (!pix_width || !pix_height) + goto invalid_input; + + switch (colorformat) { + case V4L2_PIX_FMT_NV12: + case V4L2_PIX_FMT_NV21: + y_stride = ALIGN(pix_width, 128); + uv_stride = ALIGN(pix_width, 128); + y_sclines = ALIGN(pix_height, 32); + uv_sclines = ALIGN((pix_height + 1) >> 1, 16); + y_plane = y_stride * y_sclines; + uv_plane = uv_stride * uv_sclines; + size = y_plane + uv_plane; + break; + case V4L2_PIX_FMT_QC08C: + y_stride = ALIGN(pix_width, 128); + uv_stride = ALIGN(pix_width, 128); + y_sclines = ALIGN(pix_height, 32); + uv_sclines = ALIGN((pix_height + 1) >> 1, 32); + y_meta_stride = ALIGN(DIV_ROUND_UP(pix_width, 32), 64); + uv_meta_stride = ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 16), 64); + y_ubwc_plane = + ALIGN(y_stride * y_sclines, 4096); + uv_ubwc_plane = + ALIGN(uv_stride * uv_sclines, 4096); + y_meta_scanlines = + ALIGN(DIV_ROUND_UP(pix_height, 8), 16); + y_meta_plane = + ALIGN(y_meta_stride * y_meta_scanlines, 4096); + uv_meta_scanlines = + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 8), 16); + uv_meta_plane = + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096); + size = (y_ubwc_plane + uv_ubwc_plane + y_meta_plane + + uv_meta_plane); + break; + case V4L2_PIX_FMT_QC10C: + y_stride = + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256); + uv_stride = + ALIGN(ALIGN(pix_width, 192) * 4 / 3, 256); + y_sclines = + ALIGN(pix_height, 16); + uv_sclines = + ALIGN((pix_height + 1) >> 1, 16); + y_ubwc_plane = + ALIGN(y_stride * y_sclines, 4096); + uv_ubwc_plane = + ALIGN(uv_stride * uv_sclines, 4096); + y_meta_stride = + ALIGN(DIV_ROUND_UP(pix_width, 48), 64); + y_meta_scanlines = + ALIGN(DIV_ROUND_UP(pix_height, 4), 16); + y_meta_plane = + ALIGN(y_meta_stride * y_meta_scanlines, 4096); + uv_meta_stride = + ALIGN(DIV_ROUND_UP((pix_width + 1) >> 1, 24), 64); + uv_meta_scanlines = + ALIGN(DIV_ROUND_UP((pix_height + 1) >> 1, 4), 16); + uv_meta_plane = + ALIGN(uv_meta_stride * uv_meta_scanlines, 4096); + + size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + + uv_meta_plane; + break; + case V4L2_PIX_FMT_P010: + y_stride = ALIGN(pix_width * 2, 128); + uv_stride = ALIGN(pix_width * 2, 128); + y_sclines = ALIGN(pix_height, 32); + uv_sclines = ALIGN((pix_height + 1) >> 1, 16); + y_plane = y_stride * y_sclines; + uv_plane = uv_stride * uv_sclines; + + size = y_plane + uv_plane; + break; + default: + break; + } + +invalid_input: + + return ALIGN(size, 4096); +} diff --git a/drivers/media/platform/qcom/vcodec/buffers.h b/drivers/media/platform/qcom/vcodec/buffers.h new file mode 100644 index 0000000..ac1d052 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/buffers.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _BUFFERS_H_ +#define _BUFFERS_H_ + +#include + +u32 video_raw_buffer_size(u32 colorformat, + u32 pix_width, + u32 pix_height); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile index 1941ef4..6abd54a 100644 --- a/drivers/media/platform/qcom/vcodec/venus/Makefile +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile @@ -1,8 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 # Makefile for Qualcomm Venus driver -venus-core-objs += ../firmware.o \ - ../hfi_queue.o +venus-core-objs += ../firmware.o ../hfi_queue.o ../buffers.o venus-core-objs += core.o helpers.o firmware_no_tz.o \ hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \ @@ -10,6 +9,7 @@ venus-core-objs += core.o helpers.o firmware_no_tz.o \ hfi_platform.o hfi_platform_v4.o \ hfi_platform_v6.o hfi_plat_bufs_v6.o \ +venus-dec-objs += ../buffers.o venus-dec-objs += vdec.o vdec_ctrls.o venus-enc-objs += venc.o venc_ctrls.o diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c index 8295542..95e4424 100644 --- a/drivers/media/platform/qcom/vcodec/venus/helpers.c +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.c @@ -12,6 +12,7 @@ #include #include +#include "../buffers.h" #include "core.h" #include "helpers.h" #include "hfi_helper.h" @@ -616,6 +617,27 @@ static u32 to_hfi_raw_fmt(u32 v4l2_fmt) return 0; } +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt) +{ + switch (hfi_color_fmt) { + case HFI_COLOR_FORMAT_NV12: + return V4L2_PIX_FMT_NV12; + case HFI_COLOR_FORMAT_NV21: + return V4L2_PIX_FMT_NV21; + case HFI_COLOR_FORMAT_NV12_UBWC: + return V4L2_PIX_FMT_QC08C; + case HFI_COLOR_FORMAT_YUV420_TP10_UBWC: + return V4L2_PIX_FMT_QC10C; + case HFI_COLOR_FORMAT_P010: + return V4L2_PIX_FMT_P010; + default: + break; + } + + return 0; +} +EXPORT_SYMBOL_GPL(to_v4l2_raw_fmt); + static int platform_get_bufreq(struct venus_inst *inst, u32 buftype, struct hfi_buffer_requirements *req) { @@ -639,20 +661,20 @@ static int platform_get_bufreq(struct venus_inst *inst, u32 buftype, params.out_width = inst->out_width; params.out_height = inst->out_height; params.codec = inst->fmt_out->pixfmt; - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_cap->pixfmt); + params.color_fmt = inst->fmt_cap->pixfmt; params.dec.max_mbs_per_frame = mbs_per_frame_max(inst); params.dec.buffer_size_limit = 0; params.dec.is_secondary_output = inst->opb_buftype == HFI_BUFFER_OUTPUT2; if (params.dec.is_secondary_output) - params.hfi_dpb_color_fmt = inst->dpb_fmt; + params.dpb_color_fmt = to_v4l2_raw_fmt(inst->dpb_fmt); params.dec.is_interlaced = inst->pic_struct != HFI_INTERLACE_FRAME_PROGRESSIVE; } else { params.width = inst->out_width; params.height = inst->out_height; params.codec = inst->fmt_cap->pixfmt; - params.hfi_color_fmt = to_hfi_raw_fmt(inst->fmt_out->pixfmt); + params.color_fmt = inst->fmt_out->pixfmt; params.enc.work_mode = VIDC_WORK_MODE_2; params.enc.rc_type = HFI_RATE_CONTROL_OFF; if (enc_ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) @@ -942,146 +964,10 @@ int venus_helper_set_profile_level(struct venus_inst *inst, u32 profile, u32 lev } EXPORT_SYMBOL_GPL(venus_helper_set_profile_level); -static u32 get_framesize_raw_nv12(u32 width, u32 height) -{ - u32 y_stride, uv_stride, y_plane; - u32 y_sclines, uv_sclines, uv_plane; - u32 size; - - y_stride = ALIGN(width, 128); - uv_stride = ALIGN(width, 128); - y_sclines = ALIGN(height, 32); - uv_sclines = ALIGN(((height + 1) >> 1), 16); - - y_plane = y_stride * y_sclines; - uv_plane = uv_stride * uv_sclines + SZ_4K; - size = y_plane + uv_plane + SZ_8K; - - return ALIGN(size, SZ_4K); -} - -static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height) -{ - u32 y_meta_stride, y_meta_plane; - u32 y_stride, y_plane; - u32 uv_meta_stride, uv_meta_plane; - u32 uv_stride, uv_plane; - u32 extradata = SZ_16K; - - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64); - y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(height, 8), 16); - y_meta_plane = ALIGN(y_meta_plane, SZ_4K); - - y_stride = ALIGN(width, 128); - y_plane = ALIGN(y_stride * ALIGN(height, 32), SZ_4K); - - uv_meta_stride = ALIGN(DIV_ROUND_UP(width / 2, 16), 64); - uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(height / 2, 8), 16); - uv_meta_plane = ALIGN(uv_meta_plane, SZ_4K); - - uv_stride = ALIGN(width, 128); - uv_plane = ALIGN(uv_stride * ALIGN(height / 2, 32), SZ_4K); - - return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane + - max(extradata, y_stride * 48), SZ_4K); -} - -static u32 get_framesize_raw_p010(u32 width, u32 height) -{ - u32 y_plane, uv_plane, y_stride, uv_stride, y_sclines, uv_sclines; - - y_stride = ALIGN(width * 2, 128); - uv_stride = ALIGN(width * 2, 128); - y_sclines = ALIGN(height, 32); - uv_sclines = ALIGN((height + 1) >> 1, 16); - y_plane = y_stride * y_sclines; - uv_plane = uv_stride * uv_sclines; - - return ALIGN((y_plane + uv_plane), SZ_4K); -} - -static u32 get_framesize_raw_p010_ubwc(u32 width, u32 height) -{ - u32 y_stride, uv_stride, y_sclines, uv_sclines; - u32 y_ubwc_plane, uv_ubwc_plane; - u32 y_meta_stride, y_meta_scanlines; - u32 uv_meta_stride, uv_meta_scanlines; - u32 y_meta_plane, uv_meta_plane; - u32 size; - - y_stride = ALIGN(width * 2, 256); - uv_stride = ALIGN(width * 2, 256); - y_sclines = ALIGN(height, 16); - uv_sclines = ALIGN((height + 1) >> 1, 16); - - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K); - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K); - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64); - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16); - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K); - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64); - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16); - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K); - - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane; - - return ALIGN(size, SZ_4K); -} - -static u32 get_framesize_raw_yuv420_tp10_ubwc(u32 width, u32 height) -{ - u32 y_stride, uv_stride, y_sclines, uv_sclines; - u32 y_ubwc_plane, uv_ubwc_plane; - u32 y_meta_stride, y_meta_scanlines; - u32 uv_meta_stride, uv_meta_scanlines; - u32 y_meta_plane, uv_meta_plane; - u32 extradata = SZ_16K; - u32 size; - - y_stride = ALIGN(width * 4 / 3, 256); - uv_stride = ALIGN(width * 4 / 3, 256); - y_sclines = ALIGN(height, 16); - uv_sclines = ALIGN((height + 1) >> 1, 16); - - y_ubwc_plane = ALIGN(y_stride * y_sclines, SZ_4K); - uv_ubwc_plane = ALIGN(uv_stride * uv_sclines, SZ_4K); - y_meta_stride = ALIGN(DIV_ROUND_UP(width, 48), 64); - y_meta_scanlines = ALIGN(DIV_ROUND_UP(height, 4), 16); - y_meta_plane = ALIGN(y_meta_stride * y_meta_scanlines, SZ_4K); - uv_meta_stride = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64); - uv_meta_scanlines = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16); - uv_meta_plane = ALIGN(uv_meta_stride * uv_meta_scanlines, SZ_4K); - - size = y_ubwc_plane + uv_ubwc_plane + y_meta_plane + uv_meta_plane; - size += max(extradata + SZ_8K, y_stride * 48); - - return ALIGN(size, SZ_4K); -} - -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height) -{ - switch (hfi_fmt) { - case HFI_COLOR_FORMAT_NV12: - case HFI_COLOR_FORMAT_NV21: - return get_framesize_raw_nv12(width, height); - case HFI_COLOR_FORMAT_NV12_UBWC: - return get_framesize_raw_nv12_ubwc(width, height); - case HFI_COLOR_FORMAT_P010: - return get_framesize_raw_p010(width, height); - case HFI_COLOR_FORMAT_P010_UBWC: - return get_framesize_raw_p010_ubwc(width, height); - case HFI_COLOR_FORMAT_YUV420_TP10_UBWC: - return get_framesize_raw_yuv420_tp10_ubwc(width, height); - default: - return 0; - } -} -EXPORT_SYMBOL_GPL(venus_helper_get_framesz_raw); - u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height) { - u32 hfi_fmt, sz; bool compressed; + u32 sz; switch (v4l2_fmt) { case V4L2_PIX_FMT_MPEG: @@ -1112,11 +998,7 @@ u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height) return ALIGN(sz, SZ_4K); } - hfi_fmt = to_hfi_raw_fmt(v4l2_fmt); - if (!hfi_fmt) - return 0; - - return venus_helper_get_framesz_raw(hfi_fmt, width, height); + return video_raw_buffer_size(v4l2_fmt, width, height); } EXPORT_SYMBOL_GPL(venus_helper_get_framesz); diff --git a/drivers/media/platform/qcom/vcodec/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h index 358e4f3..9b72d18 100644 --- a/drivers/media/platform/qcom/vcodec/venus/helpers.h +++ b/drivers/media/platform/qcom/vcodec/venus/helpers.h @@ -11,6 +11,7 @@ struct venus_inst; struct venus_core; +u32 to_v4l2_raw_fmt(u32 hfi_color_fmt); bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt); struct vb2_v4l2_buffer *venus_helper_find_buf(struct venus_inst *inst, unsigned int type, u32 idx); @@ -29,7 +30,6 @@ void venus_helper_m2m_device_run(void *priv); void venus_helper_m2m_job_abort(void *priv); int venus_helper_get_bufreq(struct venus_inst *inst, u32 type, struct hfi_buffer_requirements *req); -u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height); u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height); int venus_helper_set_input_resolution(struct venus_inst *inst, unsigned int width, unsigned int height); diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h index 25e6074..20f684e 100644 --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h @@ -15,8 +15,8 @@ struct hfi_plat_buffers_params { u32 out_width; u32 out_height; u32 codec; - u32 hfi_color_fmt; - u32 hfi_dpb_color_fmt; + u32 color_fmt; + u32 dpb_color_fmt; enum hfi_version version; u32 num_vpp_pipes; union { diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c index f5a6559..3e06516 100644 --- a/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c @@ -6,6 +6,7 @@ #include #include +#include "../buffers.h" #include "hfi.h" #include "hfi_plat_bufs.h" #include "helpers.h" @@ -1233,13 +1234,11 @@ static int bufreq_dec(struct hfi_plat_buffers_params *params, u32 buftype, buffer_size_limit); } else if (buftype == HFI_BUFFER_OUTPUT || buftype == HFI_BUFFER_OUTPUT2) { hfi_bufreq_set_count_min(bufreq, version, out_min_count); - bufreq->size = - venus_helper_get_framesz_raw(params->hfi_color_fmt, + bufreq->size = video_raw_buffer_size(params->color_fmt, out_width, out_height); if (buftype == HFI_BUFFER_OUTPUT && params->dec.is_secondary_output) - bufreq->size = - venus_helper_get_framesz_raw(params->hfi_dpb_color_fmt, + bufreq->size = video_raw_buffer_size(params->dpb_color_fmt, out_width, out_height); } else if (buftype == HFI_BUFFER_INTERNAL_SCRATCH(version)) { bufreq->size = dec_ops->scratch(width, height, is_interlaced); @@ -1297,8 +1296,7 @@ static int bufreq_enc(struct hfi_plat_buffers_params *params, u32 buftype, if (buftype == HFI_BUFFER_INPUT) { hfi_bufreq_set_count_min(bufreq, version, MIN_INPUT_BUFFERS); - bufreq->size = - venus_helper_get_framesz_raw(params->hfi_color_fmt, + bufreq->size = video_raw_buffer_size(params->color_fmt, width, height); } else if (buftype == HFI_BUFFER_OUTPUT || buftype == HFI_BUFFER_OUTPUT2) { diff --git a/drivers/media/platform/qcom/vcodec/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c index dbf305c..e6316be 100644 --- a/drivers/media/platform/qcom/vcodec/venus/vdec.c +++ b/drivers/media/platform/qcom/vcodec/venus/vdec.c @@ -15,6 +15,7 @@ #include #include +#include "../buffers.h" #include "hfi_venus_io.h" #include "hfi_parser.h" #include "core.h" @@ -777,9 +778,9 @@ static int vdec_output_conf(struct venus_inst *inst) return ret; inst->output_buf_size = - venus_helper_get_framesz_raw(out_fmt, width, height); + video_raw_buffer_size(to_v4l2_raw_fmt(out_fmt), width, height); inst->output2_buf_size = - venus_helper_get_framesz_raw(out2_fmt, width, height); + video_raw_buffer_size(to_v4l2_raw_fmt(out2_fmt), width, height); if (is_ubwc_fmt(out_fmt)) { inst->opb_buftype = HFI_BUFFER_OUTPUT2; From patchwork Mon Dec 18 11:31:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755557 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED10B199D3; Mon, 18 Dec 2023 11:33:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="AF4EEx1d" Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsE2N014268; Mon, 18 Dec 2023 11:33:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=oKpFY6BB+SCt9JrJ+9BOpkSvsgCgf7eGhPjITeKw9Ms=; b=AF 4EEx1d4M9v8Xjg6FUaf5Mxnm++SSuvk0ocWFjzZ8Gj4Yr/LRJgvxOujZotcVtQCA 2tlP7kCLEMYoK8yEw4qhDgK4a+LAk4ASwN+Q5dUw/pt15SjDWeM6mpU0LMxKieH1 WeVnjxTNkBhHX6fHp99+z8XOEFMtVSnsC6ooiXA+/MG9Y0lYrCRS9DH0XKNcgCsC dBHm8oNw+CixhuHbj6T8L5zvwK6SYcW03vng6XnfFpQLcm2YGqAElo6CsMg0XSc+ pqgLsjje8DVEEBlqe+wu1ctMLo6Cc/cZcQol15/pIk00oXY6Kot3OBPAiHytiKyw AutiCNZx0E6/p1x2riag== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2jx0gb99-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:09 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmpL028250; Mon, 18 Dec 2023 11:33:05 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyga-1; Mon, 18 Dec 2023 11:33:05 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Am029911; Mon, 18 Dec 2023 11:33:05 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX5xd029903; Mon, 18 Dec 2023 11:33:05 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 290F822C3; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 04/34] dt-bindings: media: Add sm8550 dt schema Date: Mon, 18 Dec 2023 17:01:59 +0530 Message-Id: <1702899149-21321-5-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: CUNoS0pzTmcp4Xvd6MVOsEby1DWN8UAv X-Proofpoint-GUID: CUNoS0pzTmcp4Xvd6MVOsEby1DWN8UAv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 phishscore=0 priorityscore=1501 suspectscore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 mlxscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add a schema description for the iris video encoder/decoder on sm8550. Signed-off-by: Dikshita Agarwal --- .../bindings/media/qcom,sm8550-iris.yaml | 177 +++++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml diff --git a/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml new file mode 100644 index 0000000..a3d9233 --- /dev/null +++ b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml @@ -0,0 +1,177 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/qcom,sm8550-iris.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Qualcomm IRIS video encode and decode accelerators + +maintainers: + - Vikash Garodia + - Dikshita Agarwal + +description: + The Iris video processing unit is a video encode and decode accelerator + present on Qualcomm platforms. + +properties: + compatible: + oneOf: + - enum: + - qcom,sm8550-iris + + reg: + maxItems: 2 + + interrupts: + maxItems: 1 + + power-domains: + minItems: 2 + maxItems: 4 + + power-domain-names: + oneOf: + - items: + - const: iris-ctl + - const: vcodec + - const: mxc + - const: mmcx + + operating-points-v2: true + + clocks: + maxItems: 3 + + clock-names: + items: + - const: gcc_video_axi0 + - const: core_clk + - const: vcodec_core + + interconnects: + maxItems: 2 + + interconnect-names: + items: + - const: iris-cnoc + - const: iris-ddr + + memory-region: + maxItems: 1 + + resets: + maxItems: 1 + + reset-names: + items: + - const: video_axi_reset + + iommus: + maxItems: 2 + + dma-coherent: true + + opp-table: + type: object + +required: + - compatible + - reg + - interrupts + - power-domains + - power-domain-names + - clocks + - clock-names + - interconnects + - interconnect-names + - memory-region + - resets + - reset-names + - iommus + - dma-coherent + - opp-table + +additionalProperties: false + +examples: + - | + #include + #include + #include + #include + #include + #include + #include + + iris: video-codec@aa00000 { + compatible = "qcom,sm8550-iris"; + + reg = <0 0x0aa00000 0 0xf0000>; + interrupts = ; + + power-domains = <&videocc VIDEO_CC_MVS0C_GDSC>, + <&videocc VIDEO_CC_MVS0_GDSC>, + <&rpmhpd SM8550_MXC>, + <&rpmhpd SM8550_MMCX>; + power-domain-names = "iris-ctl", "vcodec", "mxc", "mmcx"; + operating-points-v2 = <&iris_opp_table>; + + clocks = <&gcc GCC_VIDEO_AXI0_CLK>, + <&videocc VIDEO_CC_MVS0C_CLK>, + <&videocc VIDEO_CC_MVS0_CLK>; + clock-names = "gcc_video_axi0", "core_clk", "vcodec_core"; + + interconnects = <&gem_noc MASTER_APPSS_PROC QCOM_ICC_TAG_ALWAYS + &config_noc SLAVE_VENUS_CFG QCOM_ICC_TAG_ALWAYS>, + <&mmss_noc MASTER_VIDEO QCOM_ICC_TAG_ALWAYS + &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>; + interconnect-names = "iris-cnoc", "iris-ddr"; + + /* FW load region */ + memory-region = <&video_mem>; + + resets = <&gcc GCC_VIDEO_AXI0_CLK_ARES>; + reset-names = "video_axi_reset"; + + iommus = <&apps_smmu 0x1940 0x0000>, + <&apps_smmu 0x1947 0x0000>; + dma-coherent; + + status = "disabled"; + + iris_opp_table: opp-table { + compatible = "operating-points-v2"; + + opp-240000000 { + opp-hz = /bits/ 64 <240000000>; + required-opps = <&rpmhpd_opp_svs>, + <&rpmhpd_opp_low_svs>; + }; + + opp-338000000 { + opp-hz = /bits/ 64 <338000000>; + required-opps = <&rpmhpd_opp_svs>, + <&rpmhpd_opp_svs>; + }; + + opp-366000000 { + opp-hz = /bits/ 64 <366000000>; + required-opps = <&rpmhpd_opp_svs_l1>, + <&rpmhpd_opp_svs_l1>; + }; + + opp-444000000 { + opp-hz = /bits/ 64 <444000000>; + required-opps = <&rpmhpd_opp_turbo>, + <&rpmhpd_opp_turbo>; + }; + + opp-533333334 { + opp-hz = /bits/ 64 <533333334>; + required-opps = <&rpmhpd_opp_turbo_l1>, + <&rpmhpd_opp_turbo_l1>; + }; + }; + }; +... From patchwork Mon Dec 18 11:32:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755547 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60B6D1C6BD; Mon, 18 Dec 2023 11:40:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Cpx1liZf" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsS2I005049; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=r/9F5O9W8VqnTkFkhoN8gND2XXGs+SdpW/9jGlmW4RE=; b=Cp x1liZfv9f9Quf3Ipr/LRPZ5AV3kA0HCRvCG4qhvq7Mpcj3/35+WsSwljGy8NQ101 qEi5l6WA2SoxGN8RdbDFOmw/78voCTUJZiLUCFv5l4kpd19H6xFG7abtcyplWJDr XpSxruRSSr/Ozx/V3SxE7OzyDj58qxwO+GJPx0NSFIPKfSULw1Ve3c+oxX/4/6cX I8ddz5dfDNG3d/g2NcO1YQiFlEcSFEicE1bSwLo64oGEl4BgGaMCtiAtQtY/DfZ4 1t1PtNzYtXcnGb57eA8x8j9lQAAo8rsJpbthoI0k041jHBg4a5V/Fxxf/5RYMDMJ h+xRR7pijdhEuN5/43tQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4y2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX6PQ029956; Mon, 18 Dec 2023 11:40:01 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ad-6; Mon, 18 Dec 2023 11:40:01 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Ao029911; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6q5029965; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 2BC7722C4; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 05/34] media: MAINTAINERS: Add Qualcomm Iris video accelerator driver Date: Mon, 18 Dec 2023 17:02:00 +0530 Message-Id: <1702899149-21321-6-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 5Kw-qdSZgFM-7RCvLYF8-hwnu7zGoX0c X-Proofpoint-GUID: 5Kw-qdSZgFM-7RCvLYF8-hwnu7zGoX0c X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=822 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add an entry for Iris video encoder/decoder accelerator driver. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- MAINTAINERS | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 10a981a..a727144 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -17980,6 +17980,17 @@ S: Maintained F: Documentation/devicetree/bindings/regulator/vqmmc-ipq4019-regulator.yaml F: drivers/regulator/vqmmc-ipq4019-regulator.c +QUALCOMM IRIS VIDEO ACCELERATOR DRIVER +M: Vikash Garodia +M: Dikshita Agarwal +R: Abhinav Kumar +L: linux-media@vger.kernel.org +L: linux-arm-msm@vger.kernel.org +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/devicetree/bindings/media/qcom,*-iris.yaml +F: drivers/media/platform/qcom/iris/ + QUALCOMM NAND CONTROLLER DRIVER M: Manivannan Sadhasivam L: linux-mtd@lists.infradead.org From patchwork Mon Dec 18 11:32:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755551 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AE1A199DB; Mon, 18 Dec 2023 11:36:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="ZFmnun9K" Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsCGw032638; Mon, 18 Dec 2023 11:36:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=LX6X6WF6b2DnvVb0x9SqnHVvPQHNfSxl8FB6K8HP6sw=; b=ZF mnun9KydsAw/asYajUI0JUh42vUrjSf+0MHeYm6RTwMPXdlHC4x7gaVHerzb1KP4 fPSnnQ5cYhW3siUfRYddUSOwSP7IarEXLy4wsgqKKGKpxfuzt2j9JZdyRHnJjV2C qy1ZRRNWIRcwp6i7MFf9vL83N8fCyFhA4RATSx1hYTuNH4k1Oa42dkveDKsMwWzO zU0dHaNWvvYv6NYSQyW/xk0dWaKumFmo375ZtyfV7QLfZ0Uzq/+Wv8QUMNDNfUlx Q+Dr6aBVETfgFzEhbOdZjfyB7ktxnJkh+0oALHkWZaMVH77IA8CtpqcMKczhklt5 ngjdByKMM1+jYQDX89gQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v14xy44cs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:10 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmpM028250; Mon, 18 Dec 2023 11:36:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhc-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7XZ029998; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX62X029968; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 2E22822C6; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 06/34] media: iris: register video device to platform driver Date: Mon, 18 Dec 2023 17:02:01 +0530 Message-Id: <1702899149-21321-7-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: aYFzfkEloCRmcSdpA9dm31aQYrFM9uqY X-Proofpoint-ORIG-GUID: aYFzfkEloCRmcSdpA9dm31aQYrFM9uqY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Iris is a multi pipe based video acceleration hardware block that offloads video stream encoding and decoding from the application processor (AP). It supports H.264 and H.265 encoding and decoding, as well as VP9 decoding. The AP communicates with hardware through a well defined protocol which provides fine-grained and asynchronous control over individual hardware features. This patch introduces basic probe and remove functions. It handles setting up a video device as well as registering it with the V4L2 subsystem. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/Kconfig | 1 + drivers/media/platform/qcom/Makefile | 1 + drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +++ drivers/media/platform/qcom/vcodec/iris/Makefile | 3 + .../media/platform/qcom/vcodec/iris/iris_core.h | 34 +++++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 111 +++++++++++++++++++++ 6 files changed, 163 insertions(+) create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig index e94142f..7c88837 100644 --- a/drivers/media/platform/qcom/Kconfig +++ b/drivers/media/platform/qcom/Kconfig @@ -4,3 +4,4 @@ comment "Qualcomm media platform drivers" source "drivers/media/platform/qcom/camss/Kconfig" source "drivers/media/platform/qcom/vcodec/venus/Kconfig" +source "drivers/media/platform/qcom/vcodec/iris/Kconfig" diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile index 3d2d82b..3c76969 100644 --- a/drivers/media/platform/qcom/Makefile +++ b/drivers/media/platform/qcom/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += camss/ obj-y += vcodec/venus/ +obj-y += vcodec/iris/ diff --git a/drivers/media/platform/qcom/vcodec/iris/Kconfig b/drivers/media/platform/qcom/vcodec/iris/Kconfig new file mode 100644 index 0000000..850a5b4 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/Kconfig @@ -0,0 +1,13 @@ +config VIDEO_QCOM_IRIS + tristate "Qualcomm Iris V4L2 encoder/decoder driver" + depends on VIDEO_DEV + depends on ARCH_QCOM + select QCOM_MDT_LOADER if ARCH_QCOM + select QCOM_SCM + select DMABUF_HEAPS + select DMABUF_HEAPS_SYSTEM + help + This is a V4L2 driver for Qualcomm Iris video accelerator + hardware. It accelerates encoding and decoding operations + on various Qualcomm SoCs. + To compile this driver as a module choose m here. diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile new file mode 100644 index 0000000..5536ae0 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -0,0 +1,3 @@ +iris-objs += iris_probe.o + +obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h new file mode 100644 index 0000000..ab7fcee --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_CORE_H_ +#define _IRIS_CORE_H_ + +#include +#include + +/** + * struct iris_core - holds core parameters valid for all instances + * + * @dev: reference to device structure + * @reg_base: IO memory base address + * @irq: iris irq + * @v4l2_dev: a holder for v4l2 device structure + * @vdev_dec: iris video device structure for decoder + * @v4l2_file_ops: iris v4l2 file ops + * @v4l2_ioctl_ops: iris v4l2 ioctl ops + */ + +struct iris_core { + struct device *dev; + void __iomem *reg_base; + int irq; + struct v4l2_device v4l2_dev; + struct video_device *vdev_dec; + const struct v4l2_file_operations *v4l2_file_ops; + const struct v4l2_ioctl_ops *v4l2_ioctl_ops; +}; + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c new file mode 100644 index 0000000..2e93118 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include + +#include "iris_core.h" + +static int iris_register_video_device(struct iris_core *core) +{ + struct video_device *vdev; + int ret; + + vdev = video_device_alloc(); + if (!vdev) + return -ENOMEM; + + strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); + vdev->release = video_device_release; + vdev->fops = core->v4l2_file_ops; + vdev->ioctl_ops = core->v4l2_ioctl_ops; + vdev->vfl_dir = VFL_DIR_M2M; + vdev->v4l2_dev = &core->v4l2_dev; + vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_dec = vdev; + video_set_drvdata(vdev, core); + + return ret; + +err_vdev_release: + video_device_release(vdev); + + return ret; +} + +static void iris_remove(struct platform_device *pdev) +{ + struct iris_core *core; + + core = platform_get_drvdata(pdev); + if (!core) + return; + + video_unregister_device(core->vdev_dec); + + v4l2_device_unregister(&core->v4l2_dev); +} + +static int iris_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct iris_core *core; + int ret; + + core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL); + if (!core) + return -ENOMEM; + core->dev = dev; + + core->reg_base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(core->reg_base)) + return PTR_ERR(core->reg_base); + + core->irq = platform_get_irq(pdev, 0); + if (core->irq < 0) + return core->irq; + + ret = v4l2_device_register(dev, &core->v4l2_dev); + if (ret) + return ret; + + ret = iris_register_video_device(core); + if (ret) + goto err_v4l2_unreg; + + platform_set_drvdata(pdev, core); + + return ret; + +err_v4l2_unreg: + v4l2_device_unregister(&core->v4l2_dev); + + return ret; +} + +static const struct of_device_id iris_dt_match[] = { + { .compatible = "qcom,sm8550-iris", }, + { }, +}; +MODULE_DEVICE_TABLE(of, iris_dt_match); + +static struct platform_driver qcom_iris_driver = { + .probe = iris_probe, + .remove_new = iris_remove, + .driver = { + .name = "qcom-iris", + .of_match_table = iris_dt_match, + }, +}; + +module_platform_driver(qcom_iris_driver); +MODULE_DESCRIPTION("Qualcomm Iris video driver"); +MODULE_LICENSE("GPL"); From patchwork Mon Dec 18 11:32:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755556 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C5091A727; Mon, 18 Dec 2023 11:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="HHwPggij" Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArv0A012855; Mon, 18 Dec 2023 11:33:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=/cCJZyuRsfh7Pp2thDUrqC1Vjur5pNpx8OcU5IE0yu4=; b=HH wPggijAubWFaot9K8AKI0mqt5u5ZDHKkcq1IeVLSTz5ye96hKl6cXhjAio2MvGnW DtyTb3u21bOhJHu2zvnyKigEZW2e6Yj8LQqTBk0RmXwqkNXylrLioOQPDe9LmqYx C4G+x0c3BkiscXqkyABH5vqtHwJCJi9xcE/XQ7KPpeljjWLYPR5yipN+CI0/UP3j r6gYDX1n/BBfpK96I1mPELaWne6BE5Io7wmEo7fB6LdcghTsLgBuWt6EDXXcuJxd f/acwdQt5PNhkSCZCQirElRtSLBDnG1xSRJ5syxBZ9yz4kQv8+9A2Owrn+8z7AKd DEBHm8bmBTgwkQCYUxSA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2jx0gb9f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:10 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7xI030045; Mon, 18 Dec 2023 11:33:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhq-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7pP030023; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6B5029967; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 308CA22D0; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 07/34] media: iris: initialize power resources Date: Mon, 18 Dec 2023 17:02:02 +0530 Message-Id: <1702899149-21321-8-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: mpnP_L6Zh2IFbIFYHUa1mah8F_DXeVza X-Proofpoint-GUID: mpnP_L6Zh2IFbIFYHUa1mah8F_DXeVza X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 phishscore=0 priorityscore=1501 suspectscore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 mlxscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add support for initializing Iris "resources", which are clocks, interconnects, power domains, reset clocks, and clock frequencies used for Iris hardware. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +- .../media/platform/qcom/vcodec/iris/iris_core.h | 16 ++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 8 + .../media/platform/qcom/vcodec/iris/resources.c | 232 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/resources.h | 36 ++++ 5 files changed, 294 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 5536ae0..0748819 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -1,3 +1,4 @@ -iris-objs += iris_probe.o +iris-objs += iris_probe.o \ + resources.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index ab7fcee..c2bc95d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -19,6 +19,14 @@ * @vdev_dec: iris video device structure for decoder * @v4l2_file_ops: iris v4l2 file ops * @v4l2_ioctl_ops: iris v4l2 ioctl ops + * @bus_tbl: table of iris buses + * @bus_count: count of iris buses + * @power_domain_tbl: table of iris power domains + * @pd_count: count of iris power domains + * @clock_tbl: table of iris clocks + * @clk_count: count of iris clocks + * @reset_tbl: table of iris reset clocks + * @reset_count: count of iris reset clocks */ struct iris_core { @@ -29,6 +37,14 @@ struct iris_core { struct video_device *vdev_dec; const struct v4l2_file_operations *v4l2_file_ops; const struct v4l2_ioctl_ops *v4l2_ioctl_ops; + struct bus_info *bus_tbl; + u32 bus_count; + struct power_domain_info *power_domain_tbl; + u32 pd_count; + struct clock_info *clock_tbl; + u32 clk_count; + struct reset_info *reset_tbl; + u32 reset_count; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 2e93118..7bb9c92 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -8,6 +8,7 @@ #include #include "iris_core.h" +#include "resources.h" static int iris_register_video_device(struct iris_core *core) { @@ -73,6 +74,13 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + ret = init_resources(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init resource failed with %d\n", __func__, ret); + return ret; + } + ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c new file mode 100644 index 0000000..5aebbe4 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/resources.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "iris_core.h" +#include "resources.h" + +static const struct bus_info plat_bus_table[] = { + { NULL, "iris-cnoc", 1000, 1000 }, + { NULL, "iris-ddr", 1000, 15000000 }, +}; + +static const char * const plat_pd_table[] = { "iris-ctl", "vcodec", NULL }; +#define PD_COUNT 2 + +static const char * const plat_opp_pd_table[] = { "mxc", "mmcx", NULL }; +#define OPP_PD_COUNT 2 + +static const struct clock_info plat_clk_table[] = { + { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0, 0 }, + { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0, 0 }, + { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1, 0 }, +}; + +static const char * const plat_clk_reset_table[] = { "video_axi_reset", NULL }; +#define RESET_COUNT 1 + +static void iris_pd_release(void *res) +{ + struct device *pd = (struct device *)res; + + dev_pm_domain_detach(pd, true); +} + +static int iris_pd_get(struct iris_core *core, struct power_domain_info *pdinfo) +{ + int ret; + + pdinfo->genpd_dev = dev_pm_domain_attach_by_name(core->dev, pdinfo->name); + if (IS_ERR_OR_NULL(pdinfo->genpd_dev)) + ret = PTR_ERR(pdinfo->genpd_dev) ? : -ENODATA; + + ret = devm_add_action_or_reset(core->dev, iris_pd_release, (void *)pdinfo->genpd_dev); + if (ret) + return ret; + + return ret; +} + +static void iris_opp_dl_release(void *res) +{ + struct device_link *link = (struct device_link *)res; + + device_link_del(link); +} + +static int iris_opp_dl_get(struct device *dev, struct device *supplier) +{ + u32 flag = DL_FLAG_RPM_ACTIVE | DL_FLAG_PM_RUNTIME | DL_FLAG_STATELESS; + struct device_link *link = NULL; + int ret; + + link = device_link_add(dev, supplier, flag); + if (!link) + return -EINVAL; + + ret = devm_add_action_or_reset(dev, iris_opp_dl_release, (void *)link); + + return ret; +} + +static int init_bus(struct iris_core *core) +{ + struct bus_info *binfo = NULL; + u32 i = 0; + + core->bus_count = ARRAY_SIZE(plat_bus_table); + core->bus_tbl = devm_kzalloc(core->dev, + sizeof(struct bus_info) * core->bus_count, + GFP_KERNEL); + if (!core->bus_tbl) + return -ENOMEM; + + for (i = 0; i < core->bus_count; i++) { + binfo = &core->bus_tbl[i]; + binfo->name = plat_bus_table[i].name; + binfo->bw_min_kbps = plat_bus_table[i].bw_min_kbps; + binfo->bw_max_kbps = plat_bus_table[i].bw_max_kbps; + binfo->icc = devm_of_icc_get(core->dev, binfo->name); + if (IS_ERR(binfo->icc)) { + dev_err(core->dev, + "%s: failed to get bus: %s\n", __func__, binfo->name); + return PTR_ERR(binfo->icc); + } + } + + return 0; +} + +static int init_power_domains(struct iris_core *core) +{ + struct power_domain_info *pdinfo = NULL; + struct device **opp_vdevs = NULL; + int ret; + u32 i; + + core->pd_count = PD_COUNT; + core->power_domain_tbl = devm_kzalloc(core->dev, + sizeof(struct power_domain_info) * core->pd_count, + GFP_KERNEL); + if (!core->power_domain_tbl) + return -ENOMEM; + + for (i = 0; i < core->pd_count; i++) { + pdinfo = &core->power_domain_tbl[i]; + pdinfo->name = plat_pd_table[i]; + ret = iris_pd_get(core, pdinfo); + if (ret) { + dev_err(core->dev, + "%s: failed to get pd: %s\n", __func__, pdinfo->name); + return ret; + } + } + + ret = devm_pm_opp_attach_genpd(core->dev, plat_opp_pd_table, &opp_vdevs); + if (ret) + return ret; + + for (i = 0; i < OPP_PD_COUNT; i++) { + ret = iris_opp_dl_get(core->dev, opp_vdevs[i]); + if (ret) { + dev_err(core->dev, "%s: failed to create dl: %s\n", + __func__, dev_name(opp_vdevs[i])); + return ret; + } + } + + ret = devm_pm_opp_of_add_table(core->dev); + if (ret) { + dev_err(core->dev, "%s: failed to add opp table\n", __func__); + return ret; + } + + return ret; +} + +static int init_clocks(struct iris_core *core) +{ + struct clock_info *cinfo = NULL; + u32 i; + + core->clk_count = ARRAY_SIZE(plat_clk_table); + core->clock_tbl = devm_kzalloc(core->dev, + sizeof(struct clock_info) * core->clk_count, + GFP_KERNEL); + if (!core->clock_tbl) + return -ENOMEM; + + for (i = 0; i < core->clk_count; i++) { + cinfo = &core->clock_tbl[i]; + cinfo->name = plat_clk_table[i].name; + cinfo->clk_id = plat_clk_table[i].clk_id; + cinfo->has_scaling = plat_clk_table[i].has_scaling; + cinfo->clk = devm_clk_get(core->dev, cinfo->name); + if (IS_ERR(cinfo->clk)) { + dev_err(core->dev, + "%s: failed to get clock: %s\n", __func__, cinfo->name); + return PTR_ERR(cinfo->clk); + } + } + + return 0; +} + +static int init_reset_clocks(struct iris_core *core) +{ + struct reset_info *rinfo = NULL; + u32 i = 0; + + core->reset_count = RESET_COUNT; + core->reset_tbl = devm_kzalloc(core->dev, + sizeof(struct reset_info) * core->reset_count, + GFP_KERNEL); + if (!core->reset_tbl) + return -ENOMEM; + + for (i = 0; i < core->reset_count; i++) { + rinfo = &core->reset_tbl[i]; + rinfo->name = plat_clk_reset_table[i]; + rinfo->rst = devm_reset_control_get(core->dev, rinfo->name); + if (IS_ERR(rinfo->rst)) { + dev_err(core->dev, + "%s: failed to get reset clock: %s\n", __func__, rinfo->name); + return PTR_ERR(rinfo->rst); + } + } + + return 0; +} + +int init_resources(struct iris_core *core) +{ + int ret; + + ret = init_bus(core); + if (ret) + return ret; + + ret = init_power_domains(core); + if (ret) + return ret; + + ret = init_clocks(core); + if (ret) + return ret; + + ret = init_reset_clocks(core); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h new file mode 100644 index 0000000..d21bcc7e --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/resources.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _RESOURCES_H_ +#define _RESOURCES_H_ + +struct bus_info { + struct icc_path *icc; + const char *name; + u32 bw_min_kbps; + u32 bw_max_kbps; +}; + +struct power_domain_info { + struct device *genpd_dev; + const char *name; +}; + +struct clock_info { + struct clk *clk; + const char *name; + u32 clk_id; + bool has_scaling; + u64 prev; +}; + +struct reset_info { + struct reset_control *rst; + const char *name; +}; + +int init_resources(struct iris_core *core); + +#endif From patchwork Mon Dec 18 11:32:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755554 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F10261A599; Mon, 18 Dec 2023 11:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="mXzdEBwv" Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsiFI000419; Mon, 18 Dec 2023 11:33:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=a51Llo11f74sG1XbLU6Wq7g1dB96aeHoBOscTkyce+s=; b=mX zdEBwv7zCiWnTucfta3ulBRMfiey4awPgdm58olUTIrCEaCSgCs5JJVx1aUTEbek KpVzrP/1dVk2+KmsxfLBj7S5zWqYV6Eq4heTIiTPRniCapfQcxRy/SCDmhyQ3nVC +lMygGcf8kT7SYC51Tu01d92a96LpFhDrunTNVVFE97jI1Qq/UrlJg9/PsMHbDFX tCTgAe+CJycj3k8a9cHeGfbLpgr+c9l8n27zwuoFt9vhutO/Lkpulvkzv5oiTM64 mR7h3fusT0VrB5C3da8atdsQ5rR9ZGpAoFsrFLDcE7wEaZlC048DdqyCJbJxHlR6 Um6fzo8Zek3YqOo931kQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v14xy4443-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:11 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7FF030042; Mon, 18 Dec 2023 11:33:08 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhp-1; Mon, 18 Dec 2023 11:33:08 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7Xb029998; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6Fk029971; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 32FCE22D6; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 08/34] media: iris: introduce state machine for iris core Date: Mon, 18 Dec 2023 17:02:03 +0530 Message-Id: <1702899149-21321-9-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: -XW7r2OCz6pdpvq95lnIEhvNw-SIYYhw X-Proofpoint-ORIG-GUID: -XW7r2OCz6pdpvq95lnIEhvNw-SIYYhw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Introduces different states for core. State transitions are defined, based on which they would be allowed/rejected/ignored. IRIS_CORE_DEINIT - default state. IRIS_CORE_INIT_WAIT - wait state for video core to complete initialization. IRIS_CORE_INIT - core state with core initialized. FW loaded and HW brought out of reset, shared queues established between host driver and firmware. IRIS_CORE_ERROR - error state. | v ----------- +--->| DEINIT |<---+ | ----------- | | | | | v | | ----------- | | |INIT_WAIT| | | ----------- | | / \ | | / \ | | / \ | | v v v ----------- ----------. | INIT |-->| ERROR | ----------- ----------. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +- .../media/platform/qcom/vcodec/iris/iris_core.h | 4 ++ .../media/platform/qcom/vcodec/iris/iris_state.c | 64 ++++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++++ 4 files changed, 92 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 0748819..12a74de 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -1,4 +1,5 @@ iris-objs += iris_probe.o \ - resources.o + resources.o \ + iris_state.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index c2bc95d..56a5b7d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -9,6 +9,8 @@ #include #include +#include "iris_state.h" + /** * struct iris_core - holds core parameters valid for all instances * @@ -27,6 +29,7 @@ * @clk_count: count of iris clocks * @reset_tbl: table of iris reset clocks * @reset_count: count of iris reset clocks + * @state: current state of core */ struct iris_core { @@ -45,6 +48,7 @@ struct iris_core { u32 clk_count; struct reset_info *reset_tbl; u32 reset_count; + enum iris_core_state state; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c new file mode 100644 index 0000000..22557af --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_state.h" + +#define IRIS_STATE(name)[IRIS_CORE_##name] = "CORE_"#name + +static const char * const core_state_names[] = { + IRIS_STATE(DEINIT), + IRIS_STATE(INIT_WAIT), + IRIS_STATE(INIT), + IRIS_STATE(ERROR), +}; + +#undef IRIS_STATE + +bool core_in_valid_state(struct iris_core *core) +{ + return core->state == IRIS_CORE_INIT || + core->state == IRIS_CORE_INIT_WAIT; +} + +static const char *core_state_name(enum iris_core_state state) +{ + if ((unsigned int)state < ARRAY_SIZE(core_state_names)) + return core_state_names[state]; + + return "UNKNOWN_STATE"; +} + +static bool iris_allow_core_state_change(struct iris_core *core, + enum iris_core_state req_state) +{ + if (core->state == IRIS_CORE_DEINIT) + return req_state == IRIS_CORE_INIT_WAIT || req_state == IRIS_CORE_ERROR; + else if (core->state == IRIS_CORE_INIT_WAIT) + return req_state == IRIS_CORE_INIT || req_state == IRIS_CORE_ERROR; + else if (core->state == IRIS_CORE_INIT) + return req_state == IRIS_CORE_DEINIT || req_state == IRIS_CORE_ERROR; + else if (core->state == IRIS_CORE_ERROR) + return req_state == IRIS_CORE_DEINIT; + + dev_warn(core->dev, "core state change %s -> %s is not allowed\n", + core_state_name(core->state), core_state_name(req_state)); + + return false; +} + +int iris_change_core_state(struct iris_core *core, + enum iris_core_state request_state) +{ + if (core->state == request_state) + return 0; + + if (!iris_allow_core_state_change(core, request_state)) + return -EINVAL; + + core->state = request_state; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h new file mode 100644 index 0000000..ee20842 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_STATE_H_ +#define _IRIS_STATE_H_ + +struct iris_core; + +enum iris_core_state { + IRIS_CORE_DEINIT, + IRIS_CORE_INIT_WAIT, + IRIS_CORE_INIT, + IRIS_CORE_ERROR, +}; + +bool core_in_valid_state(struct iris_core *core); +int iris_change_core_state(struct iris_core *core, + enum iris_core_state request_state); + +#endif From patchwork Mon Dec 18 11:32:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755553 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB17A1B28F; Mon, 18 Dec 2023 11:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="S/8grHhs" Received: from pps.filterd (m0279869.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAs71B019168; Mon, 18 Dec 2023 11:33:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Els7lv4fHIpc+8oVgCKZtE/UP4RNB6Llyf5lFWQyGsk=; b=S/ 8grHhsJ0q+lTgvD7Wiwaqh99x4BJAMI4wtiMZOrC09xkgyILMwoSm9rbUOrusUFC BuvWZBPdT+HssACd3OyLyPFreYY87rQSGgdONvmZ+rTddgBBaJeKFoINgcjw4z7Z NEkA5Phr9/S7HSYWZgfx7dj5hhN6iOFDf+l/bWGqnrlDlzStpZHf4vOV3LkJns51 Yc62cea+pzXOcf39z9j0/0d3S0ZtCy2RyjoV0gIEdsqAo6/nAPyV3AEBGvJuuV4q eCNzr5hW5dBqJLGvtF+cYgTYYBK3awWwtqixKrkR73PmZIR41/xzBh1tN/a9KHBx Bhmb1cKyGwo9hJWiRcxw== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2gw0rkcx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:11 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7qJ030018; Mon, 18 Dec 2023 11:33:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyh4-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX54Y029912; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6sG029966; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3570422D7; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication Date: Mon, 18 Dec 2023 17:02:04 +0530 Message-Id: <1702899149-21321-10-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: NIs9DIMFJPcCORCSVmkJfpjG89p7Rr79 X-Proofpoint-ORIG-GUID: NIs9DIMFJPcCORCSVmkJfpjG89p7Rr79 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 impostorscore=0 mlxscore=0 adultscore=0 malwarescore=0 suspectscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxlogscore=999 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Shared queues are used for communication between driver and firmware. There are 3 types of queues: Command queue - driver to write any command to firmware. Message queue - firmware to send any response to driver. Debug queue - firmware to write debug message. Above queues are initialized and configured to firmware during probe. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 2 ++ .../media/platform/qcom/vcodec/iris/iris_core.h | 11 ++++++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 31 ++++++++++++++++++++++ 3 files changed, 44 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 12a74de..59798e5d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -1,3 +1,5 @@ +iris-objs += ../hfi_queue.o + iris-objs += iris_probe.o \ resources.o \ iris_state.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 56a5b7d..77124f9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -9,6 +9,7 @@ #include #include +#include "../hfi_queue.h" #include "iris_state.h" /** @@ -30,6 +31,11 @@ * @reset_tbl: table of iris reset clocks * @reset_count: count of iris reset clocks * @state: current state of core + * @iface_q_table: Interface queue table memory + * @command_queue: shared interface queue to send commands to firmware + * @message_queue: shared interface queue to receive responses from firmware + * @debug_queue: shared interface queue to receive debug info from firmware + * @sfr: SFR register memory */ struct iris_core { @@ -49,6 +55,11 @@ struct iris_core { struct reset_info *reset_tbl; u32 reset_count; enum iris_core_state state; + struct mem_desc iface_q_table; + struct iface_q_info command_queue; + struct iface_q_info message_queue; + struct iface_q_info debug_queue; + struct mem_desc sfr; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 7bb9c92..fd349a3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -7,6 +7,7 @@ #include #include +#include "../hfi_queue.h" #include "iris_core.h" #include "resources.h" @@ -50,6 +51,10 @@ static void iris_remove(struct platform_device *pdev) if (!core) return; + hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, + &core->command_queue, &core->message_queue, + &core->debug_queue); + video_unregister_device(core->vdev_dec); v4l2_device_unregister(&core->v4l2_dev); @@ -59,6 +64,7 @@ static int iris_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; struct iris_core *core; + u64 dma_mask; int ret; core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL); @@ -91,8 +97,33 @@ static int iris_probe(struct platform_device *pdev) platform_set_drvdata(pdev, core); + /* + * Specify the max value of address space, which can be used + * for buffer transactions. + */ + dma_mask = DMA_BIT_MASK(32); + dma_mask &= ~BIT(29); + + ret = dma_set_mask_and_coherent(dev, dma_mask); + if (ret) + goto err_vdev_unreg; + + dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32)); + dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64)); + + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr, + &core->command_queue, &core->message_queue, + &core->debug_queue, core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: interface queues init failed\n", __func__); + goto err_vdev_unreg; + } + return ret; +err_vdev_unreg: + iris_unregister_video_device(core); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); From patchwork Mon Dec 18 11:32:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757250 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 267FE1A27E; Mon, 18 Dec 2023 11:33:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="NjoabHjr" Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBScQb021562; Mon, 18 Dec 2023 11:33:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=HWrsGR9dQBQYAECyvELmVJvsiibu4cOF7NAEek6IDMw=; b=Nj oabHjr+gMf44d+g9AkB7ma5F/Uo/Lbd5BOYAxAaPUBtkSV0JtCzDADxoR5adag40 xFeV5lhcuXy3n8XYa25wA/C63kkzlA0CXrSzXMN6zrNn1zUaMP05tm+cIkGfzNdo 2SCTIwRxP0h0SHQFzdjtoJEATOStcKj8poNU/Iyw9z+0BaA/QlX1MVMTNm0CYV2m AUNbgXQs9Hw2nFNv7FAqW72Rk1JuWedHM5A0JcdCO9W7oieCxfDGVRaEmqaGDGca rPGxGxmxIgKkQ5PXR6uO7tYMqz0+TBppZFtAbyHsjHwdNFwthYTZfZ+OUljKHmaZ cURi2vWBYEecfk3Fb5tA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mjfr3a5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:10 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7t0030046; Mon, 18 Dec 2023 11:33:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhe-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmqr028240; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6ki029964; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3802122D8; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 10/34] media: iris: add PIL functionality for video firmware Date: Mon, 18 Dec 2023 17:02:05 +0530 Message-Id: <1702899149-21321-11-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: rlMvPhtwa_nFYWHeQKKLBMx4fwLr5wWF X-Proofpoint-ORIG-GUID: rlMvPhtwa_nFYWHeQKKLBMx4fwLr5wWF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 clxscore=1015 malwarescore=0 mlxscore=0 mlxlogscore=999 lowpriorityscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Load/unload firmware in memory via mdt loader. Firmware loading is part of core initialization and unloading is part of core de-initialization. This also changes the core states accordingly. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 10 ++- .../media/platform/qcom/vcodec/iris/iris_core.c | 70 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_core.h | 8 +++ .../media/platform/qcom/vcodec/iris/iris_helpers.c | 15 +++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 13 ++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 72 ++++++++++++++++++++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 14 +++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 19 +++++- .../media/platform/qcom/vcodec/iris/iris_state.c | 9 ++- 9 files changed, 225 insertions(+), 5 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 59798e5d..74bd344 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -1,7 +1,11 @@ -iris-objs += ../hfi_queue.o +iris-objs += ../hfi_queue.o ../firmware.o iris-objs += iris_probe.o \ - resources.o \ - iris_state.o + iris_state.o \ + iris_core.o \ + iris_state.o \ + iris_helpers.o \ + iris_hfi.o \ + resources.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c new file mode 100644 index 0000000..ba8960d --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_state.h" + +static int iris_core_deinit_locked(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (core->state == IRIS_CORE_DEINIT) + return 0; + + iris_hfi_core_deinit(core); + + iris_change_core_state(core, IRIS_CORE_DEINIT); + + return ret; +} + +int iris_core_deinit(struct iris_core *core) +{ + int ret; + + mutex_lock(&core->lock); + ret = iris_core_deinit_locked(core); + mutex_unlock(&core->lock); + + return ret; +} + +int iris_core_init(struct iris_core *core) +{ + int ret = 0; + + mutex_lock(&core->lock); + if (core_in_valid_state(core)) { + goto unlock; + } else if (core->state == IRIS_CORE_ERROR) { + ret = -EINVAL; + goto unlock; + } + + if (iris_change_core_state(core, IRIS_CORE_INIT_WAIT)) { + iris_change_core_state(core, IRIS_CORE_ERROR); + ret = -EINVAL; + goto unlock; + } + + ret = iris_hfi_core_init(core); + if (ret) { + iris_change_core_state(core, IRIS_CORE_ERROR); + dev_err(core->dev, "%s: core init failed\n", __func__); + iris_core_deinit_locked(core); + goto unlock; + } + +unlock: + mutex_unlock(&core->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 77124f9..2740ff1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -11,6 +11,7 @@ #include "../hfi_queue.h" #include "iris_state.h" +#include "resources.h" /** * struct iris_core - holds core parameters valid for all instances @@ -36,6 +37,8 @@ * @message_queue: shared interface queue to receive responses from firmware * @debug_queue: shared interface queue to receive debug info from firmware * @sfr: SFR register memory + * @lock: a lock for this strucure + * @use_tz: a flag that suggests presence of trustzone */ struct iris_core { @@ -60,6 +63,11 @@ struct iris_core { struct iface_q_info message_queue; struct iface_q_info debug_queue; struct mem_desc sfr; + struct mutex lock; /* lock for core structure */ + unsigned int use_tz; }; +int iris_core_init(struct iris_core *core); +int iris_core_deinit(struct iris_core *core); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c new file mode 100644 index 0000000..22c706a --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_helpers.h" + +int check_core_lock(struct iris_core *core) +{ + bool fatal = !mutex_is_locked(&core->lock); + + WARN_ON(fatal); + + return fatal ? -EINVAL : 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h new file mode 100644 index 0000000..314a8d75 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HELPERS_H_ +#define _IRIS_HELPERS_H_ + +#include "iris_core.h" + +int check_core_lock(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c new file mode 100644 index 0000000..4f51a8c --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "../firmware.h" +#include "iris_helpers.h" +#include "iris_hfi.h" + +#define CP_START 0 +#define CP_SIZE 0x25800000 +#define CP_NONPIXEL_START 0x01000000 +#define CP_NONPIXEL_SIZE 0x24800000 + +#define FW_NAME "vpu30_4v.mbn" +#define IRIS_PAS_ID 9 + +int iris_hfi_core_init(struct iris_core *core) +{ + phys_addr_t mem_phys = 0; + size_t mem_size = 0; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr, + &core->command_queue, &core->message_queue, + &core->debug_queue, core); + if (ret) + goto error; + + core->use_tz = use_tz(core->dev); + if (!core->use_tz) + goto error; + + ret = load_fw(core->dev, FW_NAME, &mem_phys, &mem_size, + IRIS_PAS_ID, core->use_tz); + if (ret) + goto error; + + ret = auth_reset_fw(IRIS_PAS_ID); + if (ret) + goto error; + + ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START, + CP_NONPIXEL_SIZE, IRIS_PAS_ID); + + return ret; + +error: + dev_err(core->dev, "%s(): failed with ret %d\n", __func__, ret); + + return ret; +} + +int iris_hfi_core_deinit(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (core->state == IRIS_CORE_DEINIT) + return 0; + + unload_fw(IRIS_PAS_ID); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h new file mode 100644 index 0000000..fcf9f28 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_H_ +#define _IRIS_HFI_H_ + +#include "iris_core.h" + +int iris_hfi_core_init(struct iris_core *core); +int iris_hfi_core_deinit(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index fd349a3..f39b4aa 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -51,6 +51,7 @@ static void iris_remove(struct platform_device *pdev) if (!core) return; + iris_core_deinit(core); hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, &core->debug_queue); @@ -58,6 +59,9 @@ static void iris_remove(struct platform_device *pdev) video_unregister_device(core->vdev_dec); v4l2_device_unregister(&core->v4l2_dev); + + mutex_destroy(&core->lock); + core->state = IRIS_CORE_DEINIT; } static int iris_probe(struct platform_device *pdev) @@ -72,6 +76,9 @@ static int iris_probe(struct platform_device *pdev) return -ENOMEM; core->dev = dev; + core->state = IRIS_CORE_DEINIT; + mutex_init(&core->lock); + core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) return PTR_ERR(core->reg_base); @@ -120,10 +127,20 @@ static int iris_probe(struct platform_device *pdev) goto err_vdev_unreg; } + ret = iris_core_init(core); + if (ret) { + dev_err_probe(core->dev, ret, "%s: core init failed\n", __func__); + goto err_queue_deinit; + } + return ret; +err_queue_deinit: + hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, + &core->command_queue, &core->message_queue, + &core->debug_queue); err_vdev_unreg: - iris_unregister_video_device(core); + video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 22557af..83bbc6b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -4,6 +4,7 @@ */ #include "iris_core.h" +#include "iris_helpers.h" #include "iris_state.h" #define IRIS_STATE(name)[IRIS_CORE_##name] = "CORE_"#name @@ -52,6 +53,12 @@ static bool iris_allow_core_state_change(struct iris_core *core, int iris_change_core_state(struct iris_core *core, enum iris_core_state request_state) { + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + if (core->state == request_state) return 0; @@ -60,5 +67,5 @@ int iris_change_core_state(struct iris_core *core, core->state = request_state; - return 0; + return ret; } From patchwork Mon Dec 18 11:32:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757246 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E44371B28C; Mon, 18 Dec 2023 11:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="TpAnRsLP" Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAs0cd008210; Mon, 18 Dec 2023 11:33:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=yjMHqh/n3Ch2ZrS6Ma1pzlwfHJQlBFURK7vxHKSwTCE=; b=Tp AnRsLPYnGNzagew7+f4F+ikwkLSvJbgC6YGH0LBwzl5IdRK3RgYy5c4GiPbuz+3B F0Won4jJaL0E5+QYnnYHrbG0x5nnHU3HAsswbHxtl87vhdM/cc0z+TuKv9B7hvrz iMrDOQVFVwYHKuknV2IiN9E+oI1X9fRz3cpYoUi9GJ7nCnuU0sKKHa8RTDivyub1 DbJ5iK3K5Vh4UjSNHPleDw7zmaNq2SC6YDdihgiD9aZC27WMIEKsItmvntxkcMZQ SdM7lfB6cFYZyjMlxcjlCQCPWfJK4vanJoTgUXKJZuyJHUunhOsTondB24OKh9Rt /jb/0KN+FCboEAxMedtA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2m49g5h6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:12 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7u7030031; Mon, 18 Dec 2023 11:33:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyh6-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7dN029984; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6a6029970; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3A92F22D9; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets Date: Mon, 18 Dec 2023 17:02:06 +0530 Message-Id: <1702899149-21321-12-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 4WoFG18BTgNUMyPf2ImxNsnxuVzUyCBA X-Proofpoint-GUID: 4WoFG18BTgNUMyPf2ImxNsnxuVzUyCBA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=999 impostorscore=0 malwarescore=0 clxscore=1015 mlxscore=0 spamscore=0 suspectscore=0 bulkscore=0 priorityscore=1501 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Host firmware interface (HFI) is well defined set of interfaces for communication between host driver and firmware. The command and responses are exchanged in form of packets. One or multiple packets are grouped under packet header. Each packet has packet type which describes the specific HFI and payload which holds the corresponding value for that HFI. Sys_init is the first packets sent to firmware, which initializes the firmware. Sys_image_version packet is to get the firmware version string. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 1 + .../media/platform/qcom/vcodec/iris/hfi_defines.h | 15 +++ .../media/platform/qcom/vcodec/iris/iris_core.h | 10 ++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 69 +++++++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 129 +++++++++++++++++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 71 ++++++++++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 5 + 7 files changed, 300 insertions(+) create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 74bd344..a2d5d74 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -6,6 +6,7 @@ iris-objs += iris_probe.o \ iris_state.o \ iris_helpers.o \ iris_hfi.o \ + iris_hfi_packet.o \ resources.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h new file mode 100644 index 0000000..4c48c90 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _HFI_DEFINES_H_ +#define _HFI_DEFINES_H_ + +#define HFI_VIDEO_ARCH_LX 0x1 + +#define HFI_CMD_INIT 0x01000001 + +#define HFI_PROP_IMAGE_VERSION 0x03000001 + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 2740ff1..de0cfef 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -39,6 +39,11 @@ * @sfr: SFR register memory * @lock: a lock for this strucure * @use_tz: a flag that suggests presence of trustzone + * @packet: pointer to packet from driver to fw + * @packet_size: size of packet + * @sys_init_id: id of sys init packet + * @header_id: id of packet header + * @packet_id: id of packet */ struct iris_core { @@ -65,6 +70,11 @@ struct iris_core { struct mem_desc sfr; struct mutex lock; /* lock for core structure */ unsigned int use_tz; + u8 *packet; + u32 packet_size; + u32 sys_init_id; + u32 header_id; + u32 packet_id; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 4f51a8c..fe16448 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -4,8 +4,67 @@ */ #include "../firmware.h" +#include "../hfi_queue.h" #include "iris_helpers.h" #include "iris_hfi.h" +#include "iris_hfi_packet.h" + +static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) +{ + struct iface_q_info *q_info; + struct hfi_header *header; + u32 packet_size, rx_req; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core_in_valid_state(core)) + return -EINVAL; + + q_info = &core->command_queue; + if (!q_info || !q_info->q_array.kernel_vaddr || !pkt) { + dev_err(core->dev, "cannot write to shared CMD Q's\n"); + return -ENODATA; + } + + header = pkt; + packet_size = header->size; + + if (!write_queue(q_info, pkt, packet_size, &rx_req)) { + dev_err(core->dev, "queue full\n"); + return -ENODATA; + } + + return ret; +} + +static int sys_init(struct iris_core *core) +{ + int ret; + + ret = hfi_packet_sys_init(core, core->packet, core->packet_size); + if (ret) + return ret; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + + return ret; +} + +static int sys_image_version(struct iris_core *core) +{ + int ret; + + ret = hfi_packet_image_version(core, core->packet, core->packet_size); + if (ret) + return ret; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + + return ret; +} #define CP_START 0 #define CP_SIZE 0x25800000 @@ -46,6 +105,16 @@ int iris_hfi_core_init(struct iris_core *core) ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START, CP_NONPIXEL_SIZE, IRIS_PAS_ID); + if (ret) + goto error; + + ret = sys_init(core); + if (ret) + goto error; + + ret = sys_image_version(core); + if (ret) + goto error; return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c new file mode 100644 index 0000000..73bba07 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_hfi_packet.h" +#include "hfi_defines.h" + +static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, + u32 header_id) +{ + struct hfi_header *hdr = (struct hfi_header *)packet; + + if (!packet || packet_size < sizeof(*hdr)) + return -EINVAL; + + memset(hdr, 0, sizeof(*hdr)); + + hdr->size = sizeof(*hdr); + hdr->session_id = session_id; + hdr->header_id = header_id; + hdr->num_packets = 0; + + return 0; +} + +static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, + u32 pkt_flags, u32 payload_type, u32 port, + u32 packet_id, void *payload, u32 payload_size) +{ + struct hfi_header *hdr; + struct hfi_packet *pkt; + u32 pkt_size; + + if (!packet) + return -EINVAL; + + hdr = (struct hfi_header *)packet; + if (hdr->size < sizeof(*hdr)) + return -EINVAL; + + pkt = (struct hfi_packet *)(packet + hdr->size); + pkt_size = sizeof(*pkt) + payload_size; + if (packet_size < hdr->size + pkt_size) + return -EINVAL; + + memset(pkt, 0, pkt_size); + pkt->size = pkt_size; + pkt->type = pkt_type; + pkt->flags = pkt_flags; + pkt->payload_info = payload_type; + pkt->port = port; + pkt->packet_id = packet_id; + if (payload_size) + memcpy((u8 *)pkt + sizeof(*pkt), + payload, payload_size); + + hdr->num_packets++; + hdr->size += pkt->size; + + return 0; +} + +int hfi_packet_sys_init(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + u32 payload = 0; + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0, + core->header_id++); + if (ret) + goto error; + + payload = HFI_VIDEO_ARCH_LX; + core->sys_init_id = core->packet_id++; + ret = hfi_create_packet(pkt, pkt_size, + HFI_CMD_INIT, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->sys_init_id, + &payload, + sizeof(u32)); + if (ret) + goto error; + + return ret; + +error: + dev_err(core->dev, "%s: create sys init packet failed\n", __func__); + + return ret; +} + +int hfi_packet_image_version(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0, + core->header_id++); + if (ret) + goto error; + + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_IMAGE_VERSION, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_GET_PROPERTY), + HFI_PAYLOAD_NONE, + HFI_PORT_NONE, + core->packet_id++, + NULL, 0); + if (ret) + goto error; + + return ret; + +error: + dev_err(core->dev, "%s: create image version packet failed\n", __func__); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h new file mode 100644 index 0000000..e36612c --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_PACKET_H_ +#define _IRIS_HFI_PACKET_H_ + +struct hfi_header { + u32 size; + u32 session_id; + u32 header_id; + u32 reserved[4]; + u32 num_packets; +}; + +struct hfi_packet { + u32 size; + u32 type; + u32 flags; + u32 payload_info; + u32 port; + u32 packet_id; + u32 reserved[2]; +}; + +enum hfi_packet_host_flags { + HFI_HOST_FLAGS_NONE = 0x00000000, + HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001, + HFI_HOST_FLAGS_RESPONSE_REQUIRED = 0x00000002, + HFI_HOST_FLAGS_NON_DISCARDABLE = 0x00000004, + HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008, +}; + +enum hfi_packet_firmware_flags { + HFI_FW_FLAGS_NONE = 0x00000000, + HFI_FW_FLAGS_SUCCESS = 0x00000001, + HFI_FW_FLAGS_INFORMATION = 0x00000002, + HFI_FW_FLAGS_SESSION_ERROR = 0x00000004, + HFI_FW_FLAGS_SYSTEM_ERROR = 0x00000008, +}; + +enum hfi_packet_payload_info { + HFI_PAYLOAD_NONE = 0x00000000, + HFI_PAYLOAD_U32 = 0x00000001, + HFI_PAYLOAD_S32 = 0x00000002, + HFI_PAYLOAD_U64 = 0x00000003, + HFI_PAYLOAD_S64 = 0x00000004, + HFI_PAYLOAD_STRUCTURE = 0x00000005, + HFI_PAYLOAD_BLOB = 0x00000006, + HFI_PAYLOAD_STRING = 0x00000007, + HFI_PAYLOAD_Q16 = 0x00000008, + HFI_PAYLOAD_U32_ENUM = 0x00000009, + HFI_PAYLOAD_32_PACKED = 0x0000000a, + HFI_PAYLOAD_U32_ARRAY = 0x0000000b, + HFI_PAYLOAD_S32_ARRAY = 0x0000000c, + HFI_PAYLOAD_64_PACKED = 0x0000000d, +}; + +enum hfi_packet_port_type { + HFI_PORT_NONE = 0x00000000, + HFI_PORT_BITSTREAM = 0x00000001, + HFI_PORT_RAW = 0x00000002, +}; + +int hfi_packet_sys_init(struct iris_core *core, + u8 *pkt, u32 pkt_size); +int hfi_packet_image_version(struct iris_core *core, + u8 *pkt, u32 pkt_size); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index f39b4aa..570c64e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -79,6 +79,11 @@ static int iris_probe(struct platform_device *pdev) core->state = IRIS_CORE_DEINIT; mutex_init(&core->lock); + core->packet_size = IFACEQ_CORE_PKT_SIZE; + core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL); + if (!core->packet) + return -ENOMEM; + core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) return PTR_ERR(core->reg_base); From patchwork Mon Dec 18 11:32:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757248 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B398E1B286; Mon, 18 Dec 2023 11:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="LQspHlPT" Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArsDF026901; Mon, 18 Dec 2023 11:33:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=4sLJrCCYfSes228+tKDxIKdmEKAMGeK6QCzBbNNSTD4=; b=LQ spHlPT700/t/JailUFCIpFAuZ27YLa0mxbQkoPX1K43MbR0CO2wNOCp7wGk5zEUS xj7NVjRh0Fg4vsK0lHtNU1fONVAiCokTCuAZ0cSKfyq+g7B9wX7XZDKW78hzl2UQ a+11E3U5ZpKYswLbYh1B3QwoPfgXf4CaJJpxd8xwIIRRM0C/s+fO446XBvBAQRB4 Z87F/R0b1NDH+28ZPL+MM1SdYekXawK8K10nit+KVyDvgIoBUm5YCC302357tkXE a4wh+Lj4zsxmFEJLg20jVR7Zr36QdPk7DSpqh3xiElb0EYrI0jQtn1UO79tSm2EC OOyM/9Zzy1EkS2pwM2bA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mjfr3a7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:12 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5MD029943; Mon, 18 Dec 2023 11:33:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhn-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX72p029995; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6B1029969; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3CF6D22F5; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling Date: Mon, 18 Dec 2023 17:02:07 +0530 Message-Id: <1702899149-21321-13-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: rD8SNUoVX7ts_0OH9eqoGmsZ6moBsFK6 X-Proofpoint-ORIG-GUID: rD8SNUoVX7ts_0OH9eqoGmsZ6moBsFK6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 clxscore=1015 malwarescore=0 mlxscore=0 mlxlogscore=999 lowpriorityscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Registers are defined differently for different VPUs. Define ops for VPU specific handling to accommodate different VPUs. Implement boot sequence of firmware and interrupt programming. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +- .../media/platform/qcom/vcodec/iris/iris_core.h | 3 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 7 + .../media/platform/qcom/vcodec/iris/iris_probe.c | 7 + .../media/platform/qcom/vcodec/iris/vpu_common.c | 71 +++++++++ .../media/platform/qcom/vcodec/iris/vpu_common.h | 32 ++++ .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 166 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 ++ 8 files changed, 302 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index a2d5d74..90241b5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -7,6 +7,8 @@ iris-objs += iris_probe.o \ iris_helpers.o \ iris_hfi.o \ iris_hfi_packet.o \ - resources.o + resources.o \ + vpu_common.o \ + vpu_iris3.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index de0cfef..64678fd 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -12,6 +12,7 @@ #include "../hfi_queue.h" #include "iris_state.h" #include "resources.h" +#include "vpu_common.h" /** * struct iris_core - holds core parameters valid for all instances @@ -44,6 +45,7 @@ * @sys_init_id: id of sys init packet * @header_id: id of packet header * @packet_id: id of packet + * @vpu_ops: a pointer to vpu ops */ struct iris_core { @@ -75,6 +77,7 @@ struct iris_core { u32 sys_init_id; u32 header_id; u32 packet_id; + const struct vpu_ops *vpu_ops; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index fe16448..7b3cbbc 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -8,6 +8,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" +#include "vpu_common.h" static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) { @@ -33,6 +34,8 @@ static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) packet_size = header->size; if (!write_queue(q_info, pkt, packet_size, &rx_req)) { + call_vpu_op(core, raise_interrupt, core); + } else { dev_err(core->dev, "queue full\n"); return -ENODATA; } @@ -108,6 +111,10 @@ int iris_hfi_core_init(struct iris_core *core) if (ret) goto error; + ret = call_vpu_op(core, boot_firmware, core); + if (ret) + goto error; + ret = sys_init(core); if (ret) goto error; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 570c64e..773481f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -92,6 +92,13 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + ret = init_vpu(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init vpu failed with %d\n", __func__, ret); + return ret; + } + ret = init_resources(core); if (ret) { dev_err_probe(core->dev, ret, diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c new file mode 100644 index 0000000..3282510 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "vpu_iris3.h" +#include "iris_core.h" +#include "iris_helpers.h" +#include "vpu_common.h" + +int write_register(struct iris_core *core, u32 reg, u32 value) +{ + void __iomem *base_addr; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + base_addr = core->reg_base; + base_addr += reg; + writel_relaxed(value, base_addr); + + /* Make sure value is written into the register */ + wmb(); + + return ret; +} + +int read_register(struct iris_core *core, u32 reg, u32 *value) +{ + void __iomem *base_addr; + + base_addr = core->reg_base; + + *value = readl_relaxed(base_addr + reg); + + /* Make sure value is read correctly from the register */ + rmb(); + + return 0; +} + +static const struct compat_handle compat_handle[] = { + { + .compat = "qcom,sm8550-iris", + .init = init_iris3, + }, +}; + +int init_vpu(struct iris_core *core) +{ + struct device *dev = NULL; + int i, ret = 0; + + dev = core->dev; + + for (i = 0; i < ARRAY_SIZE(compat_handle); i++) { + if (of_device_is_compatible(dev->of_node, compat_handle[i].compat)) { + ret = compat_handle[i].init(core); + if (ret) + return ret; + break; + } + } + + if (i == ARRAY_SIZE(compat_handle)) + return -EINVAL; + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h new file mode 100644 index 0000000..790496a --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _VPU_COMMON_H_ +#define _VPU_COMMON_H_ + +#include + +struct iris_core; + +#define call_vpu_op(d, op, ...) \ + (((d) && (d)->vpu_ops && (d)->vpu_ops->op) ? \ + ((d)->vpu_ops->op(__VA_ARGS__)) : 0) + +struct compat_handle { + const char *compat; + int (*init)(struct iris_core *core); +}; + +struct vpu_ops { + int (*boot_firmware)(struct iris_core *core); + int (*raise_interrupt)(struct iris_core *core); +}; + +int init_vpu(struct iris_core *core); + +int write_register(struct iris_core *core, u32 reg, u32 value); +int read_register(struct iris_core *core, u32 reg, u32 *value); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c new file mode 100644 index 0000000..95bf223 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include "vpu_iris3.h" + +#define VIDEO_ARCH_LX 1 + +#define CPU_BASE_OFFS_IRIS3 0x000A0000 + +#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) +#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) + +#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24) +#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28) + +/* HFI_CTRL_INIT */ +#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48) + +/* HFI_CTRL_STATUS */ +#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C) +#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000 + +#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148) + +#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168) + +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64) + +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68) + +/* HFI_QTBL_INFO */ +#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50) + +/* HFI_QTBL_ADDR */ +#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54) + +/* SFR_ADDR */ +#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C) + +#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3 +#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3 + +#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3 +#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3 + +#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3 + +#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3 + +#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3 +#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe +#define CTRL_ERROR_STATUS__M_IRIS3 \ + CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 + +#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150) +#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0 + +static int setup_ucregion_memory_map_iris3(struct iris_core *core) +{ + int ret; + u32 value; + + value = (u32)core->iface_q_table.device_addr; + ret = write_register(core, UC_REGION_ADDR_IRIS3, value); + if (ret) + return ret; + + value = SHARED_QSIZE; + ret = write_register(core, UC_REGION_SIZE_IRIS3, value); + if (ret) + return ret; + + value = (u32)core->iface_q_table.device_addr; + ret = write_register(core, QTBL_ADDR_IRIS3, value); + if (ret) + return ret; + + ret = write_register(core, QTBL_INFO_IRIS3, 0x01); + if (ret) + return ret; + + value = (u32)((u64)core->iface_q_table.kernel_vaddr); + ret = write_register(core, CPU_CS_VCICMDARG0_IRIS3, value); + if (ret) + return ret; + + value = (u32)((u64)core->iface_q_table.kernel_vaddr >> 32); + ret = write_register(core, CPU_CS_VCICMDARG1_IRIS3, value); + if (ret) + return ret; + + if (core->sfr.device_addr) { + value = (u32)core->sfr.device_addr + VIDEO_ARCH_LX; + ret = write_register(core, SFR_ADDR_IRIS3, value); + if (ret) + return ret; + } + + return ret; +} + +static int boot_firmware_iris3(struct iris_core *core) +{ + u32 ctrl_init = 0, ctrl_status = 0, count = 0, max_tries = 1000; + int ret; + + ret = setup_ucregion_memory_map_iris3(core); + if (ret) + return ret; + + ctrl_init = BIT(0); + + ret = write_register(core, CTRL_INIT_IRIS3, ctrl_init); + if (ret) + return ret; + + while (!ctrl_status && count < max_tries) { + ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status); + if (ret) + return ret; + + if ((ctrl_status & CTRL_ERROR_STATUS__M_IRIS3) == 0x4) { + dev_err(core->dev, "invalid setting for UC_REGION\n"); + break; + } + + usleep_range(50, 100); + count++; + } + + if (count >= max_tries) { + dev_err(core->dev, "Error booting up vidc firmware\n"); + return -ETIME; + } + + ret = write_register(core, CPU_CS_H2XSOFTINTEN_IRIS3, 0x1); + if (ret) + return ret; + + ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x0); + + return ret; +} + +static int raise_interrupt_iris3(struct iris_core *core) +{ + return write_register(core, CPU_IC_SOFTINT_IRIS3, 1 << CPU_IC_SOFTINT_H2A_SHFT_IRIS3); +} + +static const struct vpu_ops iris3_ops = { + .boot_firmware = boot_firmware_iris3, + .raise_interrupt = raise_interrupt_iris3, +}; + +int init_iris3(struct iris_core *core) +{ + core->vpu_ops = &iris3_ops; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h new file mode 100644 index 0000000..1424a5f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _VPU_IRIS3_H_ +#define _VPU_IRIS3_H_ + +#include "iris_core.h" + +int init_iris3(struct iris_core *core); + +#endif From patchwork Mon Dec 18 11:32:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755552 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 937971B29D; Mon, 18 Dec 2023 11:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="niS5GENc" Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAs3ZU027066; Mon, 18 Dec 2023 11:33:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=UBiV64Si8CIj4lSYpZDjtUg79mUR31H12KPH3CgY11w=; b=ni S5GENcpsmHocQwELi0F+Sn/c7Eo+LGqL/neAoBMknJ8xkMrJbViW5/asF7xn8lHR 91F1HIhjKkkBR842ILEBvkYViaXC+kff6Om+x//YEg4OO7qvMxiBOsprM+Ctgkal oCHT1lzSgcdM9tmKSCQ9Wyu+54EpryjaRlGcZ6kcedy368SMt+sJXu2lyi9ByktL uCowc+EBZVcm1ZYX9uDHfCw+TVLjN4viyO1vha6cW7X3dmh1TYSQ6qyo+927dSfM 5ziCc7wX07hcXd6pHQ7ZPMvOmZ8UZhaGwqZ+RKTp4Z7xuEBcXdv9N65ocHia7ILP WbCFhQxq0Tm16UCkMURQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mjfr3a6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:10 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7FE030042; Mon, 18 Dec 2023 11:33:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyha-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX72n029995; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6IG029972; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3F9562304; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance Date: Mon, 18 Dec 2023 17:02:08 +0530 Message-Id: <1702899149-21321-14-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: OMgNGxZktJeK8alVapkhuj-SvUeSu5mx X-Proofpoint-ORIG-GUID: OMgNGxZktJeK8alVapkhuj-SvUeSu5mx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 clxscore=1015 malwarescore=0 mlxscore=0 mlxlogscore=999 lowpriorityscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Capabilities are set of video specifications and features supported by a specific platform(SOC). Each capability is defined with min, max, range, default value and corresponding HFI. Also, platform data defines different resource details for a specific platform(SOC). This change defines resource tables for sm8550 platform data and use for initializing these resources. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 42 ++- .../media/platform/qcom/vcodec/iris/iris_core.h | 5 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 19 ++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 1 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 20 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 84 ++++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 27 +- .../platform/qcom/vcodec/iris/platform_common.c | 29 ++ .../platform/qcom/vcodec/iris/platform_common.h | 194 +++++++++++++ .../platform/qcom/vcodec/iris/platform_sm8550.c | 316 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/resources.c | 74 +++-- 12 files changed, 759 insertions(+), 56 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 90241b5..c50e3241 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -9,6 +9,8 @@ iris-objs += iris_probe.o \ iris_hfi_packet.o \ resources.o \ vpu_common.o \ - vpu_iris3.o + vpu_iris3.o \ + platform_common.o \ + platform_sm8550.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index 4c48c90..fb383b2 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -6,10 +6,46 @@ #ifndef _HFI_DEFINES_H_ #define _HFI_DEFINES_H_ -#define HFI_VIDEO_ARCH_LX 0x1 +#define HFI_VIDEO_ARCH_LX 0x1 -#define HFI_CMD_INIT 0x01000001 +#define HFI_CMD_INIT 0x01000001 -#define HFI_PROP_IMAGE_VERSION 0x03000001 +#define HFI_PROP_IMAGE_VERSION 0x03000001 + +#define HFI_PROP_UBWC_MAX_CHANNELS 0x03000003 +#define HFI_PROP_UBWC_MAL_LENGTH 0x03000004 +#define HFI_PROP_UBWC_HBB 0x03000005 +#define HFI_PROP_UBWC_BANK_SWZL_LEVEL1 0x03000006 +#define HFI_PROP_UBWC_BANK_SWZL_LEVEL2 0x03000007 +#define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008 +#define HFI_PROP_UBWC_BANK_SPREADING 0x03000009 + +#define HFI_PROP_PROFILE 0x03000107 + +#define HFI_PROP_LEVEL 0x03000108 + +#define HFI_PROP_TIER 0x03000109 + +#define HFI_PROP_STAGE 0x0300010a + +#define HFI_PROP_PIPE 0x0300010b + +#define HFI_PROP_FRAME_RATE 0x0300010c + +#define HFI_PROP_LUMA_CHROMA_BIT_DEPTH 0x0300010f + +#define HFI_PROP_CODED_FRAMES 0x03000120 + +#define HFI_PROP_CABAC_SESSION 0x03000121 + +#define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123 + +#define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128 + +#define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b + +#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 + +#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 64678fd..3c8497a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -11,6 +11,7 @@ #include "../hfi_queue.h" #include "iris_state.h" +#include "platform_common.h" #include "resources.h" #include "vpu_common.h" @@ -46,6 +47,8 @@ * @header_id: id of packet header * @packet_id: id of packet * @vpu_ops: a pointer to vpu ops + * @platform_data: a structure for platform data + * @cap: an array for supported core capabilities */ struct iris_core { @@ -78,6 +81,8 @@ struct iris_core { u32 header_id; u32 packet_id; const struct vpu_ops *vpu_ops; + struct platform_data *platform_data; + struct plat_core_cap cap[CORE_CAP_MAX + 1]; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 22c706a..c31dfd5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -13,3 +13,22 @@ int check_core_lock(struct iris_core *core) return fatal ? -EINVAL : 0; } + +int iris_init_core_caps(struct iris_core *core) +{ + struct plat_core_cap *core_platform_data; + int i, num_core_caps; + + core_platform_data = core->platform_data->core_data; + if (!core_platform_data) + return -EINVAL; + + num_core_caps = core->platform_data->core_data_size; + + for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) { + core->cap[core_platform_data[i].type].type = core_platform_data[i].type; + core->cap[core_platform_data[i].type].value = core_platform_data[i].value; + } + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 314a8d75..4c7ddbf 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -9,5 +9,6 @@ #include "iris_core.h" int check_core_lock(struct iris_core *core); +int iris_init_core_caps(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 7b3cbbc..b0390b542 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -79,8 +79,12 @@ static int sys_image_version(struct iris_core *core) int iris_hfi_core_init(struct iris_core *core) { + u32 cp_nonpixel_start, cp_nonpixel_size; phys_addr_t mem_phys = 0; + u32 cp_start, cp_size; + const char *fw_name; size_t mem_size = 0; + u32 pas_id; int ret; ret = check_core_lock(core); @@ -97,17 +101,23 @@ int iris_hfi_core_init(struct iris_core *core) if (!core->use_tz) goto error; - ret = load_fw(core->dev, FW_NAME, &mem_phys, &mem_size, - IRIS_PAS_ID, core->use_tz); + pas_id = core->platform_data->pas_id; + fw_name = core->platform_data->fwname; + ret = load_fw(core->dev, fw_name, &mem_phys, &mem_size, + pas_id, core->use_tz); if (ret) goto error; - ret = auth_reset_fw(IRIS_PAS_ID); + ret = auth_reset_fw(pas_id); if (ret) goto error; - ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START, - CP_NONPIXEL_SIZE, IRIS_PAS_ID); + cp_start = core->cap[CP_START].value; + cp_size = core->cap[CP_SIZE].value; + cp_nonpixel_start = core->cap[CP_NONPIXEL_START].value; + cp_nonpixel_size = core->cap[CP_NONPIXEL_SIZE].value; + ret = protect_secure_region(cp_start, cp_size, cp_nonpixel_start, + cp_nonpixel_size, pas_id); if (ret) goto error; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index 73bba07..f31dd84 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -89,6 +89,90 @@ int hfi_packet_sys_init(struct iris_core *core, if (ret) goto error; + payload = core->platform_data->ubwc_config->max_channels; + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_UBWC_MAX_CHANNELS, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + if (ret) + goto error; + + payload = core->platform_data->ubwc_config->mal_length; + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_UBWC_MAL_LENGTH, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + if (ret) + goto error; + + payload = core->platform_data->ubwc_config->highest_bank_bit; + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_UBWC_HBB, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + if (ret) + goto error; + + payload = core->platform_data->ubwc_config->bank_swzl_level; + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_UBWC_BANK_SWZL_LEVEL1, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + if (ret) + goto error; + + payload = core->platform_data->ubwc_config->bank_swz2_level; + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_UBWC_BANK_SWZL_LEVEL2, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + if (ret) + goto error; + + payload = core->platform_data->ubwc_config->bank_swz3_level; + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_UBWC_BANK_SWZL_LEVEL3, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + if (ret) + goto error; + + payload = core->platform_data->ubwc_config->bank_spreading; + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_UBWC_BANK_SPREADING, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + if (ret) + goto error; + return ret; error: diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 773481f..9d6a6c5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -9,6 +9,7 @@ #include "../hfi_queue.h" #include "iris_core.h" +#include "iris_helpers.h" #include "resources.h" static int iris_register_video_device(struct iris_core *core) @@ -92,6 +93,13 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + ret = init_platform(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init platform failed with %d\n", __func__, ret); + return ret; + } + ret = init_vpu(core); if (ret) { dev_err_probe(core->dev, ret, @@ -106,6 +114,13 @@ static int iris_probe(struct platform_device *pdev) return ret; } + ret = iris_init_core_caps(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init core caps failed with %d\n", __func__, ret); + return ret; + } + ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) return ret; @@ -116,12 +131,7 @@ static int iris_probe(struct platform_device *pdev) platform_set_drvdata(pdev, core); - /* - * Specify the max value of address space, which can be used - * for buffer transactions. - */ - dma_mask = DMA_BIT_MASK(32); - dma_mask &= ~BIT(29); + dma_mask = core->cap[DMA_MASK].value; ret = dma_set_mask_and_coherent(dev, dma_mask); if (ret) @@ -160,7 +170,10 @@ static int iris_probe(struct platform_device *pdev) } static const struct of_device_id iris_dt_match[] = { - { .compatible = "qcom,sm8550-iris", }, + { + .compatible = "qcom,sm8550-iris", + .data = &sm8550_data, + }, { }, }; MODULE_DEVICE_TABLE(of, iris_dt_match); diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.c b/drivers/media/platform/qcom/vcodec/iris/platform_common.c new file mode 100644 index 0000000..043adc8 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.c @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include + +#include "iris_core.h" +#include "platform_common.h" + +int init_platform(struct iris_core *core) +{ + struct platform_data *platform = NULL; + + platform = devm_kzalloc(core->dev, sizeof(*platform), + GFP_KERNEL); + if (!platform) + return -ENOMEM; + + core->platform_data = platform; + + core->platform_data = (struct platform_data *)of_device_get_match_data(core->dev); + if (!core->platform_data) + return -ENODEV; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h new file mode 100644 index 0000000..e478b02 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -0,0 +1,194 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _PLATFORM_COMMON_H_ +#define _PLATFORM_COMMON_H_ + +#include + +struct iris_core; + +#define HW_RESPONSE_TIMEOUT_VALUE (1000) + +#define BIT_DEPTH_8 (8 << 16 | 8) +#define BIT_DEPTH_10 (10 << 16 | 10) + +#define CODED_FRAMES_PROGRESSIVE 0x0 +#define CODED_FRAMES_INTERLACE 0x1 + +#define DEFAULT_MAX_HOST_BUF_COUNT 64 +#define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256 + +#define UBWC_CONFIG(mc, ml, hbb, bs1, bs2, bs3, bsp) \ +{ \ + .max_channels = mc, \ + .mal_length = ml, \ + .highest_bank_bit = hbb, \ + .bank_swzl_level = bs1, \ + .bank_swz2_level = bs2, \ + .bank_swz3_level = bs3, \ + .bank_spreading = bsp, \ +} + +enum codec_type { + H264 = BIT(0), + HEVC = BIT(1), + VP9 = BIT(2), +}; + +enum colorformat_type { + FMT_NONE = 0, + FMT_NV12C = BIT(0), + FMT_NV12 = BIT(1), + FMT_NV21 = BIT(2), + FMT_TP10C = BIT(3), +}; + +enum stage_type { + STAGE_NONE = 0, + STAGE_1 = 1, + STAGE_2 = 2, +}; + +enum pipe_type { + PIPE_NONE = 0, + PIPE_1 = 1, + PIPE_2 = 2, + PIPE_4 = 4, +}; + +extern struct platform_data sm8550_data; + +struct bw_info { + u32 mbs_per_sec; + u32 bw_ddr; + u32 bw_ddr_10bit; +}; + +struct reg_preset_info { + u32 reg; + u32 value; + u32 mask; +}; + +struct ubwc_config_data { + u32 max_channels; + u32 mal_length; + u32 highest_bank_bit; + u32 bank_swzl_level; + u32 bank_swz2_level; + u32 bank_swz3_level; + u32 bank_spreading; +}; + +enum plat_core_cap_type { + CORE_CAP_NONE = 0, + DEC_CODECS, + MAX_SESSION_COUNT, + MAX_MBPF, + MAX_MBPS, + MAX_MBPF_HQ, + MAX_MBPS_HQ, + MAX_MBPF_B_FRAME, + MAX_MBPS_B_FRAME, + MAX_ENH_LAYER_COUNT, + NUM_VPP_PIPE, + FW_UNLOAD, + FW_UNLOAD_DELAY, + HW_RESPONSE_TIMEOUT, + NON_FATAL_FAULTS, + DMA_MASK, + CP_START, + CP_SIZE, + CP_NONPIXEL_START, + CP_NONPIXEL_SIZE, + CORE_CAP_MAX, +}; + +struct plat_core_cap { + enum plat_core_cap_type type; + u32 value; +}; + +enum plat_inst_cap_type { + INST_CAP_NONE = 0, + FRAME_WIDTH, + FRAME_HEIGHT, + PIX_FMTS, + MBPF, + QUEUED_RATE, + MB_CYCLES_VSP, + MB_CYCLES_VPP, + MB_CYCLES_LP, + MB_CYCLES_FW, + MB_CYCLES_FW_VPP, + NUM_COMV, + PROFILE, + LEVEL, + HEVC_TIER, + DISPLAY_DELAY_ENABLE, + DISPLAY_DELAY, + OUTPUT_ORDER, + INPUT_BUF_HOST_MAX_COUNT, + STAGE, + PIPE, + POC, + CODED_FRAMES, + BIT_DEPTH, + DEFAULT_HEADER, + RAP_FRAME, + INST_CAP_MAX, +}; + +enum plat_inst_cap_flags { + CAP_FLAG_NONE = 0, + CAP_FLAG_DYNAMIC_ALLOWED = BIT(0), + CAP_FLAG_MENU = BIT(1), + CAP_FLAG_INPUT_PORT = BIT(2), + CAP_FLAG_OUTPUT_PORT = BIT(3), + CAP_FLAG_CLIENT_SET = BIT(4), + CAP_FLAG_BITMASK = BIT(5), + CAP_FLAG_VOLATILE = BIT(6), +}; + +struct plat_inst_cap { + enum plat_inst_cap_type cap_id; + enum codec_type codec; + s32 min; + s32 max; + u32 step_or_mask; + s32 value; + u32 v4l2_id; + u32 hfi_id; + enum plat_inst_cap_flags flags; +}; + +struct platform_data { + const struct bus_info *bus_tbl; + unsigned int bus_tbl_size; + const struct bw_info *bw_tbl_dec; + unsigned int bw_tbl_dec_size; + const char * const *pd_tbl; + unsigned int pd_tbl_size; + const char * const *opp_pd_tbl; + unsigned int opp_pd_tbl_size; + const struct clock_info *clk_tbl; + unsigned int clk_tbl_size; + const char * const *clk_rst_tbl; + unsigned int clk_rst_tbl_size; + const struct reg_preset_info *reg_prst_tbl; + unsigned int reg_prst_tbl_size; + struct ubwc_config_data *ubwc_config; + const char *fwname; + u32 pas_id; + struct plat_core_cap *core_data; + u32 core_data_size; + struct plat_inst_cap *inst_cap_data; + u32 inst_cap_data_size; +}; + +int init_platform(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c new file mode 100644 index 0000000..c75017e --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -0,0 +1,316 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include + +#include + +#include "hfi_defines.h" +#include "platform_common.h" +#include "resources.h" + +#define CODECS_ALL (H264 | HEVC | VP9) + +#define DEFAULT_FPS 30 +#define MINIMUM_FPS 1 +#define MAXIMUM_FPS 480 + +static struct plat_core_cap core_data_sm8550[] = { + {DEC_CODECS, H264 | HEVC | VP9}, + {MAX_SESSION_COUNT, 16}, + {MAX_MBPF, 278528}, /* ((8192x4352)/256) * 2 */ + {MAX_MBPS, 7833600}, /* max_load 7680x4320@60fps */ + {NUM_VPP_PIPE, 4}, + {HW_RESPONSE_TIMEOUT, HW_RESPONSE_TIMEOUT_VALUE}, + {DMA_MASK, GENMASK(31, 29) - 1}, + {CP_START, 0}, + {CP_SIZE, 0x25800000}, + {CP_NONPIXEL_START, 0x01000000}, + {CP_NONPIXEL_SIZE, 0x24800000}, +}; + +static struct plat_inst_cap instance_cap_data_sm8550[] = { + {FRAME_WIDTH, CODECS_ALL, 96, 8192, 1, 1920}, + + {FRAME_WIDTH, VP9, 96, 4096, 1, 1920}, + + {FRAME_HEIGHT, CODECS_ALL, 96, 8192, 1, 1080}, + + {FRAME_HEIGHT, VP9, 96, 4096, 1, 1080}, + + {PIX_FMTS, H264, + FMT_NV12, + FMT_NV12C, + FMT_NV12 | FMT_NV21 | FMT_NV12C, + FMT_NV12C}, + + {PIX_FMTS, HEVC | VP9, + FMT_NV12, + FMT_TP10C, + FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, + FMT_NV12C}, + + {MBPF, CODECS_ALL, 36, 138240, 1, 138240}, + + /* (4096 * 2304) / 256 */ + {MBPF, VP9, 36, 36864, 1, 36864}, + + {QUEUED_RATE, CODECS_ALL, + (MINIMUM_FPS << 16), INT_MAX, + 1, (DEFAULT_FPS << 16)}, + + {MB_CYCLES_VSP, CODECS_ALL, 25, 25, 1, 25}, + + {MB_CYCLES_VSP, VP9, 60, 60, 1, 60}, + + {MB_CYCLES_VPP, CODECS_ALL, 200, 200, 1, 200}, + + {MB_CYCLES_LP, CODECS_ALL, 200, 200, 1, 200}, + + {MB_CYCLES_FW, CODECS_ALL, 489583, 489583, 1, 489583}, + + {MB_CYCLES_FW_VPP, CODECS_ALL, 66234, 66234, 1, 66234}, + + {NUM_COMV, CODECS_ALL, + 0, INT_MAX, 1, 0}, + + {PROFILE, H264, + V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH, + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH), + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + V4L2_CID_MPEG_VIDEO_H264_PROFILE, + HFI_PROP_PROFILE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {PROFILE, HEVC, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10, + BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) | + BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE) | + BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10), + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, + V4L2_CID_MPEG_VIDEO_HEVC_PROFILE, + HFI_PROP_PROFILE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {PROFILE, VP9, + V4L2_MPEG_VIDEO_VP9_PROFILE_0, + V4L2_MPEG_VIDEO_VP9_PROFILE_2, + BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) | + BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_2), + V4L2_MPEG_VIDEO_VP9_PROFILE_0, + V4L2_CID_MPEG_VIDEO_VP9_PROFILE, + HFI_PROP_PROFILE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {LEVEL, H264, + V4L2_MPEG_VIDEO_H264_LEVEL_1_0, + V4L2_MPEG_VIDEO_H264_LEVEL_6_2, + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_2), + V4L2_MPEG_VIDEO_H264_LEVEL_6_1, + V4L2_CID_MPEG_VIDEO_H264_LEVEL, + HFI_PROP_LEVEL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {LEVEL, HEVC, + V4L2_MPEG_VIDEO_HEVC_LEVEL_1, + V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2, + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2), + V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1, + V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, + HFI_PROP_LEVEL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {LEVEL, VP9, + V4L2_MPEG_VIDEO_VP9_LEVEL_1_0, + V4L2_MPEG_VIDEO_VP9_LEVEL_6_0, + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_1) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_0) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_0) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_0) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_0) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_6_0), + V4L2_MPEG_VIDEO_VP9_LEVEL_6_0, + V4L2_CID_MPEG_VIDEO_VP9_LEVEL, + HFI_PROP_LEVEL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {HEVC_TIER, HEVC, + V4L2_MPEG_VIDEO_HEVC_TIER_MAIN, + V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, + BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) | + BIT(V4L2_MPEG_VIDEO_HEVC_TIER_HIGH), + V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, + V4L2_CID_MPEG_VIDEO_HEVC_TIER, + HFI_PROP_TIER, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {DISPLAY_DELAY_ENABLE, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE, + HFI_PROP_DECODE_ORDER_OUTPUT, + CAP_FLAG_INPUT_PORT}, + + {DISPLAY_DELAY, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY, + HFI_PROP_DECODE_ORDER_OUTPUT, + CAP_FLAG_INPUT_PORT}, + + {OUTPUT_ORDER, CODECS_ALL, + 0, 1, 1, 0, + 0, + HFI_PROP_DECODE_ORDER_OUTPUT, + CAP_FLAG_INPUT_PORT}, + + {INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL, + DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT, + 1, DEFAULT_MAX_HOST_BUF_COUNT, + 0, + HFI_PROP_BUFFER_HOST_MAX_COUNT, + CAP_FLAG_INPUT_PORT}, + + {STAGE, CODECS_ALL, + STAGE_1, + STAGE_2, 1, + STAGE_2, + 0, + HFI_PROP_STAGE}, + + {PIPE, CODECS_ALL, + PIPE_1, + PIPE_4, 1, + PIPE_4, + 0, + HFI_PROP_PIPE}, + + {POC, H264, 0, 2, 1, 1, + 0, + HFI_PROP_PIC_ORDER_CNT_TYPE}, + + {CODED_FRAMES, H264 | HEVC, + CODED_FRAMES_PROGRESSIVE, CODED_FRAMES_PROGRESSIVE, + 0, CODED_FRAMES_PROGRESSIVE, + 0, + HFI_PROP_CODED_FRAMES}, + + {BIT_DEPTH, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8, + 0, + HFI_PROP_LUMA_CHROMA_BIT_DEPTH}, + + {DEFAULT_HEADER, CODECS_ALL, + 0, 1, 1, 0, + 0, + HFI_PROP_DEC_DEFAULT_HEADER}, + + {RAP_FRAME, CODECS_ALL, + 0, 1, 1, 1, + 0, + HFI_PROP_DEC_START_FROM_RAP_FRAME, + CAP_FLAG_INPUT_PORT}, +}; + +static const struct bus_info sm8550_bus_table[] = { + { NULL, "iris-cnoc", 1000, 1000 }, + { NULL, "iris-ddr", 1000, 15000000 }, +}; + +static const struct clock_info sm8550_clk_table[] = { + { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0 }, + { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0 }, + { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1 }, +}; + +static const char * const sm8550_clk_reset_table[] = { "video_axi_reset", NULL }; + +static const char * const sm8550_pd_table[] = { "iris-ctl", "vcodec", NULL }; + +static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx", NULL }; + +static const struct bw_info sm8550_bw_table_dec[] = { + { 2073600, 1608000, 2742000 }, /* 4096x2160@60 */ + { 1036800, 826000, 1393000 }, /* 4096x2160@30 */ + { 489600, 567000, 723000 }, /* 1920x1080@60 */ + { 244800, 294000, 372000 }, /* 1920x1080@30 */ +}; + +static const struct reg_preset_info sm8550_reg_preset_table[] = { + { 0xB0088, 0x0, 0x11 }, +}; + +static struct ubwc_config_data ubwc_config_sm8550[] = { + UBWC_CONFIG(8, 32, 16, 0, 1, 1, 1), +}; + +struct platform_data sm8550_data = { + .bus_tbl = sm8550_bus_table, + .bus_tbl_size = ARRAY_SIZE(sm8550_bus_table), + .clk_tbl = sm8550_clk_table, + .clk_tbl_size = ARRAY_SIZE(sm8550_clk_table), + .clk_rst_tbl = sm8550_clk_reset_table, + .clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table), + + .bw_tbl_dec = sm8550_bw_table_dec, + .bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec), + + .pd_tbl = sm8550_pd_table, + .pd_tbl_size = ARRAY_SIZE(sm8550_pd_table), + .opp_pd_tbl = sm8550_opp_pd_table, + .opp_pd_tbl_size = ARRAY_SIZE(sm8550_opp_pd_table), + + .reg_prst_tbl = sm8550_reg_preset_table, + .reg_prst_tbl_size = ARRAY_SIZE(sm8550_reg_preset_table), + .fwname = "vpu30_4v.mbn", + .pas_id = 9, + + .core_data = core_data_sm8550, + .core_data_size = ARRAY_SIZE(core_data_sm8550), + .inst_cap_data = instance_cap_data_sm8550, + .inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550), + .ubwc_config = ubwc_config_sm8550, +}; diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c index 5aebbe4..8cfdcd6 100644 --- a/drivers/media/platform/qcom/vcodec/iris/resources.c +++ b/drivers/media/platform/qcom/vcodec/iris/resources.c @@ -3,9 +3,6 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ -#include -#include - #include #include #include @@ -15,28 +12,9 @@ #include #include "iris_core.h" +#include "platform_common.h" #include "resources.h" -static const struct bus_info plat_bus_table[] = { - { NULL, "iris-cnoc", 1000, 1000 }, - { NULL, "iris-ddr", 1000, 15000000 }, -}; - -static const char * const plat_pd_table[] = { "iris-ctl", "vcodec", NULL }; -#define PD_COUNT 2 - -static const char * const plat_opp_pd_table[] = { "mxc", "mmcx", NULL }; -#define OPP_PD_COUNT 2 - -static const struct clock_info plat_clk_table[] = { - { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0, 0 }, - { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0, 0 }, - { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1, 0 }, -}; - -static const char * const plat_clk_reset_table[] = { "video_axi_reset", NULL }; -#define RESET_COUNT 1 - static void iris_pd_release(void *res) { struct device *pd = (struct device *)res; @@ -83,10 +61,13 @@ static int iris_opp_dl_get(struct device *dev, struct device *supplier) static int init_bus(struct iris_core *core) { + const struct bus_info *bus_tbl; struct bus_info *binfo = NULL; u32 i = 0; - core->bus_count = ARRAY_SIZE(plat_bus_table); + bus_tbl = core->platform_data->bus_tbl; + + core->bus_count = core->platform_data->bus_tbl_size; core->bus_tbl = devm_kzalloc(core->dev, sizeof(struct bus_info) * core->bus_count, GFP_KERNEL); @@ -95,9 +76,9 @@ static int init_bus(struct iris_core *core) for (i = 0; i < core->bus_count; i++) { binfo = &core->bus_tbl[i]; - binfo->name = plat_bus_table[i].name; - binfo->bw_min_kbps = plat_bus_table[i].bw_min_kbps; - binfo->bw_max_kbps = plat_bus_table[i].bw_max_kbps; + binfo->name = bus_tbl[i].name; + binfo->bw_min_kbps = bus_tbl[i].bw_min_kbps; + binfo->bw_max_kbps = bus_tbl[i].bw_max_kbps; binfo->icc = devm_of_icc_get(core->dev, binfo->name); if (IS_ERR(binfo->icc)) { dev_err(core->dev, @@ -112,20 +93,24 @@ static int init_bus(struct iris_core *core) static int init_power_domains(struct iris_core *core) { struct power_domain_info *pdinfo = NULL; + const char * const *opp_pd_tbl; + const char * const *pd_tbl; struct device **opp_vdevs = NULL; + u32 opp_pd_cnt, i; int ret; - u32 i; - core->pd_count = PD_COUNT; + pd_tbl = core->platform_data->pd_tbl; + + core->pd_count = core->platform_data->pd_tbl_size; core->power_domain_tbl = devm_kzalloc(core->dev, sizeof(struct power_domain_info) * core->pd_count, GFP_KERNEL); if (!core->power_domain_tbl) return -ENOMEM; - for (i = 0; i < core->pd_count; i++) { + for (i = 0; i < (core->pd_count - 1); i++) { pdinfo = &core->power_domain_tbl[i]; - pdinfo->name = plat_pd_table[i]; + pdinfo->name = pd_tbl[i]; ret = iris_pd_get(core, pdinfo); if (ret) { dev_err(core->dev, @@ -134,11 +119,14 @@ static int init_power_domains(struct iris_core *core) } } - ret = devm_pm_opp_attach_genpd(core->dev, plat_opp_pd_table, &opp_vdevs); + opp_pd_tbl = core->platform_data->opp_pd_tbl; + opp_pd_cnt = core->platform_data->opp_pd_tbl_size; + + ret = devm_pm_opp_attach_genpd(core->dev, opp_pd_tbl, &opp_vdevs); if (ret) return ret; - for (i = 0; i < OPP_PD_COUNT; i++) { + for (i = 0; i < (opp_pd_cnt - 1) ; i++) { ret = iris_opp_dl_get(core->dev, opp_vdevs[i]); if (ret) { dev_err(core->dev, "%s: failed to create dl: %s\n", @@ -158,10 +146,13 @@ static int init_power_domains(struct iris_core *core) static int init_clocks(struct iris_core *core) { + const struct clock_info *clk_tbl; struct clock_info *cinfo = NULL; u32 i; - core->clk_count = ARRAY_SIZE(plat_clk_table); + clk_tbl = core->platform_data->clk_tbl; + + core->clk_count = core->platform_data->clk_tbl_size; core->clock_tbl = devm_kzalloc(core->dev, sizeof(struct clock_info) * core->clk_count, GFP_KERNEL); @@ -170,9 +161,9 @@ static int init_clocks(struct iris_core *core) for (i = 0; i < core->clk_count; i++) { cinfo = &core->clock_tbl[i]; - cinfo->name = plat_clk_table[i].name; - cinfo->clk_id = plat_clk_table[i].clk_id; - cinfo->has_scaling = plat_clk_table[i].has_scaling; + cinfo->name = clk_tbl[i].name; + cinfo->clk_id = clk_tbl[i].clk_id; + cinfo->has_scaling = clk_tbl[i].has_scaling; cinfo->clk = devm_clk_get(core->dev, cinfo->name); if (IS_ERR(cinfo->clk)) { dev_err(core->dev, @@ -187,18 +178,21 @@ static int init_clocks(struct iris_core *core) static int init_reset_clocks(struct iris_core *core) { struct reset_info *rinfo = NULL; + const char * const *rst_tbl; u32 i = 0; - core->reset_count = RESET_COUNT; + rst_tbl = core->platform_data->clk_rst_tbl; + + core->reset_count = core->platform_data->clk_rst_tbl_size; core->reset_tbl = devm_kzalloc(core->dev, sizeof(struct reset_info) * core->reset_count, GFP_KERNEL); if (!core->reset_tbl) return -ENOMEM; - for (i = 0; i < core->reset_count; i++) { + for (i = 0; i < (core->reset_count - 1); i++) { rinfo = &core->reset_tbl[i]; - rinfo->name = plat_clk_reset_table[i]; + rinfo->name = rst_tbl[i]; rinfo->rst = devm_reset_control_get(core->dev, rinfo->name); if (IS_ERR(rinfo->rst)) { dev_err(core->dev, From patchwork Mon Dec 18 11:32:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757238 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA19B39AE0; Mon, 18 Dec 2023 11:40:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="VwhnCAj2" Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAssR1018364; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Fn5MQyElIA/ne+j5QqgL6eLBCk/yCOj7dBOuGhLSDLY=; b=Vw hnCAj2m/ZguBQ/kyO8yLolpbgB7v3QRrs+7APsLIez2aX7o6FUIFxdihm3phRqyR 5zfSWPSS67V9jM3mPEzDfXg1gYDoPBUy8b9YH65wngMYqlR4I1HsU/57dLvzDw8V w1U3qQf1KhairlIFbIu+oVdvK9Sveqwn+XN7kmuFHRjAWihskIQrPDIg9Y/rK53D Xq1N7lzzsDF+CdZ2+sh7OaPFm4hL2OIeTkVNTCjS4hGaiI+K7u37mLcNk+kE7XY8 uxxQJcU5DiUHceXFjVl8IHIZhdIExG72pDt1yHFQshM0mMyhevQ46LwP8ObFqcay D6HssxSZxBptbxHF8cyA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v156dm7qc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX6PO029956; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ad-4; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX77k029983; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6Iw029963; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 420872305; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 14/34] media: iris: implement iris v4l2 file ops Date: Mon, 18 Dec 2023 17:02:09 +0530 Message-Id: <1702899149-21321-15-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: VPU_gMY1KiK2LJPhnV5D2jrAguq5PQvN X-Proofpoint-GUID: VPU_gMY1KiK2LJPhnV5D2jrAguq5PQvN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 lowpriorityscore=0 bulkscore=0 impostorscore=0 mlxscore=0 spamscore=0 priorityscore=1501 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Implements iris v4l2 file ops - open, close and poll. Open: Configure the vb2 queue and v4l2 file handler. Initialize a video instance and add the instance to core instance list. Open a session in video firmware via HFI_CMD_OPEN. Close: De-initialize the instance and remove it from core instance list. Close the session in firmware via HFI_CMD_CLOSE. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 2 + .../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 + .../media/platform/qcom/vcodec/iris/iris_common.h | 26 ++ .../media/platform/qcom/vcodec/iris/iris_core.c | 42 ++- .../media/platform/qcom/vcodec/iris/iris_core.h | 8 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 62 +++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 2 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 115 +++++++- drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 7 + .../platform/qcom/vcodec/iris/iris_instance.h | 48 ++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 10 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 47 ++++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 14 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 298 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 ++ 17 files changed, 719 insertions(+), 10 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index c50e3241..3c076d0 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -3,6 +3,8 @@ iris-objs += ../hfi_queue.o ../firmware.o iris-objs += iris_probe.o \ iris_state.o \ iris_core.o \ + iris_vidc.o \ + iris_vdec.o \ iris_state.o \ iris_helpers.o \ iris_hfi.o \ diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index fb383b2..423ba1a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -9,6 +9,8 @@ #define HFI_VIDEO_ARCH_LX 0x1 #define HFI_CMD_INIT 0x01000001 +#define HFI_CMD_OPEN 0x01000003 +#define HFI_CMD_CLOSE 0x01000004 #define HFI_PROP_IMAGE_VERSION 0x03000001 diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h new file mode 100644 index 0000000..3e4dd71 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef _IRIS_COMMON_H_ +#define _IRIS_COMMON_H_ + +#include +#include +#include +#include + +#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE +#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE +#define DEFAULT_WIDTH 320 +#define DEFAULT_HEIGHT 240 +#define DEFAULT_BUF_SIZE 115200 + +enum signal_session_response { + SIGNAL_CMD_STOP_INPUT = 0, + SIGNAL_CMD_STOP_OUTPUT, + SIGNAL_CMD_CLOSE, + MAX_SIGNAL, +}; + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c index ba8960d..174fad9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c @@ -3,12 +3,14 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_core.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_state.h" -static int iris_core_deinit_locked(struct iris_core *core) +int iris_core_deinit_locked(struct iris_core *core) { int ret; @@ -68,3 +70,41 @@ int iris_core_init(struct iris_core *core) return ret; } + +int iris_core_init_wait(struct iris_core *core) +{ + const int interval = 10; + int max_tries, count = 0, ret = 0; + + mutex_lock(&core->lock); + if (!core_in_valid_state(core)) { + ret = -EINVAL; + goto unlock; + } + + if (core->state == IRIS_CORE_INIT) + goto unlock; + + max_tries = core->cap[HW_RESPONSE_TIMEOUT].value / interval; + while (count < max_tries) { + if (core->state != IRIS_CORE_INIT_WAIT) + break; + msleep(interval); + count++; + } + + if (core->state == IRIS_CORE_INIT) { + ret = 0; + goto unlock; + } else { + iris_change_core_state(core, IRIS_CORE_ERROR); + iris_core_deinit_locked(core); + ret = -EINVAL; + goto unlock; + } + +unlock: + mutex_unlock(&core->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 3c8497a..b47520a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -33,6 +33,8 @@ * @clk_count: count of iris clocks * @reset_tbl: table of iris reset clocks * @reset_count: count of iris reset clocks + * @vb2_ops: iris vb2 ops + * @vb2_mem_ops: iris vb2 memory ops * @state: current state of core * @iface_q_table: Interface queue table memory * @command_queue: shared interface queue to send commands to firmware @@ -49,6 +51,7 @@ * @vpu_ops: a pointer to vpu ops * @platform_data: a structure for platform data * @cap: an array for supported core capabilities + * @instances: a list_head of all instances */ struct iris_core { @@ -67,6 +70,8 @@ struct iris_core { u32 clk_count; struct reset_info *reset_tbl; u32 reset_count; + const struct vb2_ops *vb2_ops; + struct vb2_mem_ops *vb2_mem_ops; enum iris_core_state state; struct mem_desc iface_q_table; struct iface_q_info command_queue; @@ -83,9 +88,12 @@ struct iris_core { const struct vpu_ops *vpu_ops; struct platform_data *platform_data; struct plat_core_cap cap[CORE_CAP_MAX + 1]; + struct list_head instances; }; int iris_core_init(struct iris_core *core); +int iris_core_init_wait(struct iris_core *core); int iris_core_deinit(struct iris_core *core); +int iris_core_deinit_locked(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index c31dfd5..872268d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -4,6 +4,8 @@ */ #include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_instance.h" int check_core_lock(struct iris_core *core) { @@ -32,3 +34,63 @@ int iris_init_core_caps(struct iris_core *core) return 0; } + +static int process_inst_timeout(struct iris_inst *inst) +{ + struct iris_inst *instance; + struct iris_core *core; + bool found = false; + int ret = 0; + + core = inst->core; + + mutex_lock(&core->lock); + list_for_each_entry(instance, &core->instances, list) { + if (instance == inst) { + found = true; + break; + } + } + if (!found) { + ret = -EINVAL; + goto unlock; + } + + iris_change_core_state(core, IRIS_CORE_ERROR); + + iris_core_deinit_locked(core); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int close_session(struct iris_inst *inst) +{ + u32 hw_response_timeout_val; + bool wait_for_response; + struct iris_core *core; + int ret; + + core = inst->core; + hw_response_timeout_val = core->cap[HW_RESPONSE_TIMEOUT].value; + wait_for_response = true; + ret = iris_hfi_session_close(inst); + if (ret) + wait_for_response = false; + + kfree(inst->packet); + inst->packet = NULL; + + if (wait_for_response) { + ret = wait_for_completion_timeout(&inst->completions[SIGNAL_CMD_CLOSE], + msecs_to_jiffies(hw_response_timeout_val)); + if (!ret) { + ret = -ETIMEDOUT; + process_inst_timeout(inst); + } + } + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 4c7ddbf..b9a6485 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -7,8 +7,10 @@ #define _IRIS_HELPERS_H_ #include "iris_core.h" +#include "iris_instance.h" int check_core_lock(struct iris_core *core); int iris_init_core_caps(struct iris_core *core); +int close_session(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index b0390b542..d144ae5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -8,6 +8,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" +#include "hfi_defines.h" #include "vpu_common.h" static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) @@ -43,6 +44,27 @@ static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) return ret; } +static bool validate_session(struct iris_core *core, + struct iris_inst *inst) +{ + struct iris_inst *temp; + bool valid = false; + int ret; + + ret = check_core_lock(core); + if (ret) + return false; + + list_for_each_entry(temp, &core->instances, list) { + if (temp == inst) { + valid = true; + break; + } + } + + return valid; +} + static int sys_init(struct iris_core *core) { int ret; @@ -69,14 +91,6 @@ static int sys_image_version(struct iris_core *core) return ret; } -#define CP_START 0 -#define CP_SIZE 0x25800000 -#define CP_NONPIXEL_START 0x01000000 -#define CP_NONPIXEL_SIZE 0x24800000 - -#define FW_NAME "vpu30_4v.mbn" -#define IRIS_PAS_ID 9 - int iris_hfi_core_init(struct iris_core *core) { u32 cp_nonpixel_start, cp_nonpixel_size; @@ -152,7 +166,90 @@ int iris_hfi_core_deinit(struct iris_core *core) if (core->state == IRIS_CORE_DEINIT) return 0; - unload_fw(IRIS_PAS_ID); + unload_fw(core->platform_data->pas_id); + + return ret; +} + +int iris_hfi_session_open(struct iris_inst *inst) +{ + struct iris_core *core; + int ret; + + inst->packet_size = 4096; + inst->packet = kzalloc(inst->packet_size, GFP_KERNEL); + if (!inst->packet) + return -ENOMEM; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto fail_free_packet; + } + + ret = hfi_packet_session_command(inst, + HFI_CMD_OPEN, + HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED, + HFI_PORT_NONE, + 0, + HFI_PAYLOAD_U32, + &inst->session_id, + sizeof(u32)); + if (ret) + goto fail_free_packet; + + ret = iris_hfi_queue_cmd_write(core, inst->packet); + if (ret) + goto fail_free_packet; + + mutex_unlock(&core->lock); + + return ret; + +fail_free_packet: + kfree(inst->packet); + inst->packet = NULL; + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_session_close(struct iris_inst *inst) +{ + struct iris_core *core; + int ret; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + ret = hfi_packet_session_command(inst, + HFI_CMD_CLOSE, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + HFI_PORT_NONE, + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); return ret; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index fcf9f28..8a057cc 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -6,9 +6,12 @@ #ifndef _IRIS_HFI_H_ #define _IRIS_HFI_H_ +#include "iris_instance.h" #include "iris_core.h" int iris_hfi_core_init(struct iris_core *core); int iris_hfi_core_deinit(struct iris_core *core); +int iris_hfi_session_open(struct iris_inst *inst); +int iris_hfi_session_close(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index f31dd84..1ed572e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -211,3 +211,31 @@ int hfi_packet_image_version(struct iris_core *core, return ret; } + +int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, + u32 flags, u32 port, u32 session_id, + u32 payload_type, void *payload, + u32 payload_size) +{ + struct iris_core *core; + int ret; + + core = inst->core; + + ret = hfi_create_header(inst->packet, inst->packet_size, + session_id, core->header_id++); + if (ret) + return ret; + + ret = hfi_create_packet(inst->packet, + inst->packet_size, + pkt_type, + flags, + payload_type, + port, + core->packet_id++, + payload, + payload_size); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index e36612c..9e476e9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -6,6 +6,9 @@ #ifndef _IRIS_HFI_PACKET_H_ #define _IRIS_HFI_PACKET_H_ +#include "iris_core.h" +#include "iris_instance.h" + struct hfi_header { u32 size; u32 session_id; @@ -67,5 +70,9 @@ int hfi_packet_sys_init(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_image_version(struct iris_core *core, u8 *pkt, u32 pkt_size); +int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, + u32 flags, u32 port, u32 session_id, + u32 payload_type, void *payload, + u32 payload_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h new file mode 100644 index 0000000..1bbb533 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_INSTANCE_H_ +#define _IRIS_INSTANCE_H_ + +#include + +#include "iris_core.h" +#include "iris_common.h" + +/** + * struct iris_inst - holds per video instance parameters + * + * @list: used for attach an instance to the core + * @core: pointer to core structure + * @session_id: id of current video session + * @vb2q_src: source vb2 queue + * @vb2q_dst: destination vb2 queue + * @ctx_q_lock: lock to serialize queues related ioctls + * @fh: reference of v4l2 file handler + * @fmt_src: structure of v4l2_format for source + * @fmt_dst: structure of v4l2_format for destination + * @ctrl_handler: reference of v4l2 ctrl handler + * @packet: HFI packet + * @packet_size: HFI packet size + * @completions: structure of signal completions + */ + +struct iris_inst { + struct list_head list; + struct iris_core *core; + u32 session_id; + struct vb2_queue *vb2q_src; + struct vb2_queue *vb2q_dst; + struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */ + struct v4l2_fh fh; + struct v4l2_format *fmt_src; + struct v4l2_format *fmt_dst; + struct v4l2_ctrl_handler ctrl_handler; + void *packet; + u32 packet_size; + struct completion completions[MAX_SIGNAL]; +}; + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 9d6a6c5..4f20da8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -11,6 +11,7 @@ #include "iris_core.h" #include "iris_helpers.h" #include "resources.h" +#include "iris_vidc.h" static int iris_register_video_device(struct iris_core *core) { @@ -85,6 +86,8 @@ static int iris_probe(struct platform_device *pdev) if (!core->packet) return -ENOMEM; + INIT_LIST_HEAD(&core->instances); + core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) return PTR_ERR(core->reg_base); @@ -107,6 +110,13 @@ static int iris_probe(struct platform_device *pdev) return ret; } + ret = init_ops(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init ops failed with %d\n", __func__, ret); + return ret; + } + ret = init_resources(core); if (ret) { dev_err_probe(core->dev, ret, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c new file mode 100644 index 0000000..984be34 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_common.h" +#include "iris_vdec.h" + +void vdec_inst_init(struct iris_inst *inst) +{ + struct v4l2_format *f; + + inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL); + inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL); + inst->vb2q_src = kzalloc(sizeof(*inst->vb2q_src), GFP_KERNEL); + inst->vb2q_dst = kzalloc(sizeof(*inst->vb2q_dst), GFP_KERNEL); + + f = inst->fmt_src; + f->type = INPUT_MPLANE; + f->fmt.pix_mp.width = DEFAULT_WIDTH; + f->fmt.pix_mp.height = DEFAULT_HEIGHT; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264; + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = 0; + f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + + f = inst->fmt_dst; + f->type = OUTPUT_MPLANE; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_QC08C; + f->fmt.pix_mp.width = ALIGN(DEFAULT_WIDTH, 128); + f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32); + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128); + f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; +} + +void vdec_inst_deinit(struct iris_inst *inst) +{ + kfree(inst->fmt_dst); + kfree(inst->fmt_src); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h new file mode 100644 index 0000000..dc8f43f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VDEC_H_ +#define _IRIS_VDEC_H_ + +#include "iris_instance.h" + +void vdec_inst_init(struct iris_inst *inst); +void vdec_inst_deinit(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c new file mode 100644 index 0000000..d02da8b --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -0,0 +1,298 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_common.h" +#include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_instance.h" +#include "iris_vdec.h" +#include "iris_vidc.h" + +static int vidc_v4l2_fh_init(struct iris_inst *inst) +{ + struct iris_core *core; + + core = inst->core; + + if (inst->fh.vdev) + return -EINVAL; + + v4l2_fh_init(&inst->fh, core->vdev_dec); + inst->fh.ctrl_handler = &inst->ctrl_handler; + v4l2_fh_add(&inst->fh); + + return 0; +} + +static int vidc_v4l2_fh_deinit(struct iris_inst *inst) +{ + if (!inst->fh.vdev) + return 0; + + v4l2_fh_del(&inst->fh); + inst->fh.ctrl_handler = NULL; + v4l2_fh_exit(&inst->fh); + + return 0; +} + +static int vb2q_init(struct iris_inst *inst, + struct vb2_queue *q, enum v4l2_buf_type type) +{ + struct iris_core *core; + + core = inst->core; + + q->lock = &inst->ctx_q_lock; + q->type = type; + q->io_modes = VB2_MMAP | VB2_DMABUF; + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + q->ops = core->vb2_ops; + q->mem_ops = core->vb2_mem_ops; + q->drv_priv = inst; + q->copy_timestamp = 1; + q->min_buffers_needed = 0; + return vb2_queue_init(q); +} + +static int vidc_vb2_queue_init(struct iris_inst *inst) +{ + int ret; + + ret = vb2q_init(inst, inst->vb2q_src, INPUT_MPLANE); + if (ret) + return ret; + + ret = vb2q_init(inst, inst->vb2q_dst, OUTPUT_MPLANE); + if (ret) + goto fail_vb2q_src_deinit; + + return ret; + +fail_vb2q_src_deinit: + vb2_queue_release(inst->vb2q_src); + + return ret; +} + +static int vidc_vb2_queue_deinit(struct iris_inst *inst) +{ + vb2_queue_release(inst->vb2q_src); + kfree(inst->vb2q_src); + inst->vb2q_src = NULL; + + vb2_queue_release(inst->vb2q_dst); + kfree(inst->vb2q_dst); + inst->vb2q_dst = NULL; + + return 0; +} + +static int vidc_add_session(struct iris_inst *inst) +{ + struct iris_core *core; + struct iris_inst *i; + u32 count = 0; + int ret = 0; + + core = inst->core; + + mutex_lock(&core->lock); + if (core->state != IRIS_CORE_INIT) { + ret = -EINVAL; + goto unlock; + } + list_for_each_entry(i, &core->instances, list) + count++; + + if (count < core->cap[MAX_SESSION_COUNT].value) + list_add_tail(&inst->list, &core->instances); + else + ret = -EAGAIN; +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +static int vidc_remove_session(struct iris_inst *inst) +{ + struct iris_inst *i, *temp; + struct iris_core *core; + + core = inst->core; + + mutex_lock(&core->lock); + list_for_each_entry_safe(i, temp, &core->instances, list) { + if (i->session_id == inst->session_id) { + list_del_init(&i->list); + break; + } + } + mutex_unlock(&core->lock); + + return 0; +} + +static struct iris_inst *get_vidc_inst(struct file *filp, void *fh) +{ + if (!filp || !filp->private_data) + return NULL; + + return container_of(filp->private_data, + struct iris_inst, fh); +} + +int vidc_open(struct file *filp) +{ + struct iris_core *core = video_drvdata(filp); + struct iris_inst *inst = NULL; + int i = 0; + int ret; + + ret = iris_core_init(core); + if (ret) + return ret; + + ret = iris_core_init_wait(core); + if (ret) + return ret; + + inst = kzalloc(sizeof(*inst), GFP_KERNEL); + if (!inst) + return -ENOMEM; + + inst->core = core; + inst->session_id = hash32_ptr(inst); + mutex_init(&inst->ctx_q_lock); + for (i = 0; i < MAX_SIGNAL; i++) + init_completion(&inst->completions[i]); + + ret = vidc_add_session(inst); + if (ret) + goto fail_free_inst; + + ret = vidc_v4l2_fh_init(inst); + if (ret) + goto fail_remove_session; + + vdec_inst_init(inst); + + ret = vidc_vb2_queue_init(inst); + if (ret) + goto fail_inst_deinit; + + ret = iris_hfi_session_open(inst); + if (ret) { + dev_err(core->dev, "%s: session open failed\n", __func__); + goto fail_core_deinit; + } + filp->private_data = &inst->fh; + + return 0; + +fail_core_deinit: + iris_core_deinit(core); + vidc_vb2_queue_deinit(inst); +fail_inst_deinit: + vdec_inst_deinit(inst); + vidc_v4l2_fh_deinit(inst); +fail_remove_session: + vidc_remove_session(inst); +fail_free_inst: + mutex_destroy(&inst->ctx_q_lock); + kfree(inst); + + return ret; +} + +int vidc_close(struct file *filp) +{ + struct iris_inst *inst; + + inst = get_vidc_inst(filp, NULL); + if (!inst) + return -EINVAL; + + vdec_inst_deinit(inst); + close_session(inst); + vidc_vb2_queue_deinit(inst); + vidc_v4l2_fh_deinit(inst); + vidc_remove_session(inst); + mutex_destroy(&inst->ctx_q_lock); + kfree(inst); + + filp->private_data = NULL; + + return 0; +} + +static __poll_t get_poll_flags(struct iris_inst *inst, u32 plane) +{ + struct vb2_buffer *vb = NULL; + struct vb2_queue *q = NULL; + unsigned long flags = 0; + __poll_t poll = 0; + + if (plane == INPUT_MPLANE) + q = inst->vb2q_src; + else if (plane == OUTPUT_MPLANE) + q = inst->vb2q_dst; + + if (!q) + return EPOLLERR; + + spin_lock_irqsave(&q->done_lock, flags); + if (!list_empty(&q->done_list)) + vb = list_first_entry(&q->done_list, struct vb2_buffer, + done_entry); + if (vb && (vb->state == VB2_BUF_STATE_DONE || + vb->state == VB2_BUF_STATE_ERROR)) { + if (plane == OUTPUT_MPLANE) + poll |= EPOLLIN | EPOLLRDNORM; + else if (plane == INPUT_MPLANE) + poll |= EPOLLOUT | EPOLLWRNORM; + } + spin_unlock_irqrestore(&q->done_lock, flags); + + return poll; +} + +static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt) +{ + struct iris_inst *inst; + __poll_t poll = 0; + + inst = get_vidc_inst(filp, NULL); + if (!inst) + return EPOLLERR; + + poll_wait(filp, &inst->fh.wait, pt); + poll_wait(filp, &inst->vb2q_src->done_wq, pt); + poll_wait(filp, &inst->vb2q_dst->done_wq, pt); + + if (v4l2_event_pending(&inst->fh)) + poll |= EPOLLPRI; + + poll |= get_poll_flags(inst, INPUT_MPLANE); + poll |= get_poll_flags(inst, OUTPUT_MPLANE); + + return poll; +} + +static const struct v4l2_file_operations v4l2_file_ops = { + .owner = THIS_MODULE, + .open = vidc_open, + .release = vidc_close, + .unlocked_ioctl = video_ioctl2, + .poll = vidc_poll, +}; + +int init_ops(struct iris_core *core) +{ + core->v4l2_file_ops = &v4l2_file_ops; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h new file mode 100644 index 0000000..aebff44 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VIDC_H_ +#define _IRIS_VIDC_H_ + +#include "iris_core.h" + +int init_ops(struct iris_core *core); +int vidc_open(struct file *filp); +int vidc_close(struct file *filp); + +#endif From patchwork Mon Dec 18 11:32:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757243 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E540A1A738; Mon, 18 Dec 2023 11:36:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="ncHS4f50" Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBXju6014283; Mon, 18 Dec 2023 11:36:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=UVtFoZQTiLhQyF2RBCz838+WNd8b3K5QQcCs6KV3zyg=; b=nc HS4f50Ya9mYZG3RtiqOeL1YvVol3QLvfJqoih/w0nDwmjuXpBGIkC9amng+5udxL ejl35DPgrP5Ty5hnrvURo4/bmpTIXFvN5oSCelhC0VXsRu3933w0D2dOJqqvp+iN ypXKhpv8epRY28aAnzHsTiwVvruhipFiv8mKmrnfo7Tx/bMQOWB9bTDqafXPN7If 6ANizV3KM8onTnDxtwh+vV7Wml9F6tubCKWVHBw2WwdDCgpa4fn4mvVtE7/1O81f JD8DXf+kxTI0kW5+2ZK1HQy68ECxmM1f8i7hMuT7vMprkc9Dmf8YSa1J3rp9H4+b 6Hdiw21pNenWsrDiskkA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v156dm7f3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:13 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmpN028250; Mon, 18 Dec 2023 11:36:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyv2-1; Mon, 18 Dec 2023 11:36:09 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmqt028240; Mon, 18 Dec 2023 11:36:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6kk029964; Mon, 18 Dec 2023 11:36:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 44B292306; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware Date: Mon, 18 Dec 2023 17:02:10 +0530 Message-Id: <1702899149-21321-16-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 4taH3vpbOrmM7Mi-RerpCbJpvu8R3cCg X-Proofpoint-GUID: 4taH3vpbOrmM7Mi-RerpCbJpvu8R3cCg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 lowpriorityscore=0 bulkscore=0 impostorscore=0 mlxscore=0 spamscore=0 priorityscore=1501 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Allocate interrupt resources, enable the interrupt line and IRQ handling. Register the IRQ handler to be called when interrupt occurs and the function to be called from IRQ handler thread. The threads invoke the driver's response handler which handles all different responses from firmware. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 1 + .../media/platform/qcom/vcodec/iris/hfi_defines.h | 10 ++ .../media/platform/qcom/vcodec/iris/iris_core.h | 4 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 29 +++- drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 + .../platform/qcom/vcodec/iris/iris_hfi_response.c | 184 +++++++++++++++++++++ .../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 27 +++ .../media/platform/qcom/vcodec/iris/vpu_common.h | 2 + .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 44 +++++ 10 files changed, 323 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 3c076d0..85fdf63 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -8,6 +8,7 @@ iris-objs += iris_probe.o \ iris_state.o \ iris_helpers.o \ iris_hfi.o \ + iris_hfi_response.o \ iris_hfi_packet.o \ resources.o \ vpu_common.o \ diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index 423ba1a..9dd5f11 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -6,6 +6,8 @@ #ifndef _HFI_DEFINES_H_ #define _HFI_DEFINES_H_ +#include + #define HFI_VIDEO_ARCH_LX 0x1 #define HFI_CMD_INIT 0x01000001 @@ -50,4 +52,12 @@ #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 +#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 + +struct hfi_debug_header { + u32 size; + u32 debug_level; + u32 reserved[2]; +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index b47520a..c125bce 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -45,6 +45,7 @@ * @use_tz: a flag that suggests presence of trustzone * @packet: pointer to packet from driver to fw * @packet_size: size of packet + * @response_packet: a pointer to response packet from fw to driver * @sys_init_id: id of sys init packet * @header_id: id of packet header * @packet_id: id of packet @@ -52,6 +53,7 @@ * @platform_data: a structure for platform data * @cap: an array for supported core capabilities * @instances: a list_head of all instances + * @intr_status: interrupt status */ struct iris_core { @@ -82,6 +84,7 @@ struct iris_core { unsigned int use_tz; u8 *packet; u32 packet_size; + u8 *response_packet; u32 sys_init_id; u32 header_id; u32 packet_id; @@ -89,6 +92,7 @@ struct iris_core { struct platform_data *platform_data; struct plat_core_cap cap[CORE_CAP_MAX + 1]; struct list_head instances; + u32 intr_status; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index d144ae5..4b8d8c7 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -5,10 +5,11 @@ #include "../firmware.h" #include "../hfi_queue.h" +#include "hfi_defines.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" -#include "hfi_defines.h" +#include "iris_hfi_response.h" #include "vpu_common.h" static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) @@ -253,3 +254,29 @@ int iris_hfi_session_close(struct iris_inst *inst) return ret; } + +irqreturn_t iris_hfi_isr(int irq, void *data) +{ + disable_irq_nosync(irq); + + return IRQ_WAKE_THREAD; +} + +irqreturn_t iris_hfi_isr_handler(int irq, void *data) +{ + struct iris_core *core = data; + + if (!core) + return IRQ_NONE; + + mutex_lock(&core->lock); + call_vpu_op(core, clear_interrupt, core); + mutex_unlock(&core->lock); + + __response_handler(core); + + if (!call_vpu_op(core, watchdog, core, core->intr_status)) + enable_irq(irq); + + return IRQ_HANDLED; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index 8a057cc..8a62986 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -14,4 +14,7 @@ int iris_hfi_core_deinit(struct iris_core *core); int iris_hfi_session_open(struct iris_inst *inst); int iris_hfi_session_close(struct iris_inst *inst); +irqreturn_t iris_hfi_isr(int irq, void *data); +irqreturn_t iris_hfi_isr_handler(int irq, void *data); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c new file mode 100644 index 0000000..829f3f6 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "hfi_defines.h" +#include "iris_hfi_packet.h" +#include "iris_hfi_response.h" + +static void print_sfr_message(struct iris_core *core) +{ + struct sfr_buffer *vsfr = NULL; + u32 vsfr_size = 0; + void *p = NULL; + + vsfr = (struct sfr_buffer *)core->sfr.kernel_vaddr; + if (vsfr) { + if (vsfr->bufsize != core->sfr.size) + return; + vsfr_size = vsfr->bufsize - sizeof(u32); + p = memchr(vsfr->rg_data, '\0', vsfr_size); + /* SFR isn't guaranteed to be NULL terminated */ + if (!p) + vsfr->rg_data[vsfr_size - 1] = '\0'; + } +} + +static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, + u32 core_resp_pkt_size) +{ + u32 response_pkt_size = 0; + u8 *response_limit; + + if (!response_pkt || !core_resp_pkt || !core_resp_pkt_size) + return -EINVAL; + + response_limit = core_resp_pkt + core_resp_pkt_size; + + if (response_pkt < core_resp_pkt || response_pkt > response_limit) + return -EINVAL; + + response_pkt_size = *(u32 *)response_pkt; + if (!response_pkt_size) + return -EINVAL; + + if (response_pkt_size < sizeof(struct hfi_packet)) + return -EINVAL; + + if (response_pkt + response_pkt_size > response_limit) + return -EINVAL; + + return 0; +} + +static int validate_hdr_packet(struct iris_core *core, + struct hfi_header *hdr) +{ + struct hfi_packet *packet; + int i, ret = 0; + u8 *pkt; + + if (hdr->size < sizeof(*hdr) + sizeof(*packet)) + return -EINVAL; + + pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + + for (i = 0; i < hdr->num_packets; i++) { + packet = (struct hfi_packet *)pkt; + ret = validate_packet(pkt, core->response_packet, core->packet_size); + if (ret) + return ret; + + pkt += packet->size; + } + + return ret; +} + +static int handle_system_error(struct iris_core *core, + struct hfi_packet *pkt) +{ + print_sfr_message(core); + + iris_core_deinit(core); + + return 0; +} + +static int handle_response(struct iris_core *core, void *response) +{ + struct hfi_header *hdr; + int ret; + + hdr = (struct hfi_header *)response; + ret = validate_hdr_packet(core, hdr); + if (ret) + return handle_system_error(core, NULL); + + return ret; +} + +static int iris_hfi_queue_read(struct iris_core *core, void *pkt, + struct iface_q_info *q_info) +{ + u32 tx_req; + + if (!core_in_valid_state(core)) + return -EINVAL; + + if (!q_info || !q_info->q_array.kernel_vaddr || !pkt) { + dev_err(core->dev, "cannot read from shared Q's\n"); + return -ENODATA; + } + + if (read_queue(q_info, pkt, &tx_req)) + return -ENODATA; + + return 0; +} + +int __response_handler(struct iris_core *core) +{ + int ret = 0; + + if (call_vpu_op(core, watchdog, core, core->intr_status)) { + struct hfi_packet pkt = {.type = HFI_SYS_ERROR_WD_TIMEOUT}; + + mutex_lock(&core->lock); + iris_change_core_state(core, IRIS_CORE_ERROR); + dev_err(core->dev, "%s: CPU WD error received\n", __func__); + mutex_unlock(&core->lock); + + return handle_system_error(core, &pkt); + } + + memset(core->response_packet, 0, core->packet_size); + while (!iris_hfi_queue_read(core, core->response_packet, &core->message_queue)) { + ret = handle_response(core, core->response_packet); + if (ret) + continue; + if (core->state != IRIS_CORE_INIT) + break; + memset(core->response_packet, 0, core->packet_size); + } + + iris_flush_debug_queue(core, core->response_packet, core->packet_size); + + return ret; +} + +void iris_flush_debug_queue(struct iris_core *core, + u8 *packet, u32 packet_size) +{ + struct hfi_debug_header *pkt; + bool local_packet = false; + u8 *log; + + if (!packet || !packet_size) { + packet = kzalloc(IFACEQ_CORE_PKT_SIZE, GFP_KERNEL); + if (!packet) + return; + + packet_size = IFACEQ_CORE_PKT_SIZE; + + local_packet = true; + } + + while (!iris_hfi_queue_read(core, packet, &core->debug_queue)) { + pkt = (struct hfi_debug_header *)packet; + + if (pkt->size < sizeof(*pkt)) + continue; + + if (pkt->size >= packet_size) + continue; + + packet[pkt->size] = '\0'; + log = (u8 *)packet + sizeof(*pkt) + 1; + dev_dbg(core->dev, "%s", log); + } + + if (local_packet) + kfree(packet); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h new file mode 100644 index 0000000..c1d2d27 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_RESPONSE_H_ +#define _IRIS_HFI_RESPONSE_H_ + +#include "iris_core.h" + +struct sfr_buffer { + u32 bufsize; + u8 rg_data[]; +}; + +int __response_handler(struct iris_core *core); +void iris_flush_debug_queue(struct iris_core *core, + u8 *packet, u32 packet_size); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 4f20da8..8c591da 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -10,9 +10,25 @@ #include "../hfi_queue.h" #include "iris_core.h" #include "iris_helpers.h" +#include "iris_hfi.h" #include "resources.h" #include "iris_vidc.h" +static int init_iris_isr(struct iris_core *core) +{ + int ret; + + ret = devm_request_threaded_irq(core->dev, core->irq, iris_hfi_isr, + iris_hfi_isr_handler, IRQF_TRIGGER_HIGH, "iris", core); + if (ret) { + dev_err(core->dev, "%s: Failed to allocate iris IRQ\n", __func__); + return ret; + } + disable_irq_nosync(core->irq); + + return ret; +} + static int iris_register_video_device(struct iris_core *core) { struct video_device *vdev; @@ -86,6 +102,10 @@ static int iris_probe(struct platform_device *pdev) if (!core->packet) return -ENOMEM; + core->response_packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL); + if (!core->response_packet) + return -ENOMEM; + INIT_LIST_HEAD(&core->instances); core->reg_base = devm_platform_ioremap_resource(pdev, 0); @@ -96,6 +116,13 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + ret = init_iris_isr(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: Failed to init isr with %d\n", __func__, ret); + return ret; + } + ret = init_platform(core); if (ret) { dev_err_probe(core->dev, ret, diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h index 790496a..6512039 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h @@ -22,6 +22,8 @@ struct compat_handle { struct vpu_ops { int (*boot_firmware)(struct iris_core *core); int (*raise_interrupt)(struct iris_core *core); + int (*clear_interrupt)(struct iris_core *core); + int (*watchdog)(struct iris_core *core, u32 intr_status); }; int init_vpu(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c index 95bf223..a34d0ed 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c @@ -17,6 +17,8 @@ #define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24) #define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28) +#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C) + /* HFI_CTRL_INIT */ #define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48) @@ -57,10 +59,19 @@ #define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe #define CTRL_ERROR_STATUS__M_IRIS3 \ CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 +#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 \ + CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 + +#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000 +#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C) +#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 +#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4 #define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150) #define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0 +#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 + static int setup_ucregion_memory_map_iris3(struct iris_core *core) { int ret; @@ -153,9 +164,42 @@ static int raise_interrupt_iris3(struct iris_core *core) return write_register(core, CPU_IC_SOFTINT_IRIS3, 1 << CPU_IC_SOFTINT_H2A_SHFT_IRIS3); } +static int clear_interrupt_iris3(struct iris_core *core) +{ + u32 intr_status = 0, mask = 0; + int ret; + + ret = read_register(core, WRAPPER_INTR_STATUS_IRIS3, &intr_status); + if (ret) + return ret; + + mask = (WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 | + WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 | + CTRL_INIT_IDLE_MSG_BMSK_IRIS3); + + if (intr_status & mask) + core->intr_status |= intr_status; + + ret = write_register(core, CPU_CS_A2HSOFTINTCLR_IRIS3, 1); + + return ret; +} + +static int watchdog_iris3(struct iris_core *core, u32 intr_status) +{ + if (intr_status & WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3) { + dev_err(core->dev, "%s: received watchdog interrupt\n", __func__); + return -ETIME; + } + + return 0; +} + static const struct vpu_ops iris3_ops = { .boot_firmware = boot_firmware_iris3, .raise_interrupt = raise_interrupt_iris3, + .clear_interrupt = clear_interrupt_iris3, + .watchdog = watchdog_iris3, }; int init_iris3(struct iris_core *core) From patchwork Mon Dec 18 11:32:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755544 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6ADBF3A292; Mon, 18 Dec 2023 11:40:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="XdPExALJ" Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsbmO017739; Mon, 18 Dec 2023 11:40:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=bQmJDd4OHQmCq8/fSTL5KeqyMd4FZGSIiAMOA3pckZc=; b=Xd PExALJaihoLtpzxTg4ezkPIAnyIwDFq+QJaJrXMi/cfpPNyzrorubj04xj8CPzcJ nYvQm+AwgJHa2OIbpCHBeuM7hmE9Hg7vDzlZEF32VJiuGYmXsqIT6Pi43gTdXifM BUfB4lfEpqIDJmvwRSgI2HiiRqWY6IVCpe7Lq7otlxcyxX/5m3DL4z/9C+fwuEC4 i2SKwsg8J96/5mypd84yOigC42YMb5w2VGTQ39skeDw9LW+pbMiP2ZyOjZuC0oK7 aPLkLhSo65yoa5HwXrM2t4+5UUgNKp8Nq6wgKrbqHBWVoJ3KJxMnECHQhHM4r+CZ TcQaqeDHO4PpmkrImKUg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v156dm7qh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:05 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0FS004875; Mon, 18 Dec 2023 11:40:02 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ac-5; Mon, 18 Dec 2023 11:40:02 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX77o029983; Mon, 18 Dec 2023 11:36:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8AN030059; Mon, 18 Dec 2023 11:36:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 471CE2307; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities Date: Mon, 18 Dec 2023 17:02:11 +0530 Message-Id: <1702899149-21321-17-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: l4slAT7wbzXowxPp0aK2oLY-Z9ETT4lx X-Proofpoint-GUID: l4slAT7wbzXowxPp0aK2oLY-Z9ETT4lx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 lowpriorityscore=0 bulkscore=0 impostorscore=0 mlxscore=0 spamscore=0 priorityscore=1501 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement iris v4l2_ctrl_ops - s_ctrl and g_volatile_ctrl. Core capability structure has all the supported capabilities for all supported codecs. Initializes instance capability from core capability based on the codec type. Add following to each capability: Children: define dependencies for a specific capability. Adjust: define function to adjust the value of capability based on client configuration or dependency with other capability. Set: define function to set the adjusted value to firmware. Prepare dependency graph for all inter-dependent capabilities. This is used to adjust the value of capabilities and set the same to firmware. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 1 + .../media/platform/qcom/vcodec/iris/iris_core.h | 4 + .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 679 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 30 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 31 +- .../media/platform/qcom/vcodec/iris/iris_helpers.h | 13 +- drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 29 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 26 + .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 3 + .../platform/qcom/vcodec/iris/iris_instance.h | 9 + .../media/platform/qcom/vcodec/iris/iris_probe.c | 8 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 16 + .../platform/qcom/vcodec/iris/platform_common.h | 13 + .../platform/qcom/vcodec/iris/platform_sm8550.c | 84 ++- 15 files changed, 921 insertions(+), 28 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 85fdf63..7fdee5b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -6,6 +6,7 @@ iris-objs += iris_probe.o \ iris_vidc.o \ iris_vdec.o \ iris_state.o \ + iris_ctrls.o \ iris_helpers.o \ iris_hfi.o \ iris_hfi_response.o \ diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index c125bce..30f7ad7 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -50,8 +50,10 @@ * @header_id: id of packet header * @packet_id: id of packet * @vpu_ops: a pointer to vpu ops + * @dec_codecs_count: supported codec count for decoder * @platform_data: a structure for platform data * @cap: an array for supported core capabilities + * @inst_caps: a pointer to supported instance capabilities * @instances: a list_head of all instances * @intr_status: interrupt status */ @@ -89,8 +91,10 @@ struct iris_core { u32 header_id; u32 packet_id; const struct vpu_ops *vpu_ops; + u32 dec_codecs_count; struct platform_data *platform_data; struct plat_core_cap cap[CORE_CAP_MAX + 1]; + struct plat_inst_caps *inst_caps; struct list_head instances; u32 intr_status; }; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c new file mode 100644 index 0000000..6b7aeaa --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -0,0 +1,679 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#include +#include +#include + +#include "platform_common.h" +#include "iris_helpers.h" +#include "iris_hfi_packet.h" +#include "iris_hfi.h" +#include "iris_ctrls.h" + +#define MIN_CAPTURE_BUFFERS 4 +#define MIN_OUTPUT_BUFFERS 4 + +static inline bool are_all_childrens_visited(struct plat_inst_cap *cap, + bool lookup[INST_CAP_MAX]) +{ + bool found = true; + int i; + + for (i = 0; i < MAX_NUM_CHILD; i++) { + if (cap->children[i] == INST_CAP_NONE) + continue; + + if (!lookup[cap->children[i]]) { + found = false; + break; + } + } + + return found; +} + +static bool is_valid_cap_id(enum plat_inst_cap_type cap_id) +{ + return cap_id > INST_CAP_NONE && cap_id < INST_CAP_MAX; +} + +static enum plat_inst_cap_type get_cap_id(struct iris_inst *inst, u32 id) +{ + enum plat_inst_cap_type iter = INST_CAP_NONE; + enum plat_inst_cap_type cap_id = INST_CAP_NONE; + + do { + if (inst->cap[iter].v4l2_id == id) { + cap_id = inst->cap[iter].cap_id; + break; + } + iter++; + } while (iter < INST_CAP_MAX); + + return cap_id; +} + +static int add_node_list(struct list_head *list, enum plat_inst_cap_type cap_id) +{ + struct cap_entry *entry = NULL; + + entry = kzalloc(sizeof(*entry), GFP_KERNEL); + if (!entry) + return -ENOMEM; + + INIT_LIST_HEAD(&entry->list); + entry->cap_id = cap_id; + list_add(&entry->list, list); + + return 0; +} + +static int add_children(struct list_head *list, + struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + struct plat_inst_cap *cap; + int i, ret = 0; + + cap = &inst->cap[cap_id]; + + for (i = 0; i < MAX_NUM_CHILD; i++) { + if (!cap->children[i]) + break; + + if (!is_valid_cap_id(cap->children[i])) + continue; + + ret = add_node_list(list, cap->children[i]); + if (ret) + return ret; + } + + return ret; +} + +static int adjust_cap(struct iris_inst *inst, + enum plat_inst_cap_type cap_id, + struct v4l2_ctrl *ctrl) +{ + struct plat_inst_cap *cap; + + cap = &inst->cap[cap_id]; + if (!inst->cap[cap_id].cap_id) + return 0; + + if (!cap->adjust) { + if (ctrl) + inst->cap[cap_id].value = ctrl->val; + return 0; + } + + return cap->adjust(inst, ctrl); +} + +static int adjust_dynamic_property(struct iris_inst *inst, + enum plat_inst_cap_type cap_id, + struct v4l2_ctrl *ctrl, + struct list_head *firmware_list, + struct list_head *children_list, + bool cap_present[INST_CAP_MAX]) +{ + struct cap_entry *entry = NULL, *temp = NULL; + struct plat_inst_cap *cap; + s32 prev_value; + int ret; + + cap = &inst->cap[0]; + + if (!(cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED)) + return -EBUSY; + + prev_value = cap[cap_id].value; + ret = adjust_cap(inst, cap_id, ctrl); + if (ret) + return ret; + + ret = add_node_list(firmware_list, cap_id); + if (ret) + return ret; + cap_present[cap->cap_id] = true; + + if (cap[cap_id].value == prev_value) + return 0; + + ret = add_children(children_list, inst, cap_id); + if (ret) + return ret; + + list_for_each_entry_safe(entry, temp, children_list, list) { + if (!cap[entry->cap_id].adjust) { + list_del_init(&entry->list); + kfree(entry); + continue; + } + + prev_value = cap[entry->cap_id].value; + ret = adjust_cap(inst, entry->cap_id, NULL); + if (ret) + return ret; + + if (cap[entry->cap_id].value != prev_value) { + if (!cap_present[cap->cap_id]) { + ret = add_node_list(firmware_list, cap_id); + if (ret) + return ret; + cap_present[cap->cap_id] = true; + } + + ret = add_children(children_list, inst, entry->cap_id); + if (ret) + return ret; + } + + list_del_init(&entry->list); + kfree(entry); + } + + if (!list_empty(children_list)) + return -EINVAL; + + return ret; +} + +static int set_dynamic_property(struct iris_inst *inst, + struct list_head *firmware_list) +{ + struct cap_entry *entry = NULL, *temp = NULL; + struct plat_inst_cap *cap; + int ret = 0; + + list_for_each_entry_safe(entry, temp, firmware_list, list) { + cap = &inst->cap[entry->cap_id]; + if (cap->set) { + ret = cap->set(inst, entry->cap_id); + if (ret) + return -EINVAL; + } + list_del_init(&entry->list); + kfree(entry); + } + + return ret; +} + +static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl) +{ + enum plat_inst_cap_type cap_id; + struct iris_inst *inst = NULL; + int ret = 0; + + switch (ctrl->id) { + case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE: + ctrl->val = MIN_CAPTURE_BUFFERS; + break; + case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT: + ctrl->val = MIN_OUTPUT_BUFFERS; + break; + default: + inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler); + cap_id = get_cap_id(inst, ctrl->id); + if (is_valid_cap_id(cap_id)) + ctrl->val = inst->cap[cap_id].value; + else + ret = -EINVAL; + } + + return ret; +} + +static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct cap_entry *entry = NULL, *temp = NULL; + struct list_head children_list, firmware_list; + enum plat_inst_cap_type cap_id; + bool cap_present[INST_CAP_MAX]; + struct plat_inst_cap *cap; + struct iris_inst *inst; + int ret = 0; + + inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler); + cap = &inst->cap[0]; + + INIT_LIST_HEAD(&firmware_list); + INIT_LIST_HEAD(&children_list); + memset(&cap_present, 0, sizeof(cap_present)); + + cap_id = get_cap_id(inst, ctrl->id); + if (!is_valid_cap_id(cap_id)) + return -EINVAL; + + cap[cap_id].flags |= CAP_FLAG_CLIENT_SET; + + if (!inst->vb2q_src->streaming) { + inst->cap[cap_id].value = ctrl->val; + } else { + ret = adjust_dynamic_property(inst, cap_id, ctrl, + &firmware_list, &children_list, + cap_present); + if (ret) + goto free_list; + + ret = set_dynamic_property(inst, &firmware_list); + } + +free_list: + list_for_each_entry_safe(entry, temp, &children_list, list) { + list_del_init(&entry->list); + kfree(entry); + } + + list_for_each_entry_safe(entry, temp, &firmware_list, list) { + list_del_init(&entry->list); + kfree(entry); + } + + return ret; +} + +static const struct v4l2_ctrl_ops ctrl_ops = { + .s_ctrl = vdec_op_s_ctrl, + .g_volatile_ctrl = vdec_op_g_volatile_ctrl, +}; + +int ctrls_init(struct iris_inst *inst) +{ + int num_ctrls = 0, ctrl_idx = 0; + struct plat_inst_cap *cap; + struct iris_core *core; + int idx = 0; + int ret = 0; + + core = inst->core; + cap = &inst->cap[0]; + + for (idx = 0; idx < INST_CAP_MAX; idx++) { + if (cap[idx].v4l2_id) + num_ctrls++; + } + if (!num_ctrls) + return -EINVAL; + + ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, + INST_CAP_MAX * core->dec_codecs_count); + if (ret) + return ret; + + for (idx = 0; idx < INST_CAP_MAX; idx++) { + struct v4l2_ctrl *ctrl; + + if (!cap[idx].v4l2_id) + continue; + + if (ctrl_idx >= num_ctrls) { + ret = -EINVAL; + goto error; + } + + if (cap[idx].flags & CAP_FLAG_MENU) { + ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler, + &ctrl_ops, + cap[idx].v4l2_id, + cap[idx].max, + ~(cap[idx].step_or_mask), + cap[idx].value); + } else { + ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler, + &ctrl_ops, + cap[idx].v4l2_id, + cap[idx].min, + cap[idx].max, + cap[idx].step_or_mask, + cap[idx].value); + } + if (!ctrl) { + ret = -EINVAL; + goto error; + } + + ret = inst->ctrl_handler.error; + if (ret) + goto error; + + if ((cap[idx].flags & CAP_FLAG_VOLATILE) || + (ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_CAPTURE || + ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_OUTPUT)) + ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE; + + ctrl->flags |= V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; + ctrl_idx++; + } + inst->num_ctrls = num_ctrls; + + return 0; +error: + v4l2_ctrl_handler_free(&inst->ctrl_handler); + + return ret; +} + +int iris_init_core_caps(struct iris_core *core) +{ + struct plat_core_cap *core_platform_data; + int i, num_core_caps; + + core_platform_data = core->platform_data->core_data; + if (!core_platform_data) + return -EINVAL; + + num_core_caps = core->platform_data->core_data_size; + + for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) { + core->cap[core_platform_data[i].type].type = core_platform_data[i].type; + core->cap[core_platform_data[i].type].value = core_platform_data[i].value; + } + + return 0; +} + +static int update_inst_capability(struct plat_inst_cap *in, + struct plat_inst_caps *capability) +{ + if (!in || !capability) + return -EINVAL; + + if (in->cap_id >= INST_CAP_MAX) + return -EINVAL; + + capability->cap[in->cap_id].cap_id = in->cap_id; + capability->cap[in->cap_id].min = in->min; + capability->cap[in->cap_id].max = in->max; + capability->cap[in->cap_id].step_or_mask = in->step_or_mask; + capability->cap[in->cap_id].value = in->value; + capability->cap[in->cap_id].flags = in->flags; + capability->cap[in->cap_id].v4l2_id = in->v4l2_id; + capability->cap[in->cap_id].hfi_id = in->hfi_id; + memcpy(capability->cap[in->cap_id].children, in->children, + sizeof(capability->cap[in->cap_id].children)); + capability->cap[in->cap_id].adjust = in->adjust; + capability->cap[in->cap_id].set = in->set; + + return 0; +} + +int iris_init_instance_caps(struct iris_core *core) +{ + struct plat_inst_cap *inst_plat_cap_data = NULL; + u8 dec_codecs_count = 0; + int num_inst_cap; + u32 dec_valid_codecs; + int i, j, check_bit = 0; + int ret = 0; + + inst_plat_cap_data = core->platform_data->inst_cap_data; + if (!inst_plat_cap_data) + return -EINVAL; + + dec_valid_codecs = core->cap[DEC_CODECS].value; + dec_codecs_count = hweight32(dec_valid_codecs); + core->dec_codecs_count = dec_codecs_count; + + core->inst_caps = devm_kzalloc(core->dev, + dec_codecs_count * sizeof(struct plat_inst_caps), + GFP_KERNEL); + if (!core->inst_caps) + return -ENOMEM; + + for (i = 0; i < dec_codecs_count; i++) { + while (check_bit < (sizeof(dec_valid_codecs) * 8)) { + if (dec_valid_codecs & BIT(check_bit)) { + core->inst_caps[i].codec = dec_valid_codecs & + BIT(check_bit); + check_bit++; + break; + } + check_bit++; + } + } + + num_inst_cap = core->platform_data->inst_cap_data_size; + + for (i = 0; i < num_inst_cap; i++) { + for (j = 0; j < dec_codecs_count; j++) { + if ((inst_plat_cap_data[i].codec & + core->inst_caps[j].codec)) { + ret = update_inst_capability(&inst_plat_cap_data[i], + &core->inst_caps[j]); + if (ret) + return ret; + } + } + } + + return ret; +} + +int get_inst_capability(struct iris_inst *inst) +{ + struct iris_core *core; + int i; + + core = inst->core; + + for (i = 0; i < core->dec_codecs_count; i++) { + if (core->inst_caps[i].codec == inst->codec) { + memcpy(&inst->cap[0], &core->inst_caps[i].cap[0], + (INST_CAP_MAX + 1) * sizeof(struct plat_inst_cap)); + } + } + + return 0; +} + +int prepare_dependency_list(struct iris_inst *inst) +{ + struct cap_entry *entry = NULL, *temp = NULL; + struct plat_inst_cap *cap, *temp_cap; + int caps_to_prepare, pending_list_counter, + pending_at_start = 0; + struct list_head prepared_list, pending_list; + bool is_prepared[INST_CAP_MAX]; + bool is_pending[INST_CAP_MAX]; + int i, ret = 0; + + cap = &inst->cap[0]; + + if (!list_empty(&inst->caps_list)) + return 0; + + INIT_LIST_HEAD(&prepared_list); + INIT_LIST_HEAD(&pending_list); + memset(&is_prepared, 0, sizeof(is_prepared)); + memset(&is_pending, 0, sizeof(is_pending)); + + for (i = 1; i < INST_CAP_MAX; i++) { + temp_cap = &cap[i]; + if (!is_valid_cap_id(temp_cap->cap_id)) + continue; + + if (!temp_cap->children[0]) { + if (!is_prepared[temp_cap->cap_id]) { + ret = add_node_list(&prepared_list, temp_cap->cap_id); + if (ret) + goto free_list; + is_prepared[temp_cap->cap_id] = true; + } + } else { + if (!is_pending[temp_cap->cap_id]) { + ret = add_node_list(&pending_list, temp_cap->cap_id); + if (ret) + goto free_list; + is_pending[temp_cap->cap_id] = true; + } + } + } + + list_for_each_entry(entry, &pending_list, list) + pending_at_start++; + + caps_to_prepare = pending_at_start; + pending_list_counter = pending_at_start; + + list_for_each_entry_safe(entry, temp, &pending_list, list) { + list_del_init(&entry->list); + is_pending[entry->cap_id] = false; + pending_list_counter--; + temp_cap = &cap[entry->cap_id]; + + if (are_all_childrens_visited(temp_cap, is_prepared)) { + list_add(&entry->list, &prepared_list); + is_prepared[entry->cap_id] = true; + caps_to_prepare--; + } else { + list_add_tail(&entry->list, &pending_list); + is_pending[entry->cap_id] = true; + } + + if (!pending_list_counter) { + if (pending_at_start == caps_to_prepare) { + ret = -EINVAL; + goto free_list; + } + pending_at_start = caps_to_prepare; + pending_list_counter = caps_to_prepare; + } + } + + if (!list_empty(&pending_list)) { + ret = -EINVAL; + goto free_list; + } + + list_replace_init(&prepared_list, &inst->caps_list); + +free_list: + list_for_each_entry_safe(entry, temp, &pending_list, list) { + list_del_init(&entry->list); + kfree(entry); + } + list_for_each_entry_safe(entry, temp, &prepared_list, list) { + list_del_init(&entry->list); + kfree(entry); + } + + return ret; +} + +int set_u32_enum(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = inst->cap[cap_id].value; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_value, sizeof(u32)); +} + +int set_u32(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = inst->cap[cap_id].value; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + +int set_stage(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + struct v4l2_format *inp_f; + u32 work_mode = STAGE_2; + u32 width, height; + u32 hfi_id; + + hfi_id = inst->cap[cap_id].hfi_id; + + inp_f = inst->fmt_src; + height = inp_f->fmt.pix_mp.height; + width = inp_f->fmt.pix_mp.width; + if (res_is_less_than(width, height, 1280, 720)) + work_mode = STAGE_1; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &work_mode, sizeof(u32)); +} + +int set_pipe(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + u32 work_route, hfi_id; + + work_route = inst->cap[PIPE].value; + hfi_id = inst->cap[cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &work_route, sizeof(u32)); +} + +int adjust_v4l2_properties(struct iris_inst *inst) +{ + struct cap_entry *entry = NULL, *temp = NULL; + int ret = 0; + + list_for_each_entry_safe(entry, temp, &inst->caps_list, list) { + ret = adjust_cap(inst, entry->cap_id, NULL); + if (ret) + return ret; + } + + return ret; +} + +int adjust_output_order(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 display_delay = -1, display_delay_enable = -1; + u32 adjusted_value; + + adjusted_value = ctrl ? ctrl->val : + inst->cap[OUTPUT_ORDER].value; + + display_delay = inst->cap[DISPLAY_DELAY].value; + display_delay_enable = inst->cap[DISPLAY_DELAY_ENABLE].value; + + if (display_delay_enable && !display_delay) + adjusted_value = 1; + + inst->cap[OUTPUT_ORDER].value = adjusted_value; + + return 0; +} + +int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + u32 adjusted_value; + s32 pix_fmt = -1; + + adjusted_value = ctrl ? ctrl->val : + inst->cap[PROFILE].value; + + pix_fmt = inst->cap[PIX_FMTS].value; + + if (pix_fmt == FMT_TP10C) + adjusted_value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10; + else + adjusted_value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN; + + inst->cap[PROFILE].value = adjusted_value; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h new file mode 100644 index 0000000..0f67f4f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_CTRLS_H_ +#define _IRIS_CTRLS_H_ + +#include "iris_instance.h" + +struct cap_entry { + struct list_head list; + enum plat_inst_cap_type cap_id; +}; + +int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_pipe(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_u32(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int adjust_output_order(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int vidc_ctrl_handler_deinit(struct iris_inst *inst); +int prepare_dependency_list(struct iris_inst *inst); +int iris_init_instance_caps(struct iris_core *core); +int iris_init_core_caps(struct iris_core *core); +int get_inst_capability(struct iris_inst *inst); +int adjust_v4l2_properties(struct iris_inst *inst); +int ctrls_init(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 872268d..8d8bc3a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -5,6 +5,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" +#include "iris_hfi_packet.h" #include "iris_instance.h" int check_core_lock(struct iris_core *core) @@ -16,23 +17,29 @@ int check_core_lock(struct iris_core *core) return fatal ? -EINVAL : 0; } -int iris_init_core_caps(struct iris_core *core) +bool res_is_less_than(u32 width, u32 height, + u32 ref_width, u32 ref_height) { - struct plat_core_cap *core_platform_data; - int i, num_core_caps; + u32 num_mbs = NUM_MBS_PER_FRAME(height, width); + u32 max_side = max(ref_width, ref_height); - core_platform_data = core->platform_data->core_data; - if (!core_platform_data) - return -EINVAL; + if (num_mbs < NUM_MBS_PER_FRAME(ref_height, ref_width) && + width < max_side && + height < max_side) + return true; - num_core_caps = core->platform_data->core_data_size; + return false; +} - for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) { - core->cap[core_platform_data[i].type].type = core_platform_data[i].type; - core->cap[core_platform_data[i].type].value = core_platform_data[i].value; - } +u32 get_port_info(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT) + return HFI_PORT_BITSTREAM; + else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) + return HFI_PORT_RAW; - return 0; + return HFI_PORT_NONE; } static int process_inst_timeout(struct iris_inst *inst) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index b9a6485..60c79124 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -6,11 +6,22 @@ #ifndef _IRIS_HELPERS_H_ #define _IRIS_HELPERS_H_ +#include +#include + #include "iris_core.h" #include "iris_instance.h" +#include "platform_common.h" + +#define NUM_MBS_PER_FRAME(__height, __width) \ + ((ALIGN(__height, 16) / 16) * (ALIGN(__width, 16) / 16)) int check_core_lock(struct iris_core *core); -int iris_init_core_caps(struct iris_core *core); +bool res_is_less_than(u32 width, u32 height, + u32 ref_width, u32 ref_height); +u32 get_port_info(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); + int close_session(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 4b8d8c7..24ddb98 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -10,6 +10,7 @@ #include "iris_hfi.h" #include "iris_hfi_packet.h" #include "iris_hfi_response.h" +#include "iris_helpers.h" #include "vpu_common.h" static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) @@ -280,3 +281,31 @@ irqreturn_t iris_hfi_isr_handler(int irq, void *data) return IRQ_HANDLED; } + +int iris_hfi_set_property(struct iris_inst *inst, + u32 packet_type, u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + struct iris_core *core; + int ret; + + core = inst->core; + mutex_lock(&core->lock); + + ret = hfi_packet_session_property(inst, + packet_type, + flag, + plane, + payload_type, + payload, + payload_size); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index 8a62986..bf991bb 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -13,6 +13,9 @@ int iris_hfi_core_init(struct iris_core *core); int iris_hfi_core_deinit(struct iris_core *core); int iris_hfi_session_open(struct iris_inst *inst); int iris_hfi_session_close(struct iris_inst *inst); +int iris_hfi_set_property(struct iris_inst *inst, + u32 packet_type, u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size); irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index 1ed572e..749d978 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -239,3 +239,29 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, return ret; } + +int hfi_packet_session_property(struct iris_inst *inst, + u32 pkt_type, u32 flags, u32 port, + u32 payload_type, void *payload, u32 payload_size) +{ + struct iris_core *core; + int ret; + + core = inst->core; + + ret = hfi_create_header(inst->packet, inst->packet_size, + inst->session_id, core->header_id++); + if (ret) + return ret; + + ret = hfi_create_packet(inst->packet, inst->packet_size, + pkt_type, + flags, + payload_type, + port, + core->packet_id++, + payload, + payload_size); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index 9e476e9..bea7ed9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -74,5 +74,8 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, u32 flags, u32 port, u32 session_id, u32 payload_type, void *payload, u32 payload_size); +int hfi_packet_session_property(struct iris_inst *inst, + u32 pkt_type, u32 flags, u32 port, + u32 payload_type, void *payload, u32 payload_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 1bbb533..f6a3066 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -10,6 +10,7 @@ #include "iris_core.h" #include "iris_common.h" +#include "platform_common.h" /** * struct iris_inst - holds per video instance parameters @@ -27,6 +28,10 @@ * @packet: HFI packet * @packet_size: HFI packet size * @completions: structure of signal completions + * @cap: array of supported instance capabilities + * @num_ctrls: supported number of controls + * @caps_list: list head of capability + * @codec: codec type */ struct iris_inst { @@ -43,6 +48,10 @@ struct iris_inst { void *packet; u32 packet_size; struct completion completions[MAX_SIGNAL]; + struct plat_inst_cap cap[INST_CAP_MAX + 1]; + u32 num_ctrls; + struct list_head caps_list; + enum codec_type codec; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 8c591da..50fb93e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -13,6 +13,7 @@ #include "iris_hfi.h" #include "resources.h" #include "iris_vidc.h" +#include "iris_ctrls.h" static int init_iris_isr(struct iris_core *core) { @@ -158,6 +159,13 @@ static int iris_probe(struct platform_device *pdev) return ret; } + ret = iris_init_instance_caps(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init inst caps failed with %d\n", __func__, ret); + return ret; + } + ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index d02da8b..3a26edb 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -9,6 +9,7 @@ #include "iris_instance.h" #include "iris_vdec.h" #include "iris_vidc.h" +#include "iris_ctrls.h" static int vidc_v4l2_fh_init(struct iris_inst *inst) { @@ -174,6 +175,10 @@ int vidc_open(struct file *filp) if (ret) goto fail_free_inst; + INIT_LIST_HEAD(&inst->caps_list); + for (i = 0; i < MAX_SIGNAL; i++) + init_completion(&inst->completions[i]); + ret = vidc_v4l2_fh_init(inst); if (ret) goto fail_remove_session; @@ -184,6 +189,14 @@ int vidc_open(struct file *filp) if (ret) goto fail_inst_deinit; + ret = get_inst_capability(inst); + if (ret) + goto fail_queue_deinit; + + ret = ctrls_init(inst); + if (ret) + goto fail_queue_deinit; + ret = iris_hfi_session_open(inst); if (ret) { dev_err(core->dev, "%s: session open failed\n", __func__); @@ -194,7 +207,9 @@ int vidc_open(struct file *filp) return 0; fail_core_deinit: + v4l2_ctrl_handler_free(&inst->ctrl_handler); iris_core_deinit(core); +fail_queue_deinit: vidc_vb2_queue_deinit(inst); fail_inst_deinit: vdec_inst_deinit(inst); @@ -216,6 +231,7 @@ int vidc_close(struct file *filp) if (!inst) return -EINVAL; + v4l2_ctrl_handler_free(&inst->ctrl_handler); vdec_inst_deinit(inst); close_session(inst); vidc_vb2_queue_deinit(inst); diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index e478b02..8305c65 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -7,8 +7,10 @@ #define _PLATFORM_COMMON_H_ #include +#include struct iris_core; +struct iris_inst; #define HW_RESPONSE_TIMEOUT_VALUE (1000) @@ -17,6 +19,7 @@ struct iris_core; #define CODED_FRAMES_PROGRESSIVE 0x0 #define CODED_FRAMES_INTERLACE 0x1 +#define MAX_NUM_CHILD 10 #define DEFAULT_MAX_HOST_BUF_COUNT 64 #define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256 @@ -163,6 +166,16 @@ struct plat_inst_cap { u32 v4l2_id; u32 hfi_id; enum plat_inst_cap_flags flags; + enum plat_inst_cap_type children[MAX_NUM_CHILD]; + int (*adjust)(struct iris_inst *inst, + struct v4l2_ctrl *ctrl); + int (*set)(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); +}; + +struct plat_inst_caps { + enum codec_type codec; + struct plat_inst_cap cap[INST_CAP_MAX + 1]; }; struct platform_data { diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index c75017e..0759ac5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -9,6 +9,7 @@ #include #include "hfi_defines.h" +#include "iris_ctrls.h" #include "platform_common.h" #include "resources.h" @@ -47,7 +48,16 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { FMT_NV12 | FMT_NV21 | FMT_NV12C, FMT_NV12C}, - {PIX_FMTS, HEVC | VP9, + {PIX_FMTS, HEVC, + FMT_NV12, + FMT_TP10C, + FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, + FMT_NV12C, + 0, 0, + CAP_FLAG_NONE, + {PROFILE}}, + + {PIX_FMTS, VP9, FMT_NV12, FMT_TP10C, FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, @@ -88,7 +98,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, V4L2_CID_MPEG_VIDEO_H264_PROFILE, HFI_PROP_PROFILE, - CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_u32_enum}, {PROFILE, HEVC, V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, @@ -99,7 +112,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, V4L2_CID_MPEG_VIDEO_HEVC_PROFILE, HFI_PROP_PROFILE, - CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + adjust_profile, + set_u32_enum}, {PROFILE, VP9, V4L2_MPEG_VIDEO_VP9_PROFILE_0, @@ -109,7 +125,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { V4L2_MPEG_VIDEO_VP9_PROFILE_0, V4L2_CID_MPEG_VIDEO_VP9_PROFILE, HFI_PROP_PROFILE, - CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_u32_enum}, {LEVEL, H264, V4L2_MPEG_VIDEO_H264_LEVEL_1_0, @@ -137,7 +156,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { V4L2_MPEG_VIDEO_H264_LEVEL_6_1, V4L2_CID_MPEG_VIDEO_H264_LEVEL, HFI_PROP_LEVEL, - CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_u32_enum}, {LEVEL, HEVC, V4L2_MPEG_VIDEO_HEVC_LEVEL_1, @@ -158,7 +180,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1, V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, HFI_PROP_LEVEL, - CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_u32_enum}, {LEVEL, VP9, V4L2_MPEG_VIDEO_VP9_LEVEL_1_0, @@ -178,7 +203,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { V4L2_MPEG_VIDEO_VP9_LEVEL_6_0, V4L2_CID_MPEG_VIDEO_VP9_LEVEL, HFI_PROP_LEVEL, - CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_u32_enum}, {HEVC_TIER, HEVC, V4L2_MPEG_VIDEO_HEVC_TIER_MAIN, @@ -188,46 +216,69 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, V4L2_CID_MPEG_VIDEO_HEVC_TIER, HFI_PROP_TIER, - CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_u32_enum}, {DISPLAY_DELAY_ENABLE, CODECS_ALL, 0, 1, 1, 0, V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE, HFI_PROP_DECODE_ORDER_OUTPUT, - CAP_FLAG_INPUT_PORT}, + CAP_FLAG_INPUT_PORT, + {OUTPUT_ORDER}, + NULL, + NULL}, {DISPLAY_DELAY, CODECS_ALL, 0, 1, 1, 0, V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY, HFI_PROP_DECODE_ORDER_OUTPUT, - CAP_FLAG_INPUT_PORT}, + CAP_FLAG_INPUT_PORT, + {OUTPUT_ORDER}, + NULL, + NULL}, {OUTPUT_ORDER, CODECS_ALL, 0, 1, 1, 0, 0, HFI_PROP_DECODE_ORDER_OUTPUT, - CAP_FLAG_INPUT_PORT}, + CAP_FLAG_INPUT_PORT, + {0}, + adjust_output_order, + set_u32}, {INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL, DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT, 1, DEFAULT_MAX_HOST_BUF_COUNT, 0, HFI_PROP_BUFFER_HOST_MAX_COUNT, - CAP_FLAG_INPUT_PORT}, + CAP_FLAG_INPUT_PORT, + {0}, + NULL, + set_u32}, {STAGE, CODECS_ALL, STAGE_1, STAGE_2, 1, STAGE_2, 0, - HFI_PROP_STAGE}, + HFI_PROP_STAGE, + CAP_FLAG_NONE, + {0}, + NULL, + set_stage}, {PIPE, CODECS_ALL, PIPE_1, PIPE_4, 1, PIPE_4, 0, - HFI_PROP_PIPE}, + HFI_PROP_PIPE, + CAP_FLAG_NONE, + {0}, + NULL, + set_pipe}, {POC, H264, 0, 2, 1, 1, 0, @@ -252,7 +303,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { 0, 1, 1, 1, 0, HFI_PROP_DEC_START_FROM_RAP_FRAME, - CAP_FLAG_INPUT_PORT}, + CAP_FLAG_INPUT_PORT, + {0}, + NULL, + set_u32}, }; static const struct bus_info sm8550_bus_table[] = { From patchwork Mon Dec 18 11:32:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755550 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10CE91B26E; Mon, 18 Dec 2023 11:36:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="QZrGMNdu" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBJih7018958; Mon, 18 Dec 2023 11:36:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Ii26TUpLDcoDUZzoj7f1vyuG0tLLz7zFj9QofAAkMJc=; b=QZ rGMNdu31ybxjmQvSAmfYuBIR32xUGJN3EkhF4PeV+9cBGrOSAlplCfxg5g1WMim4 3tubvgSWRvaDsq19MlX+ucIlyF5r8oZe2D1KiKlqaNupN9CvdpX8hU53kqc3yR+m YILDLSvYRQ+EQqENs4CMbMLYyl13XQz+dg4EISp10+fkCSxZTzxpu8Aqd6AgHupf W8rISJ7d2es94YGHJYRQAyeBUP4X1caD5JeWyOhg+Ov9xW0Rz42VJ1TYRRV/kL0o dP1GZ/MYGWBMFseqzaPYMHya1RsOCD4dU+1EQgRtzr3ARFNvGuijYmb5+T95H538 l4KP/c+tyoctDXyoWjFw== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4my-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:12 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmVB028246; Mon, 18 Dec 2023 11:36:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyjr-1; Mon, 18 Dec 2023 11:33:09 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Aq029911; Mon, 18 Dec 2023 11:33:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6q7029965; Mon, 18 Dec 2023 11:33:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 49C562308; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 17/34] media: iris: implement vb2_ops queue setup Date: Mon, 18 Dec 2023 17:02:12 +0530 Message-Id: <1702899149-21321-18-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: cSe5nSbz30KhCIGHqG7yaTukO9KC5Hzn X-Proofpoint-GUID: cSe5nSbz30KhCIGHqG7yaTukO9KC5Hzn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement queue_setup vb2_ops. Calculate the buffer count and buffer size as par video hardware requirement and updates to client. Also, allocate the video driver buffers for output and capture plane. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +- .../media/platform/qcom/vcodec/iris/iris_buffer.c | 179 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_buffer.h | 49 ++++++ .../media/platform/qcom/vcodec/iris/iris_common.h | 71 +++++++- .../media/platform/qcom/vcodec/iris/iris_helpers.c | 24 +++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +- .../platform/qcom/vcodec/iris/iris_instance.h | 6 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 71 ++++++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 15 ++ .../media/platform/qcom/vcodec/iris/iris_vdec.c | 5 +- .../media/platform/qcom/vcodec/iris/iris_vidc.c | 19 ++- .../platform/qcom/vcodec/iris/platform_common.h | 16 +- 12 files changed, 442 insertions(+), 21 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 7fdee5b..a94e36b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -1,9 +1,10 @@ -iris-objs += ../hfi_queue.o ../firmware.o +iris-objs += ../hfi_queue.o ../firmware.o ../buffers.o iris-objs += iris_probe.o \ iris_state.o \ iris_core.o \ iris_vidc.o \ + iris_vb2.o \ iris_vdec.o \ iris_state.o \ iris_ctrls.o \ @@ -11,6 +12,7 @@ iris-objs += iris_probe.o \ iris_hfi.o \ iris_hfi_response.o \ iris_hfi_packet.o \ + iris_buffer.o \ resources.o \ vpu_common.o \ vpu_iris3.o \ diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c new file mode 100644 index 0000000..b9cffbf --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "../buffers.h" +#include "iris_buffer.h" +#include "iris_helpers.h" +#include "iris_instance.h" + +static int input_min_count(struct iris_inst *inst) +{ + return MIN_BUFFERS; +} + +static int output_min_count(struct iris_inst *inst) +{ + int output_min_count; + + switch (inst->codec) { + case H264: + case HEVC: + output_min_count = 4; + break; + case VP9: + output_min_count = 9; + break; + default: + output_min_count = 4; + break; + } + + return output_min_count; +} + +int iris_get_buf_min_count(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return input_min_count(inst); + case BUF_OUTPUT: + return output_min_count(inst); + default: + return 0; + } +} + +static u32 input_buffer_size(struct iris_inst *inst) +{ + u32 base_res_mbs = NUM_MBS_4k; + u32 frame_size, num_mbs; + struct v4l2_format *f; + u32 div_factor = 1; + u32 codec; + + f = inst->fmt_src; + codec = f->fmt.pix_mp.pixelformat; + + num_mbs = get_mbpf(inst); + if (num_mbs > NUM_MBS_4k) { + div_factor = 4; + base_res_mbs = inst->cap[MBPF].value; + } else { + base_res_mbs = NUM_MBS_4k; + if (codec == V4L2_PIX_FMT_VP9) + div_factor = 1; + else + div_factor = 2; + } + + frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor; + + /* multiply by 10/8 (1.25) to get size for 10 bit case */ + if (codec == V4L2_PIX_FMT_VP9 || codec == V4L2_PIX_FMT_HEVC) + frame_size = frame_size + (frame_size >> 2); + + return ALIGN(frame_size, SZ_4K); +} + +static u32 output_buffer_size(struct iris_inst *inst) +{ + struct v4l2_format *f; + u32 size; + + f = inst->fmt_dst; + + size = video_raw_buffer_size(f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width, + f->fmt.pix_mp.height); + return size; +} + +int iris_get_buffer_size(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return input_buffer_size(inst); + case BUF_OUTPUT: + return output_buffer_size(inst); + default: + return 0; + } +} + +struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return &inst->buffers.input; + case BUF_OUTPUT: + return &inst->buffers.output; + case BUF_READ_ONLY: + return &inst->buffers.read_only; + case BUF_BIN: + return &inst->buffers.bin; + case BUF_ARP: + return &inst->buffers.arp; + case BUF_COMV: + return &inst->buffers.comv; + case BUF_NON_COMV: + return &inst->buffers.non_comv; + case BUF_LINE: + return &inst->buffers.line; + case BUF_DPB: + return &inst->buffers.dpb; + case BUF_PERSIST: + return &inst->buffers.persist; + case BUF_VPSS: + return &inst->buffers.vpss; + default: + return NULL; + } +} + +int iris_allocate_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type, + u32 num_buffers) +{ + struct iris_buffer *buf = NULL; + struct iris_buffers *buffers; + int idx = 0; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return -EINVAL; + + for (idx = 0; idx < num_buffers; idx++) { + buf = kzalloc(sizeof(*buf), GFP_KERNEL); + if (!buf) + return -EINVAL; + + INIT_LIST_HEAD(&buf->list); + list_add_tail(&buf->list, &buffers->list); + buf->type = buf_type; + buf->index = idx; + } + + return 0; +} + +int iris_free_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + list_del_init(&buf->list); + kfree(buf); + } + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h new file mode 100644 index 0000000..1cd76a9 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_BUFFER_H_ +#define _IRIS_BUFFER_H_ + +#define MIN_BUFFERS 4 + +#include "iris_common.h" + +struct iris_inst; + +struct iris_buffers { + struct list_head list; // list of "struct iris_buffer" + u32 min_count; + u32 actual_count; + u32 size; + bool reuse; +}; + +struct iris_buffers_info { + struct iris_buffers input; + struct iris_buffers output; + struct iris_buffers read_only; + struct iris_buffers bin; + struct iris_buffers arp; + struct iris_buffers comv; + struct iris_buffers non_comv; + struct iris_buffers line; + struct iris_buffers dpb; + struct iris_buffers persist; + struct iris_buffers vpss; +}; + +int iris_get_buf_min_count(struct iris_inst *inst, + enum iris_buffer_type buffer_type); +int iris_get_buffer_size(struct iris_inst *inst, + enum iris_buffer_type buffer_type); +struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst, + enum iris_buffer_type buffer_type); +int iris_allocate_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type, + u32 num_buffers); +int iris_free_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 3e4dd71..4edadc3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -5,6 +5,7 @@ #ifndef _IRIS_COMMON_H_ #define _IRIS_COMMON_H_ +#include #include #include #include @@ -14,7 +15,31 @@ #define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE #define DEFAULT_WIDTH 320 #define DEFAULT_HEIGHT 240 -#define DEFAULT_BUF_SIZE 115200 + +#define MB_IN_PIXEL (16 * 16) + +#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) + +enum codec_type { + H264 = BIT(0), + HEVC = BIT(1), + VP9 = BIT(2), +}; + +enum colorformat_type { + FMT_NONE = 0, + FMT_NV12C = BIT(0), + FMT_NV12 = BIT(1), + FMT_NV21 = BIT(2), + FMT_TP10C = BIT(3), +}; + +struct rect_desc { + u32 left; + u32 top; + u32 width; + u32 height; +}; enum signal_session_response { SIGNAL_CMD_STOP_INPUT = 0, @@ -23,4 +48,48 @@ enum signal_session_response { MAX_SIGNAL, }; +enum iris_buffer_type { + BUF_NONE, + BUF_INPUT, + BUF_OUTPUT, + BUF_READ_ONLY, + BUF_BIN, + BUF_ARP, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, + BUF_DPB, + BUF_PERSIST, + BUF_VPSS, +}; + +enum iris_buffer_attributes { + BUF_ATTR_DEFERRED = BIT(0), + BUF_ATTR_READ_ONLY = BIT(1), + BUF_ATTR_PENDING_RELEASE = BIT(2), + BUF_ATTR_QUEUED = BIT(3), + BUF_ATTR_DEQUEUED = BIT(4), + BUF_ATTR_BUFFER_DONE = BIT(5), +}; + +struct iris_buffer { + struct list_head list; + struct iris_inst *inst; + enum iris_buffer_type type; + u32 index; + int fd; + u32 buffer_size; + u32 data_offset; + u32 data_size; + u64 device_addr; + void *kvaddr; + unsigned long dma_attrs; + u32 flags; + u64 timestamp; + enum iris_buffer_attributes attr; + void *dmabuf; + struct sg_table *sg_table; + struct dma_buf_attachment *attach; +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 8d8bc3a..54a7851 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -42,6 +42,30 @@ u32 get_port_info(struct iris_inst *inst, return HFI_PORT_NONE; } +enum iris_buffer_type v4l2_type_to_driver(u32 type) +{ + switch (type) { + case INPUT_MPLANE: + return BUF_INPUT; + case OUTPUT_MPLANE: + return BUF_OUTPUT; + default: + return 0; + } +} + +int get_mbpf(struct iris_inst *inst) +{ + int height = 0, width = 0; + struct v4l2_format *inp_f; + + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + + return NUM_MBS_PER_FRAME(height, width); +} + static int process_inst_timeout(struct iris_inst *inst) { struct iris_inst *instance; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 60c79124..3bae969 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -11,6 +11,7 @@ #include "iris_core.h" #include "iris_instance.h" +#include "iris_buffer.h" #include "platform_common.h" #define NUM_MBS_PER_FRAME(__height, __width) \ @@ -21,7 +22,8 @@ bool res_is_less_than(u32 width, u32 height, u32 ref_width, u32 ref_height); u32 get_port_info(struct iris_inst *inst, enum plat_inst_cap_type cap_id); - +enum iris_buffer_type v4l2_type_to_driver(u32 type); +int get_mbpf(struct iris_inst *inst); int close_session(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index f6a3066..a5c6cb48 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -8,6 +8,8 @@ #include +#include "iris_buffer.h" +#include "iris_common.h" #include "iris_core.h" #include "iris_common.h" #include "platform_common.h" @@ -25,6 +27,7 @@ * @fmt_src: structure of v4l2_format for source * @fmt_dst: structure of v4l2_format for destination * @ctrl_handler: reference of v4l2 ctrl handler + * @crop: structure of crop info * @packet: HFI packet * @packet_size: HFI packet size * @completions: structure of signal completions @@ -32,6 +35,7 @@ * @num_ctrls: supported number of controls * @caps_list: list head of capability * @codec: codec type + * @buffers: structure of buffer info */ struct iris_inst { @@ -45,6 +49,7 @@ struct iris_inst { struct v4l2_format *fmt_src; struct v4l2_format *fmt_dst; struct v4l2_ctrl_handler ctrl_handler; + struct rect_desc crop; void *packet; u32 packet_size; struct completion completions[MAX_SIGNAL]; @@ -52,6 +57,7 @@ struct iris_inst { u32 num_ctrls; struct list_head caps_list; enum codec_type codec; + struct iris_buffers_info buffers; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c new file mode 100644 index 0000000..b040d27 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_buffer.h" +#include "iris_core.h" +#include "iris_helpers.h" +#include "iris_instance.h" +#include "iris_vb2.h" + +int iris_vb2_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, unsigned int *num_planes, + unsigned int sizes[], struct device *alloc_devs[]) +{ + enum iris_buffer_type buffer_type = 0; + struct iris_buffers *buffers; + struct iris_inst *inst; + struct iris_core *core; + struct v4l2_format *f; + int ret; + + if (!q || !num_buffers || !num_planes || !sizes) + return -EINVAL; + + inst = vb2_get_drv_priv(q); + if (!inst || !inst->core) + return -EINVAL; + + core = inst->core; + + if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + f = inst->fmt_src; + else + f = inst->fmt_dst; + + if (*num_planes) { + if (*num_planes != f->fmt.pix_mp.num_planes || + sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage) + return -EINVAL; + } + + buffer_type = v4l2_type_to_driver(q->type); + if (!buffer_type) + return -EINVAL; + + ret = iris_free_buffers(inst, buffer_type); + if (ret) + return ret; + + buffers = iris_get_buffer_list(inst, buffer_type); + if (!buffers) + return -EINVAL; + + buffers->min_count = iris_get_buf_min_count(inst, buffer_type); + if (*num_buffers < buffers->min_count) + *num_buffers = buffers->min_count; + buffers->actual_count = *num_buffers; + *num_planes = 1; + + buffers->size = iris_get_buffer_size(inst, buffer_type); + + f->fmt.pix_mp.plane_fmt[0].sizeimage = buffers->size; + sizes[0] = f->fmt.pix_mp.plane_fmt[0].sizeimage; + + ret = iris_allocate_buffers(inst, buffer_type, *num_buffers); + + q->dev = core->dev; + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h new file mode 100644 index 0000000..8a8e1039 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VB2_H_ +#define _IRIS_VB2_H_ + +#include + +int iris_vb2_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, unsigned int *num_planes, + unsigned int sizes[], struct device *alloc_devs[]); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 984be34..1b957a13 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -3,6 +3,7 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include "iris_buffer.h" #include "iris_common.h" #include "iris_vdec.h" @@ -22,7 +23,7 @@ void vdec_inst_init(struct iris_inst *inst) f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264; f->fmt.pix_mp.num_planes = 1; f->fmt.pix_mp.plane_fmt[0].bytesperline = 0; - f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT); f->fmt.pix_mp.field = V4L2_FIELD_NONE; f = inst->fmt_dst; @@ -32,7 +33,7 @@ void vdec_inst_init(struct iris_inst *inst) f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32); f->fmt.pix_mp.num_planes = 1; f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128); - f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); f->fmt.pix_mp.field = V4L2_FIELD_NONE; f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 3a26edb..410de720 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -10,6 +10,7 @@ #include "iris_vdec.h" #include "iris_vidc.h" #include "iris_ctrls.h" +#include "iris_vb2.h" static int vidc_v4l2_fh_init(struct iris_inst *inst) { @@ -168,13 +169,22 @@ int vidc_open(struct file *filp) inst->core = core; inst->session_id = hash32_ptr(inst); mutex_init(&inst->ctx_q_lock); - for (i = 0; i < MAX_SIGNAL; i++) - init_completion(&inst->completions[i]); ret = vidc_add_session(inst); if (ret) goto fail_free_inst; + INIT_LIST_HEAD(&inst->buffers.input.list); + INIT_LIST_HEAD(&inst->buffers.output.list); + INIT_LIST_HEAD(&inst->buffers.read_only.list); + INIT_LIST_HEAD(&inst->buffers.bin.list); + INIT_LIST_HEAD(&inst->buffers.arp.list); + INIT_LIST_HEAD(&inst->buffers.comv.list); + INIT_LIST_HEAD(&inst->buffers.non_comv.list); + INIT_LIST_HEAD(&inst->buffers.line.list); + INIT_LIST_HEAD(&inst->buffers.dpb.list); + INIT_LIST_HEAD(&inst->buffers.persist.list); + INIT_LIST_HEAD(&inst->buffers.vpss.list); INIT_LIST_HEAD(&inst->caps_list); for (i = 0; i < MAX_SIGNAL; i++) init_completion(&inst->completions[i]); @@ -306,9 +316,14 @@ static const struct v4l2_file_operations v4l2_file_ops = { .poll = vidc_poll, }; +static const struct vb2_ops iris_vb2_ops = { + .queue_setup = iris_vb2_queue_setup, +}; + int init_ops(struct iris_core *core) { core->v4l2_file_ops = &v4l2_file_ops; + core->vb2_ops = &iris_vb2_ops; return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 8305c65..e242614 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -9,6 +9,8 @@ #include #include +#include "iris_common.h" + struct iris_core; struct iris_inst; @@ -35,20 +37,6 @@ struct iris_inst; .bank_spreading = bsp, \ } -enum codec_type { - H264 = BIT(0), - HEVC = BIT(1), - VP9 = BIT(2), -}; - -enum colorformat_type { - FMT_NONE = 0, - FMT_NV12C = BIT(0), - FMT_NV12 = BIT(1), - FMT_NV21 = BIT(2), - FMT_TP10C = BIT(3), -}; - enum stage_type { STAGE_NONE = 0, STAGE_1 = 1, From patchwork Mon Dec 18 11:32:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757240 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDB14200C7; Mon, 18 Dec 2023 11:40:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Iow/hLS0" Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBDuCL018171; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=U89F2RsbUBam7okcX4qquI23KYt3c/bBrBXYoIhu7nc=; b=Io w/hLS0iYU+2wA536pvvoU0BPIdStACjIV8AU/h+87eQsegFLHRhM6Hxqxh8Lmk3B SSFddYPVaBBwJ0cxjOS84ggBpae47gPxsAQ2qQWADmiKqyDuUAje1ccXKEz1f6Gn uifQHzkc7MRQJAhUKCWtcTB00nJ6dYqB1XVE7evRhqKmkD0RLFLX86S6PkAQltAO M5TFvN1uezVsptxx+7JZem5mEJjSLJVboayoPe9qTT+seWeTBEbrPxd35dND7KPF D0eYtfrnDmEQNvERMLgnQdxpSN4z1yuKdgUR90vkfHQyuHSQnHnJXjlX+6MDa2KM AaFvGrcc/izLM855ieWQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2n1781tu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0wx004854; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ag-3; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX8Ya030075; Mon, 18 Dec 2023 11:33:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8hq030062; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 4C494230A; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops Date: Mon, 18 Dec 2023 17:02:13 +0530 Message-Id: <1702899149-21321-19-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: X0VFrL2CzS5cfywwCIMg4PGknbvtbMVt X-Proofpoint-GUID: X0VFrL2CzS5cfywwCIMg4PGknbvtbMVt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=985 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 adultscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 malwarescore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Implement the iris vb2 mem ops for buffer management for DMABUF streaming mode. Update video driver buffer with dma buf information. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/iris_probe.c | 1 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 147 +++++++++++++++++++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 10 ++ .../media/platform/qcom/vcodec/iris/iris_vidc.c | 11 ++ 4 files changed, 169 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 50fb93e..bf484a3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -233,5 +233,6 @@ static struct platform_driver qcom_iris_driver = { }; module_platform_driver(qcom_iris_driver); +MODULE_IMPORT_NS(DMA_BUF); MODULE_DESCRIPTION("Qualcomm Iris video driver"); MODULE_LICENSE("GPL"); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index b040d27..a57b5fb 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -69,3 +69,150 @@ int iris_vb2_queue_setup(struct vb2_queue *q, return ret; } + +void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, + struct dma_buf *dbuf, unsigned long size) +{ + enum iris_buffer_type buf_type; + struct iris_buffers *buffers; + struct iris_buffer *iter; + struct iris_buffer *buf; + struct iris_inst *inst; + bool found = false; + + if (!vb || !dev || !dbuf || !vb->vb2_queue) + return ERR_PTR(-EINVAL); + + inst = vb->vb2_queue->drv_priv; + + buf_type = v4l2_type_to_driver(vb->type); + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return NULL; + + list_for_each_entry(iter, &buffers->list, list) { + if (iter->index == vb->index) { + found = true; + buf = iter; + break; + } + } + + if (!found) + return NULL; + + buf->inst = inst; + buf->dmabuf = dbuf; + + buf->attach = dma_buf_attach(dbuf, dev); + if (IS_ERR(buf->attach)) { + buf->attach = NULL; + return NULL; + } + + return buf; +} + +int iris_vb2_map_dmabuf(void *buf_priv) +{ + struct iris_buffer *buf = buf_priv; + struct iris_core *core; + struct iris_inst *inst; + + if (!buf || !buf->inst) + return -EINVAL; + + inst = buf->inst; + core = inst->core; + + if (!buf->attach) { + dev_err(core->dev, "trying to map a non attached buffer\n"); + return -EINVAL; + } + + buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL); + if (IS_ERR(buf->sg_table)) + return -EINVAL; + + if (!buf->sg_table->sgl) { + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + buf->sg_table = NULL; + return -EINVAL; + } + + buf->device_addr = sg_dma_address(buf->sg_table->sgl); + + return 0; +} + +void iris_vb2_unmap_dmabuf(void *buf_priv) +{ + struct iris_buffer *buf = buf_priv; + struct iris_core *core; + struct iris_inst *inst; + + if (!buf || !buf->inst) + return; + + inst = buf->inst; + core = inst->core; + + if (!buf->attach) { + dev_err(core->dev, "trying to unmap a non attached buffer\n"); + return; + } + + if (!buf->sg_table) { + dev_err(core->dev, "dmabuf buffer is already unmapped\n"); + return; + } + + if (buf->attach && buf->sg_table) { + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + buf->sg_table = NULL; + buf->device_addr = 0x0; + } +} + +void iris_vb2_detach_dmabuf(void *buf_priv) +{ + struct iris_buffer *buf = buf_priv; + struct iris_core *core; + struct iris_inst *inst; + + if (!buf || !buf->inst) + return; + + inst = buf->inst; + core = inst->core; + + if (buf->sg_table) { + dev_err(core->dev, "trying to detach an unmapped buffer\n"); + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + buf->sg_table = NULL; + } + + if (buf->attach && buf->dmabuf) { + dma_buf_detach(buf->dmabuf, buf->attach); + buf->attach = NULL; + } + + buf->dmabuf = NULL; + buf->inst = NULL; +} + +void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, + unsigned long size) +{ + return (void *)0xdeadbeef; +} + +void iris_vb2_put(void *buf_priv) +{ +} + +int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma) +{ + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h index 8a8e1039..4342034 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h @@ -12,4 +12,14 @@ int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, unsigned int sizes[], struct device *alloc_devs[]); +/* vb2_mem_ops */ +void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size); +void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, struct dma_buf *dbuf, + unsigned long size); +void iris_vb2_put(void *buf_priv); +int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma); +void iris_vb2_detach_dmabuf(void *buf_priv); +int iris_vb2_map_dmabuf(void *buf_priv); +void iris_vb2_unmap_dmabuf(void *buf_priv); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 410de720..124333a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -320,10 +320,21 @@ static const struct vb2_ops iris_vb2_ops = { .queue_setup = iris_vb2_queue_setup, }; +static struct vb2_mem_ops iris_vb2_mem_ops = { + .alloc = iris_vb2_alloc, + .put = iris_vb2_put, + .mmap = iris_vb2_mmap, + .attach_dmabuf = iris_vb2_attach_dmabuf, + .detach_dmabuf = iris_vb2_detach_dmabuf, + .map_dmabuf = iris_vb2_map_dmabuf, + .unmap_dmabuf = iris_vb2_unmap_dmabuf, +}; + int init_ops(struct iris_core *core) { core->v4l2_file_ops = &v4l2_file_ops; core->vb2_ops = &iris_vb2_ops; + core->vb2_mem_ops = &iris_vb2_mem_ops; return 0; } From patchwork Mon Dec 18 11:32:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757242 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D30881A731; Mon, 18 Dec 2023 11:36:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="h6wv04H/" Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBSDl9013281; Mon, 18 Dec 2023 11:36:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=AT6XCjkhjq3ISyvWWHdZVQ0FD6UCnq0F78em27ik0vU=; b=h6 wv04H/d/RwSFopH0Hr8u+mHOkwflTKOwvpRS1D0cgM1qU8FswZKKCVMtRMiVho4i uJORx3/yMGj5pMoV/YNFqH2exuXZlybDr/lX1k4hqBvYwcaJv+XkOVmrf9ZjmsS9 NtT/SbHA+qByfQUGpLi4LOjRRYVKVFple16UkrN2j1csQ+q8Lv/Oy7uTEBGdHST2 t4CMrOkDDBGYfljOyF5UvFP9wwSR6Vn2UK2BuwSDVaKuSYLTNkFyvOeHdl9Wkw+6 UKNE/oAdHXxz6Pf6Wgmtq25agmnUbqHniatLYdDJIroNKwl1McV3fXDHavL8MHmt JdUgpURVrKmUPuEzodQg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2jx0gbh9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:13 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5MF029943; Mon, 18 Dec 2023 11:36:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyju-1; Mon, 18 Dec 2023 11:33:10 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX72t029995; Mon, 18 Dec 2023 11:33:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6B3029969; Mon, 18 Dec 2023 11:33:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 4EB74230B; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 19/34] media: iris: implement HFI to queue and release buffers Date: Mon, 18 Dec 2023 17:02:14 +0530 Message-Id: <1702899149-21321-20-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: DU0PAr79gSsWFZMPeAeSOdjevmmzDE0j X-Proofpoint-GUID: DU0PAr79gSsWFZMPeAeSOdjevmmzDE0j X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 phishscore=0 priorityscore=1501 suspectscore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 mlxscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Introduce and implement HFIs to queue and release all input, output and internal buffers to/from firmware. HFI_CMD_BUFFER - to submit the buffer to firmware. HFI_CMD_BUFFER with HFI_BUF_HOST_FLAG_RELEASE - to request firmware to release the buffer. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 25 +++++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 85 ++++++++++++++++++++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 78 ++++++++++++++++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 15 ++++ 5 files changed, 207 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index 9dd5f11..c044f78 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -13,6 +13,7 @@ #define HFI_CMD_INIT 0x01000001 #define HFI_CMD_OPEN 0x01000003 #define HFI_CMD_CLOSE 0x01000004 +#define HFI_CMD_BUFFER 0x01000009 #define HFI_PROP_IMAGE_VERSION 0x03000001 @@ -60,4 +61,28 @@ struct hfi_debug_header { u32 reserved[2]; }; +enum hfi_buffer_type { + HFI_BUFFER_BITSTREAM = 0x00000001, + HFI_BUFFER_RAW = 0x00000002, + HFI_BUFFER_METADATA = 0x00000003, + HFI_BUFFER_SUBCACHE = 0x00000004, + HFI_BUFFER_PARTIAL_DATA = 0x00000005, + HFI_BUFFER_DPB = 0x00000006, + HFI_BUFFER_BIN = 0x00000007, + HFI_BUFFER_LINE = 0x00000008, + HFI_BUFFER_ARP = 0x00000009, + HFI_BUFFER_COMV = 0x0000000A, + HFI_BUFFER_NON_COMV = 0x0000000B, + HFI_BUFFER_PERSIST = 0x0000000C, + HFI_BUFFER_VPSS = 0x0000000D, +}; + +enum hfi_buffer_host_flags { + HFI_BUF_HOST_FLAG_NONE = 0x00000000, + HFI_BUF_HOST_FLAG_RELEASE = 0x00000001, + HFI_BUF_HOST_FLAG_READONLY = 0x00000010, + HFI_BUF_HOST_FLAG_CODEC_CONFIG = 0x00000100, + HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200, +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 24ddb98..8f1e456 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -309,3 +309,88 @@ int iris_hfi_set_property(struct iris_inst *inst, return ret; } + +int iris_hfi_queue_buffer(struct iris_inst *inst, + struct iris_buffer *buffer) +{ + struct hfi_buffer hfi_buffer; + struct iris_core *core; + int ret; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + ret = get_hfi_buffer(buffer, &hfi_buffer); + if (ret) + goto unlock; + + ret = hfi_packet_session_command(inst, + HFI_CMD_BUFFER, + HFI_HOST_FLAGS_INTR_REQUIRED, + get_hfi_port_from_buffer_type(buffer->type), + inst->session_id, + HFI_PAYLOAD_STRUCTURE, + &hfi_buffer, + sizeof(hfi_buffer)); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_release_buffer(struct iris_inst *inst, + struct iris_buffer *buffer) +{ + struct hfi_buffer hfi_buffer; + struct iris_core *core; + int ret; + + if (!inst->packet || !buffer) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + ret = get_hfi_buffer(buffer, &hfi_buffer); + if (ret) + goto unlock; + + hfi_buffer.flags |= HFI_BUF_HOST_FLAG_RELEASE; + + ret = hfi_packet_session_command(inst, + HFI_CMD_BUFFER, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + get_hfi_port_from_buffer_type(buffer->type), + inst->session_id, + HFI_PAYLOAD_STRUCTURE, + &hfi_buffer, + sizeof(hfi_buffer)); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index bf991bb..4aefdc4 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -19,5 +19,9 @@ int iris_hfi_set_property(struct iris_inst *inst, irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); +int iris_hfi_queue_buffer(struct iris_inst *inst, + struct iris_buffer *buffer); +int iris_hfi_release_buffer(struct iris_inst *inst, + struct iris_buffer *buffer); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index 749d978..a3544d8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -4,9 +4,87 @@ */ #include "iris_core.h" +#include "iris_helpers.h" #include "iris_hfi_packet.h" #include "hfi_defines.h" +u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type) +{ + u32 hfi_port = HFI_PORT_NONE; + + switch (buffer_type) { + case BUF_INPUT: + case BUF_BIN: + case BUF_COMV: + case BUF_NON_COMV: + case BUF_LINE: + hfi_port = HFI_PORT_BITSTREAM; + break; + case BUF_OUTPUT: + case BUF_DPB: + hfi_port = HFI_PORT_RAW; + break; + case BUF_PERSIST: + hfi_port = HFI_PORT_NONE; + break; + default: + break; + } + + return hfi_port; +} + +static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return HFI_BUFFER_BITSTREAM; + case BUF_OUTPUT: + return HFI_BUFFER_RAW; + case BUF_BIN: + return HFI_BUFFER_BIN; + case BUF_COMV: + return HFI_BUFFER_COMV; + case BUF_NON_COMV: + return HFI_BUFFER_NON_COMV; + case BUF_LINE: + return HFI_BUFFER_LINE; + case BUF_DPB: + return HFI_BUFFER_DPB; + case BUF_PERSIST: + return HFI_BUFFER_PERSIST; + default: + return 0; + } +} + +int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) +{ + memset(buf, 0, sizeof(*buf)); + buf->type = hfi_buf_type_from_driver(buffer->type); + buf->index = buffer->index; + buf->base_address = buffer->device_addr; + buf->addr_offset = 0; + buf->buffer_size = buffer->buffer_size; + /* + * for decoder input buffers, firmware (BSE HW) needs 256 aligned + * buffer size otherwise it will truncate or ignore the data after 256 + * aligned size which may lead to error concealment + */ + if (buffer->type == BUF_INPUT) + buf->buffer_size = ALIGN(buffer->buffer_size, 256); + buf->data_offset = buffer->data_offset; + buf->data_size = buffer->data_size; + if (buffer->attr & BUF_ATTR_READ_ONLY) + buf->flags |= HFI_BUF_HOST_FLAG_READONLY; + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + buf->flags |= HFI_BUF_HOST_FLAG_RELEASE; + buf->flags |= HFI_BUF_HOST_FLAGS_CB_NON_SECURE; + buf->timestamp = buffer->timestamp; + + return 0; +} + static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, u32 header_id) { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index bea7ed9..bf18553 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -27,6 +27,19 @@ struct hfi_packet { u32 reserved[2]; }; +struct hfi_buffer { + u32 type; + u32 index; + u64 base_address; + u32 addr_offset; + u32 buffer_size; + u32 data_offset; + u32 data_size; + u64 timestamp; + u32 flags; + u32 reserved[5]; +}; + enum hfi_packet_host_flags { HFI_HOST_FLAGS_NONE = 0x00000000, HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001, @@ -66,6 +79,8 @@ enum hfi_packet_port_type { HFI_PORT_RAW = 0x00000002, }; +u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type); +int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); int hfi_packet_sys_init(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_image_version(struct iris_core *core, From patchwork Mon Dec 18 11:32:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757236 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54D243B2A8; Mon, 18 Dec 2023 11:40:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="HPHbcOzD" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArrqu023249; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Z+YyFcHZoap2j/dtnB//4TT29/XubabIqnFS3x68d88=; b=HP HbcOzDWp7UlWaoaKcCuEb5T6KOP4q547IkWThF+crClrpXawwacmMldnptjDN5Jr S+Et8v2377q0AaeIsP/KfecRZYgilMuOyGGibOxwzn+sb9IKHu0uIvk14d/HBxgr 9ctOiYgaQTkhSKLDH76DAjvuNZ1jzn/vKeQGhpzvIro2av33oYMbn/Nnl3zKNbfX xelRpPnWy4vBZ0LVWGIEe/QcqS/l7LMfcICoqc+/946s0zkZGroi38chvPF0PFvu tT3GQg+rbhOdEYXHfdfDh21swvz2qP+Z7vNI7Qg6NBlOAnrQ+JBff4I0/xECw3u4 CTDmaSN/nbVqqvWcpOsA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2md2r4n7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0Ho004853; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ae-4; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7dP029984; Mon, 18 Dec 2023 11:36:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6a8029970; Mon, 18 Dec 2023 11:36:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 51204230C; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation Date: Mon, 18 Dec 2023 17:02:15 +0530 Message-Id: <1702899149-21321-21-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 0cYxUNueEMdBjUclxbGTb8vDKFBWxrpS X-Proofpoint-GUID: 0cYxUNueEMdBjUclxbGTb8vDKFBWxrpS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 suspectscore=0 lowpriorityscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Video driver needs various kind of internal buffers for frame processing. Internal buffer size calculation depends on hardware architecture, color format, resolution and codec. Add APIs to calculate min count and size of different internal buffers for different codecs for iris3. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 1 + .../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++ .../media/platform/qcom/vcodec/iris/iris_common.h | 1 + .../media/platform/qcom/vcodec/iris/iris_core.h | 2 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 + .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 + .../platform/qcom/vcodec/iris/iris_instance.h | 2 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 + .../media/platform/qcom/vcodec/iris/vpu_common.h | 8 + .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 + .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++ .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++ 12 files changed, 1132 insertions(+) create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index a94e36b..7e3d9f1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -16,6 +16,7 @@ iris-objs += iris_probe.o \ resources.o \ vpu_common.o \ vpu_iris3.o \ + vpu_iris3_buffer.o \ platform_common.o \ platform_sm8550.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c index b9cffbf..6d4e722 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -17,6 +17,14 @@ static int output_min_count(struct iris_inst *inst) { int output_min_count; + /* fw_min_count > 0 indicates reconfig event has already arrived */ + if (inst->fw_min_count) { + if (is_split_mode_enabled(inst) && inst->codec == VP9) + return min_t(u32, 4, inst->fw_min_count); + else + return inst->fw_min_count; + } + switch (inst->codec) { case H264: case HEVC: @@ -33,6 +41,38 @@ static int output_min_count(struct iris_inst *inst) return output_min_count; } +static u32 internal_buffer_count(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + u32 count = 0; + + if (buffer_type == BUF_BIN || buffer_type == BUF_LINE || + buffer_type == BUF_PERSIST) { + count = 1; + } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) { + if (inst->codec == H264 || inst->codec == HEVC) + count = 1; + else + count = 0; + } else { + count = 0; + } + + return count; +} + +static int dpb_count(struct iris_inst *inst) +{ + int count = 0; + + if (is_split_mode_enabled(inst)) { + count = inst->fw_min_count ? + inst->fw_min_count : inst->buffers.output.min_count; + } + + return count; +} + int iris_get_buf_min_count(struct iris_inst *inst, enum iris_buffer_type buffer_type) { @@ -41,6 +81,14 @@ int iris_get_buf_min_count(struct iris_inst *inst, return input_min_count(inst); case BUF_OUTPUT: return output_min_count(inst); + case BUF_BIN: + case BUF_COMV: + case BUF_NON_COMV: + case BUF_LINE: + case BUF_PERSIST: + return internal_buffer_count(inst, buffer_type); + case BUF_DPB: + return dpb_count(inst); default: return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 4edadc3..a83d1c1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -15,6 +15,7 @@ #define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE #define DEFAULT_WIDTH 320 #define DEFAULT_HEIGHT 240 +#define DEFAULT_BSE_VPP_DELAY 2 #define MB_IN_PIXEL (16 * 16) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 30f7ad7..154991c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -50,6 +50,7 @@ * @header_id: id of packet header * @packet_id: id of packet * @vpu_ops: a pointer to vpu ops + * @session_ops: a pointer to session level ops * @dec_codecs_count: supported codec count for decoder * @platform_data: a structure for platform data * @cap: an array for supported core capabilities @@ -91,6 +92,7 @@ struct iris_core { u32 header_id; u32 packet_id; const struct vpu_ops *vpu_ops; + const struct vpu_session_ops *session_ops; u32 dec_codecs_count; struct platform_data *platform_data; struct plat_core_cap cap[CORE_CAP_MAX + 1]; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 54a7851..335885f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -66,6 +66,19 @@ int get_mbpf(struct iris_inst *inst) return NUM_MBS_PER_FRAME(height, width); } +bool is_linear_colorformat(u32 colorformat) +{ + return colorformat == V4L2_PIX_FMT_NV12 || colorformat == V4L2_PIX_FMT_NV21; +} + +bool is_split_mode_enabled(struct iris_inst *inst) +{ + if (is_linear_colorformat(inst->fmt_dst->fmt.pix_mp.pixelformat)) + return true; + + return false; +} + static int process_inst_timeout(struct iris_inst *inst) { struct iris_inst *instance; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 3bae969..9e85510 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -12,6 +12,7 @@ #include "iris_core.h" #include "iris_instance.h" #include "iris_buffer.h" +#include "iris_instance.h" #include "platform_common.h" #define NUM_MBS_PER_FRAME(__height, __width) \ @@ -26,4 +27,7 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type); int get_mbpf(struct iris_inst *inst); int close_session(struct iris_inst *inst); +bool is_linear_colorformat(u32 colorformat); +bool is_split_mode_enabled(struct iris_inst *inst); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index a5c6cb48..5d4c856 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -36,6 +36,7 @@ * @caps_list: list head of capability * @codec: codec type * @buffers: structure of buffer info + * @fw_min_count: minimnum count of buffers needed by fw */ struct iris_inst { @@ -58,6 +59,7 @@ struct iris_inst { struct list_head caps_list; enum codec_type codec; struct iris_buffers_info buffers; + u32 fw_min_count; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 1b957a13..b131a50 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -39,6 +39,7 @@ void vdec_inst_init(struct iris_inst *inst) f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + inst->fw_min_count = 0; } void vdec_inst_deinit(struct iris_inst *inst) diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h index 6512039..7fba8c2 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h @@ -26,6 +26,14 @@ struct vpu_ops { int (*watchdog)(struct iris_core *core, u32 intr_status); }; +#define call_session_op(c, op, ...) \ + (((c) && (c)->session_ops && (c)->session_ops->op) ? \ + ((c)->session_ops->op(__VA_ARGS__)) : 0) + +struct vpu_session_ops { + int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type); +}; + int init_vpu(struct iris_core *core); int write_register(struct iris_core *core, u32 reg, u32 value); diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c index a34d0ed..efea5aa 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c @@ -6,6 +6,7 @@ #include #include "vpu_iris3.h" +#include "vpu_iris3_buffer.h" #define VIDEO_ARCH_LX 1 @@ -202,9 +203,14 @@ static const struct vpu_ops iris3_ops = { .watchdog = watchdog_iris3, }; +static const struct vpu_session_ops iris3_session_ops = { + .int_buf_size = iris_int_buf_size_iris3, +}; + int init_iris3(struct iris_core *core) { core->vpu_ops = &iris3_ops; + core->session_ops = &iris3_session_ops; return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c new file mode 100644 index 0000000..44f9342 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c @@ -0,0 +1,201 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_helpers.h" +#include "iris_instance.h" +#include "vpu_iris3_buffer.h" + +static u32 dec_bin_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_vpp_pipes; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + + core = inst->core; + + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (inst->codec == H264) + size = hfi_buffer_bin_h264d(width, height, num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_bin_h265d(width, height, num_vpp_pipes); + else if (inst->codec == VP9) + size = hfi_buffer_bin_vp9d(width, height, + num_vpp_pipes); + return size; +} + +static u32 dec_comv_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_comv; + struct v4l2_format *f; + u32 size = 0; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + num_comv = inst->buffers.output.min_count; + + if (inst->codec == H264) + size = hfi_buffer_comv_h264d(width, height, num_comv); + else if (inst->codec == HEVC) + size = hfi_buffer_comv_h265d(width, height, num_comv); + + inst->cap[NUM_COMV].value = num_comv; + + return size; +} + +static u32 dec_non_comv_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_vpp_pipes; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + + core = inst->core; + + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (inst->codec == H264) + size = hfi_buffer_non_comv_h264d(width, height, num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_non_comv_h265d(width, height, num_vpp_pipes); + + return size; +} + +static u32 dec_line_size_iris3(struct iris_inst *inst) +{ + u32 width, height, out_min_count, num_vpp_pipes; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + bool is_opb; + + core = inst->core; + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + + is_opb = true; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + out_min_count = inst->buffers.output.min_count; + if (inst->codec == H264) + size = hfi_buffer_line_h264d(width, height, is_opb, + num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_line_h265d(width, height, is_opb, + num_vpp_pipes); + else if (inst->codec == VP9) + size = hfi_buffer_line_vp9d(width, height, out_min_count, + is_opb, num_vpp_pipes); + return size; +} + +static u32 dec_persist_size_iris3(struct iris_inst *inst) +{ + u32 size = 0; + + if (inst->codec == H264) + size = hfi_buffer_persist_h264d(0); + else if (inst->codec == HEVC) + size = hfi_buffer_persist_h265d(0); + else if (inst->codec == VP9) + size = hfi_buffer_persist_vp9d(); + + return size; +} + +static u32 dec_dpb_size_iris3(struct iris_inst *inst) +{ + struct v4l2_format *f; + u32 width, height; + u32 color_fmt; + u32 size = 0; + + f = inst->fmt_dst; + color_fmt = f->fmt.pix_mp.pixelformat; + if (!is_linear_colorformat(color_fmt)) + return size; + + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (color_fmt == V4L2_PIX_FMT_NV12 || + color_fmt == V4L2_PIX_FMT_QC08C) { + size = + hfi_nv12_ubwc_il_calc_buf_size_v2(width, height, + ALIGN(width, 128), + ALIGN(height, 32), + ALIGN(width, 128), + ALIGN((height + 1) >> 1, 32), + ALIGN(DIV_ROUND_UP(width, 32), 64), + ALIGN(DIV_ROUND_UP(height, 8), 16), + ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64), + ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16)); + } else if (color_fmt == V4L2_PIX_FMT_QC10C) { + size = + hfi_yuv420_tp10_ubwc_calc_buf_size(ALIGN(ALIGN(width, 192) * 4 / 3, 256), + ALIGN(height, 16), + ALIGN(ALIGN(width, 192) * 4 / 3, 256), + ALIGN((height + 1) >> 1, 16), + ALIGN(DIV_ROUND_UP(width, 48), 64), + ALIGN(DIV_ROUND_UP(height, 4), 16), + ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64), + ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16)); + } + + return size; +} + +struct iris_buf_type_handle { + enum iris_buffer_type type; + u32 (*handle)(struct iris_inst *inst); +}; + +int iris_int_buf_size_iris3(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + const struct iris_buf_type_handle *buf_type_handle_arr = NULL; + u32 size = 0, buf_type_handle_size = 0; + int i; + + static const struct iris_buf_type_handle dec_internal_buf_type_handle[] = { + {BUF_BIN, dec_bin_size_iris3 }, + {BUF_COMV, dec_comv_size_iris3 }, + {BUF_NON_COMV, dec_non_comv_size_iris3 }, + {BUF_LINE, dec_line_size_iris3 }, + {BUF_PERSIST, dec_persist_size_iris3 }, + {BUF_DPB, dec_dpb_size_iris3 }, + }; + + buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle); + buf_type_handle_arr = dec_internal_buf_type_handle; + + if (!buf_type_handle_arr || !buf_type_handle_size) + return size; + + for (i = 0; i < buf_type_handle_size; i++) { + if (buf_type_handle_arr[i].type == buffer_type) { + size = buf_type_handle_arr[i].handle(inst); + break; + } + } + + return size; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h new file mode 100644 index 0000000..b520c79 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h @@ -0,0 +1,845 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _VPU_IRIS3_BUFFER_H_ +#define _VPU_IRIS3_BUFFER_H_ + +#include +#include +#include + +#include "iris_instance.h" + +#define DMA_ALIGNMENT 256 + +#define BUFFER_ALIGNMENT_512_BYTES 512 +#define BUFFER_ALIGNMENT_256_BYTES 256 +#define BUFFER_ALIGNMENT_64_BYTES 64 +#define BUFFER_ALIGNMENT_32_BYTES 32 +#define BUFFER_ALIGNMENT_16_BYTES 16 + +#define HFI_ALIGNMENT_4096 (4096) + +#define HFI_COL_FMT_NV12C_Y_TILE_HEIGHT (8) +#define HFI_COL_FMT_NV12C_Y_TILE_WIDTH (32) +#define HFI_COL_FMT_NV12C_UV_TILE_HEIGHT (8) +#define HFI_COL_FMT_NV12C_UV_TILE_WIDTH (16) + +#define NUM_HW_PIC_BUF 32 +#define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf)) + +#define MAX_TILE_COLUMNS 32 + +#define LCU_MAX_SIZE_PELS 64 +#define LCU_MIN_SIZE_PELS 16 + +#define HDR10_HIST_EXTRADATA_SIZE (4 * 1024) + +#define BIN_BUFFER_THRESHOLD (1280 * 736) + +#define VPP_CMD_MAX_SIZE (BIT(20)) + +#define H264D_MAX_SLICE 1800 + +#define SIZE_H264D_BUFTAB_T (256) +#define SIZE_H264D_HW_PIC_T (BIT(11)) +#define SIZE_H264D_BSE_CMD_PER_BUF (32 * 4) +#define SIZE_H264D_VPP_CMD_PER_BUF (512) + +#define NUM_SLIST_BUF_H264 (256 + 32) +#define SIZE_SLIST_BUF_H264 (512) +#define H264_DISPLAY_BUF_SIZE (3328) +#define H264_NUM_FRM_INFO (66) + +#define H265_NUM_TILE_COL 32 +#define H265_NUM_TILE_ROW 128 +#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1) +#define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32)) + +#define NUM_SLIST_BUF_H265 (80 + 20) +#define SIZE_SLIST_BUF_H265 (BIT(10)) +#define H265_DISPLAY_BUF_SIZE (3072) +#define H265_NUM_FRM_INFO (48) + +#define VP9_NUM_FRAME_INFO_BUF 32 +#define VP9_NUM_PROBABILITY_TABLE_BUF (VP9_NUM_FRAME_INFO_BUF + 4) +#define VP9_PROB_TABLE_SIZE (3840) +#define VP9_FRAME_INFO_BUF_SIZE (6144) + +#define VP9_UDC_HEADER_BUF_SIZE (3 * 128) +#define MAX_SUPERFRAME_HEADER_LEN (34) +#define CCE_TILE_OFFSET_SIZE ALIGN(32 * 4 * 4, BUFFER_ALIGNMENT_32_BYTES) + +#define SIZE_SEI_USERDATA (4096) +#define SIZE_DOLBY_RPU_METADATA (41 * 1024) + +#define H264_CABAC_HDR_RATIO_HD_TOT 1 +#define H264_CABAC_RES_RATIO_HD_TOT 3 + +#define H265D_MAX_SLICE 1200 +#define SIZE_H265D_HW_PIC_T SIZE_H264D_HW_PIC_T +#define H265_CABAC_HDR_RATIO_HD_TOT 2 +#define H265_CABAC_RES_RATIO_HD_TOT 2 +#define SIZE_H265D_VPP_CMD_PER_BUF (256) + +#define VPX_DECODER_FRAME_CONCURENCY_LVL (2) +#define VPX_DECODER_FRAME_BIN_HDR_BUDGET 1 +#define VPX_DECODER_FRAME_BIN_RES_BUDGET 3 +#define VPX_DECODER_FRAME_BIN_DENOMINATOR 2 + +#define VPX_DECODER_FRAME_BIN_RES_BUDGET_RATIO (3 / 2) + +#define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 64 +#define MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE 64 +#define MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE 64 + +#define MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE (128 / 8) +#define MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE (128 / 8) +#define MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE (128 / 8) + +#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE (64 * 2 * 3) +#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640 + +static inline +u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 size_yuv, size_bin_hdr, size_bin_res; + + size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ? + ((BIN_BUFFER_THRESHOLD * 3) >> 1) : + ((frame_width * frame_height * 3) >> 1); + size_bin_hdr = size_yuv * H264_CABAC_HDR_RATIO_HD_TOT; + size_bin_res = size_yuv * H264_CABAC_RES_RATIO_HD_TOT; + size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, + DMA_ALIGNMENT) * num_vpp_pipes; + size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, + DMA_ALIGNMENT) * num_vpp_pipes; + + return size_bin_hdr + size_bin_res; +} + +static inline +u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 n_aligned_w, n_aligned_h; + + n_aligned_w = + ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES); + n_aligned_h = + ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES); + + return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h, + num_vpp_pipes); +} + +static inline +u32 size_h265d_hw_bin_buffer(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 size_yuv, size_bin_hdr, size_bin_res; + + size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ? + ((BIN_BUFFER_THRESHOLD * 3) >> 1) : + ((frame_width * frame_height * 3) >> 1); + size_bin_hdr = size_yuv * H265_CABAC_HDR_RATIO_HD_TOT; + size_bin_res = size_yuv * H265_CABAC_RES_RATIO_HD_TOT; + size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, DMA_ALIGNMENT) * + num_vpp_pipes; + size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, DMA_ALIGNMENT) * + num_vpp_pipes; + + return size_bin_hdr + size_bin_res; +} + +static inline +u32 hfi_buffer_bin_h265d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 n_aligned_w, n_aligned_h; + + n_aligned_w = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES); + n_aligned_h = ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES); + return size_h265d_hw_bin_buffer(n_aligned_w, n_aligned_h, + num_vpp_pipes); +} + +static inline +u32 hfi_buffer_bin_vp9d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 _size_yuv, _size; + + _size_yuv = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES) * + ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES) * 3 / 2; + _size = ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) * + VPX_DECODER_FRAME_BIN_HDR_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR * + VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes), + DMA_ALIGNMENT) + + ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) * + VPX_DECODER_FRAME_BIN_RES_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR * + VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes), + DMA_ALIGNMENT); + + return _size * num_vpp_pipes; +} + +static inline +u32 hfi_buffer_comv_h264d(u32 frame_width, u32 frame_height, + u32 _comv_bufcount) +{ + u32 frame_width_in_mbs = ((frame_width + 15) >> 4); + u32 frame_height_in_mbs = ((frame_height + 15) >> 4); + u32 col_mv_aligned_width = (frame_width_in_mbs << 7); + u32 col_zero_aligned_width = (frame_width_in_mbs << 2); + u32 col_zero_size = 0, size_colloc = 0; + + col_mv_aligned_width = + ALIGN(col_mv_aligned_width, BUFFER_ALIGNMENT_16_BYTES); + col_zero_aligned_width = + ALIGN(col_zero_aligned_width, BUFFER_ALIGNMENT_16_BYTES); + col_zero_size = col_zero_aligned_width * + ((frame_height_in_mbs + 1) >> 1); + col_zero_size = + ALIGN(col_zero_size, BUFFER_ALIGNMENT_64_BYTES); + col_zero_size <<= 1; + col_zero_size = + ALIGN(col_zero_size, BUFFER_ALIGNMENT_512_BYTES); + size_colloc = col_mv_aligned_width * + ((frame_height_in_mbs + 1) >> 1); + size_colloc = + ALIGN(size_colloc, BUFFER_ALIGNMENT_64_BYTES); + size_colloc <<= 1; + size_colloc = + ALIGN(size_colloc, BUFFER_ALIGNMENT_512_BYTES); + size_colloc += (col_zero_size + SIZE_H264D_BUFTAB_T * 2); + + return (size_colloc * (_comv_bufcount)) + + BUFFER_ALIGNMENT_512_BYTES; +} + +static inline +u32 hfi_buffer_comv_h265d(u32 frame_width, u32 frame_height, + u32 _comv_bufcount) +{ + u32 _size; + + _size = ALIGN(((((frame_width + 15) >> 4) * + ((frame_height + 15) >> 4)) << 8), + BUFFER_ALIGNMENT_512_BYTES); + _size *= _comv_bufcount; + _size += BUFFER_ALIGNMENT_512_BYTES; + + return _size; +} + +static inline +u32 size_h264d_bse_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _height = ALIGN(frame_height, + BUFFER_ALIGNMENT_32_BYTES); + return min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) * + SIZE_H264D_BSE_CMD_PER_BUF; +} + +static inline +u32 size_h264d_vpp_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _size, _height; + + _height = ALIGN(frame_height, BUFFER_ALIGNMENT_32_BYTES); + _size = min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) * + SIZE_H264D_VPP_CMD_PER_BUF; + + if (_size > VPP_CMD_MAX_SIZE) + _size = VPP_CMD_MAX_SIZE; + + return _size; +} + +static inline +u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 _size_bse, _size_vpp, _size; + + _size_bse = size_h264d_bse_cmd_buf(frame_width, frame_height); + _size_vpp = size_h264d_vpp_cmd_buf(frame_width, frame_height); + _size = ALIGN(_size_bse, DMA_ALIGNMENT) + + ALIGN(_size_vpp, DMA_ALIGNMENT) + + ALIGN(SIZE_HW_PIC(SIZE_H264D_HW_PIC_T), DMA_ALIGNMENT); + + return ALIGN(_size, DMA_ALIGNMENT); +} + +static inline +u32 size_h265d_bse_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _size; + + _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) * + NUM_HW_PIC_BUF, DMA_ALIGNMENT); + _size = min_t(u32, _size, H265D_MAX_SLICE + 1); + _size = 2 * _size * SIZE_H265D_BSE_CMD_PER_BUF; + + return _size; +} + +static inline +u32 size_h265d_vpp_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _size; + + _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) * + NUM_HW_PIC_BUF, DMA_ALIGNMENT); + _size = min_t(u32, _size, H265D_MAX_SLICE + 1); + _size = ALIGN(_size, 4); + _size = 2 * _size * SIZE_H265D_VPP_CMD_PER_BUF; + if (_size > VPP_CMD_MAX_SIZE) + _size = VPP_CMD_MAX_SIZE; + + return _size; +} + +static inline +u32 hfi_buffer_non_comv_h265d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 _size_bse, _size_vpp, _size; + + _size_bse = size_h265d_bse_cmd_buf(frame_width, frame_height); + _size_vpp = size_h265d_vpp_cmd_buf(frame_width, frame_height); + _size = ALIGN(_size_bse, DMA_ALIGNMENT) + + ALIGN(_size_vpp, DMA_ALIGNMENT) + + ALIGN(NUM_HW_PIC_BUF * 20 * 22 * 4, DMA_ALIGNMENT) + + ALIGN(2 * sizeof(u16) * + (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS), + DMA_ALIGNMENT) + + ALIGN(SIZE_HW_PIC(SIZE_H265D_HW_PIC_T), + DMA_ALIGNMENT) + + HDR10_HIST_EXTRADATA_SIZE; + + return ALIGN(_size, DMA_ALIGNMENT); +} + +static inline u32 size_h264d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * ALIGN(frame_width, 16) * 3; +} + +static inline u32 size_h264d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4); +} + +static inline u32 size_h264d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4); +} + +static inline u32 size_h264d_lb_se_top_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4); +} + +static inline u32 size_h264d_lb_se_left_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4); +} + +static inline u32 size_h264d_lb_pe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4); +} + +static inline u32 size_h264d_lb_vsp_top(u32 frame_width, u32 frame_height) +{ + return (((frame_width + 15) >> 4) << 7); +} + +static inline u32 size_h264d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height) +{ + return ALIGN(frame_height, 16) * 32; +} + +static inline u32 size_h264d_qp(u32 frame_width, u32 frame_height) +{ + return ((frame_width + 63) >> 6) * ((frame_height + 63) >> 6) * 128; +} + +static inline +u32 size_vpss_lb(u32 frame_width, u32 frame_height, u32 num_vpp_pipes) +{ + u32 vpss_4tap_left_buffer_size = 0, vpss_div2_left_buffer_size = 0; + u32 vpss_4tap_top_buffer_size = 0, vpss_div2_top_buffer_size = 0; + u32 opb_lb_wr_llb_y_buffer_size, opb_lb_wr_llb_uv_buffer_size; + u32 opb_wr_top_line_chroma_buffer_size; + u32 opb_wr_top_line_luma_buffer_size; + u32 macrotiling_size = 32, size; + + opb_wr_top_line_luma_buffer_size = + ALIGN(frame_width, macrotiling_size) / + macrotiling_size * 256; + opb_wr_top_line_luma_buffer_size = + ALIGN(opb_wr_top_line_luma_buffer_size, DMA_ALIGNMENT) + + (MAX_TILE_COLUMNS - 1) * 256; + opb_wr_top_line_luma_buffer_size = + max_t(u32, opb_wr_top_line_luma_buffer_size, + (32 * ALIGN(frame_height, 8))); + opb_wr_top_line_chroma_buffer_size = + opb_wr_top_line_luma_buffer_size; + opb_lb_wr_llb_uv_buffer_size = + ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, + BUFFER_ALIGNMENT_32_BYTES); + opb_lb_wr_llb_y_buffer_size = + ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, + BUFFER_ALIGNMENT_32_BYTES); + size = num_vpp_pipes * 2 * + (vpss_4tap_top_buffer_size + vpss_div2_top_buffer_size) + + 2 * (vpss_4tap_left_buffer_size + vpss_div2_left_buffer_size) + + opb_wr_top_line_luma_buffer_size + + opb_wr_top_line_chroma_buffer_size + + opb_lb_wr_llb_uv_buffer_size + + opb_lb_wr_llb_y_buffer_size; + + return size; +} + +static inline +u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height, + bool is_opb, u32 num_vpp_pipes) +{ + u32 vpss_lb_size = 0; + u32 _size; + + _size = ALIGN(size_h264d_lb_fe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_fe_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_fe_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h264d_lb_se_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_se_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h264d_lb_pe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_vsp_top(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height), + DMA_ALIGNMENT) * 2 + + ALIGN(size_h264d_qp(frame_width, frame_height), + DMA_ALIGNMENT); + _size = ALIGN(_size, DMA_ALIGNMENT); + if (is_opb) + vpss_lb_size = size_vpss_lb(frame_width, frame_height, + num_vpp_pipes); + + _size = ALIGN((_size + vpss_lb_size), + DMA_ALIGNMENT); + + return _size; +} + +static inline +u32 size_h265d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * + (ALIGN(frame_width, 64) + 8) * 2; +} + +static inline +u32 size_h265d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * + (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS); +} + +static inline +u32 size_h265d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS); +} + +static inline +u32 size_h265d_lb_se_top_ctrl(u32 frame_width, u32 frame_height) +{ + return (LCU_MAX_SIZE_PELS / 8 * (128 / 8)) * ((frame_width + 15) >> 4); +} + +static inline +u32 size_h265d_lb_se_left_ctrl(u32 frame_width, u32 frame_height) +{ + return max_t(u32, ((frame_height + 16 - 1) / 8) * + MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE, + max_t(u32, ((frame_height + 32 - 1) / 8) * + MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE, + ((frame_height + 64 - 1) / 8) * + MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)); +} + +static inline +u32 size_h265d_lb_pe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE * + (ALIGN(frame_width, LCU_MIN_SIZE_PELS) / LCU_MIN_SIZE_PELS); +} + +static inline +u32 size_h265d_lb_vsp_top(u32 frame_width, u32 frame_height) +{ + return ((frame_width + 63) >> 6) * 128; +} + +static inline +u32 size_h265d_lb_vsp_left(u32 frame_width, u32 frame_height) +{ + return ((frame_height + 63) >> 6) * 128; +} + +static inline +u32 size_h265d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height) +{ + return size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height); +} + +static inline +u32 size_h265d_qp(u32 frame_width, u32 frame_height) +{ + return size_h264d_qp(frame_width, frame_height); +} + +static inline +u32 hfi_buffer_line_h265d(u32 frame_width, u32 frame_height, + bool is_opb, u32 num_vpp_pipes) +{ + u32 vpss_lb_size = 0, _size; + + _size = ALIGN(size_h265d_lb_fe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_fe_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_fe_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h265d_lb_se_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h265d_lb_se_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_pe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_vsp_top(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_vsp_left(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h265d_lb_recon_dma_metadata_wr(frame_width, frame_height), + DMA_ALIGNMENT) * 4 + + ALIGN(size_h265d_qp(frame_width, frame_height), + DMA_ALIGNMENT); + if (is_opb) + vpss_lb_size = size_vpss_lb(frame_width, frame_height, num_vpp_pipes); + + return ALIGN((_size + vpss_lb_size), DMA_ALIGNMENT); +} + +static inline +u32 size_vpxd_lb_fe_left_ctrl(u32 frame_width, u32 frame_height) +{ + return max_t(u32, ((frame_height + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE, + max_t(u32, ((frame_height + 31) >> 5) * MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE, + ((frame_height + 63) >> 6) * MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)); +} + +static inline +u32 size_vpxd_lb_fe_top_ctrl(u32 frame_width, u32 frame_height) +{ + return ((ALIGN(frame_width, 64) + 8) * 10 * 2); +} + +static inline +u32 size_vpxd_lb_se_top_ctrl(u32 frame_width, u32 frame_height) +{ + return ((frame_width + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE; +} + +static inline +u32 size_vpxd_lb_se_left_ctrl(u32 frame_width, u32 frame_height) +{ + return max_t(u32, ((frame_height + 15) >> 4) * MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE, + max_t(u32, ((frame_height + 31) >> 5) * MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE, + ((frame_height + 63) >> 6) * MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)); +} + +static inline +u32 size_vpxd_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height) +{ + return ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, + BUFFER_ALIGNMENT_32_BYTES); +} + +static inline +u32 size_mp2d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return ((ALIGN(frame_width, 16) + 8) * 10 * 2); +} + +static inline +u32 size_vp9d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return (ALIGN(ALIGN(frame_width, 8), 64) + 8) * 10 * 2; +} + +static inline +u32 size_vp9d_lb_pe_top_data(u32 frame_width, u32 frame_height) +{ + return ((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 176); +} + +static inline +u32 size_vp9d_lb_vsp_top(u32 frame_width, u32 frame_height) +{ + return (((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 64 * 8) + 256); +} + +static inline u32 hfi_iris3_vp9d_comv_size(void) +{ + return (((8192 + 63) >> 6) * ((4320 + 63) >> 6) * 8 * 8 * 2 * 8); +} + +static inline +u32 size_vp9d_qp(u32 frame_width, u32 frame_height) +{ + return size_h264d_qp(frame_width, frame_height); +} + +static inline +u32 hfi_iris3_vp9d_lb_size(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + return ALIGN(size_vpxd_lb_fe_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_vpxd_lb_se_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_vp9d_lb_vsp_top(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vpxd_lb_fe_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + 2 * ALIGN(size_vpxd_lb_recon_dma_metadata_wr(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vpxd_lb_se_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vp9d_lb_pe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vp9d_lb_fe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vp9d_qp(frame_width, frame_height), + DMA_ALIGNMENT); +} + +static inline +u32 hfi_buffer_line_vp9d(u32 frame_width, u32 frame_height, + u32 _yuv_bufcount_min, bool is_opb, + u32 num_vpp_pipes) +{ + u32 vpss_lb_size = 0; + u32 _lb_size = 0; + + _lb_size = hfi_iris3_vp9d_lb_size(frame_width, frame_height, + num_vpp_pipes); + + if (is_opb) + vpss_lb_size = size_vpss_lb(frame_width, frame_height, + num_vpp_pipes); + + return _lb_size + vpss_lb_size; +} + +static inline u32 hfi_buffer_persist_h264d(u32 rpu_enabled) +{ + return ALIGN(SIZE_SLIST_BUF_H264 * NUM_SLIST_BUF_H264 + + H264_DISPLAY_BUF_SIZE * H264_NUM_FRM_INFO + + NUM_HW_PIC_BUF * SIZE_SEI_USERDATA + + (rpu_enabled) * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA, + DMA_ALIGNMENT); +} + +static inline +u32 hfi_buffer_persist_h265d(u32 rpu_enabled) +{ + return ALIGN((SIZE_SLIST_BUF_H265 * NUM_SLIST_BUF_H265 + + H265_NUM_FRM_INFO * H265_DISPLAY_BUF_SIZE + + H265_NUM_TILE * sizeof(u32) + + NUM_HW_PIC_BUF * SIZE_SEI_USERDATA + + rpu_enabled * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA), + DMA_ALIGNMENT); +} + +static inline u32 hfi_buffer_persist_vp9d(void) +{ + return ALIGN(VP9_NUM_PROBABILITY_TABLE_BUF * VP9_PROB_TABLE_SIZE, + DMA_ALIGNMENT) + + ALIGN(hfi_iris3_vp9d_comv_size(), DMA_ALIGNMENT) + + ALIGN(MAX_SUPERFRAME_HEADER_LEN, DMA_ALIGNMENT) + + ALIGN(VP9_UDC_HEADER_BUF_SIZE, DMA_ALIGNMENT) + + ALIGN(VP9_NUM_FRAME_INFO_BUF * CCE_TILE_OFFSET_SIZE, + DMA_ALIGNMENT) + + ALIGN(VP9_NUM_FRAME_INFO_BUF * VP9_FRAME_INFO_BUF_SIZE, + DMA_ALIGNMENT) + + HDR10_HIST_EXTRADATA_SIZE; +} + +static inline +u32 hfi_nv12_ubwc_il_calc_y_buf_size(u32 frame_width, u32 frame_height, + u32 stride_multiple, + u32 min_buf_height_multiple) +{ + u32 stride, buf_height; + + stride = ALIGN(frame_width, stride_multiple); + buf_height = ALIGN(frame_height, min_buf_height_multiple); + + return ALIGN(stride * buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_nv12_ubwc_il_calc_uv_buf_size(u32 frame_width, u32 frame_height, + u32 stride_multiple, + u32 min_buf_height_multiple) +{ + u32 uv_stride, uv_buf_height; + + uv_stride = ALIGN(frame_width, stride_multiple); + uv_buf_height = ALIGN(((frame_height + 1) >> 1), + min_buf_height_multiple); + + return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_ubwc_calc_metadata_plane_stride(u32 frame_width, + u32 metadata_stride_multiple, + u32 tile_width_in_pels) +{ + return ALIGN(((frame_width + (tile_width_in_pels - 1)) / tile_width_in_pels), + metadata_stride_multiple); +} + +static inline +u32 hfi_ubwc_metadata_plane_bufheight(u32 frame_height, + u32 metadata_height_multiple, + u32 tile_height_in_pels) +{ + return ALIGN(((frame_height + (tile_height_in_pels - 1)) / tile_height_in_pels), + metadata_height_multiple); +} + +static inline +u32 hfi_ubwc_uv_metadata_plane_stride(u32 frame_width, + u32 metadata_stride_multiple, + u32 tile_width_in_pels) +{ + return ALIGN(((((frame_width + 1) >> 1) + tile_width_in_pels - 1) / + tile_width_in_pels), metadata_stride_multiple); +} + +static inline +u32 hfi_ubwc_uv_metadata_plane_bufheight(u32 frame_height, + u32 metadata_height_multiple, + u32 tile_height_in_pels) +{ + return ALIGN(((((frame_height + 1) >> 1) + tile_height_in_pels - 1) / + tile_height_in_pels), metadata_height_multiple); +} + +static inline +u32 hfi_ubwc_metadata_plane_buffer_size(u32 _metadata_tride, u32 _metadata_buf_height) +{ + return ALIGN(_metadata_tride * _metadata_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_nv12_ubwc_il_calc_buf_size_v2(u32 frame_width, + u32 frame_height, + u32 y_stride_multiple, + u32 y_buffer_height_multiple, + u32 uv_stride_multiple, + u32 uv_buffer_height_multiple, + u32 y_metadata_stride_multiple, + u32 y_metadata_buffer_height_multiple, + u32 uv_metadata_stride_multiple, + u32 uv_metadata_buffer_height_multiple) +{ + u32 y_buf_size, uv_buf_size, y_meta_size, uv_meta_size; + u32 half_height = (frame_height + 1) >> 1; + u32 stride, _height; + + y_buf_size = + hfi_nv12_ubwc_il_calc_y_buf_size(frame_width, half_height, + y_stride_multiple, + y_buffer_height_multiple); + uv_buf_size = + hfi_nv12_ubwc_il_calc_uv_buf_size(frame_width, half_height, + uv_stride_multiple, + uv_buffer_height_multiple); + stride = + hfi_ubwc_calc_metadata_plane_stride(frame_width, + y_metadata_stride_multiple, + HFI_COL_FMT_NV12C_Y_TILE_WIDTH); + _height = + hfi_ubwc_metadata_plane_bufheight(half_height, + y_metadata_buffer_height_multiple, + HFI_COL_FMT_NV12C_Y_TILE_HEIGHT); + y_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height); + stride = + hfi_ubwc_uv_metadata_plane_stride(frame_width, + uv_metadata_stride_multiple, + HFI_COL_FMT_NV12C_UV_TILE_WIDTH); + _height = + hfi_ubwc_uv_metadata_plane_bufheight(half_height, + uv_metadata_buffer_height_multiple, + HFI_COL_FMT_NV12C_UV_TILE_HEIGHT); + uv_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height); + + return (y_buf_size + uv_buf_size + y_meta_size + uv_meta_size) << 1; +} + +static inline +u32 hfi_yuv420_tp10_ubwc_calc_y_buf_size(u32 y_stride, + u32 y_buf_height) +{ + return ALIGN(y_stride * y_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_yuv420_tp10_ubwc_calc_uv_buf_size(u32 uv_stride, + u32 uv_buf_height) +{ + return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_yuv420_tp10_ubwc_calc_buf_size(u32 y_stride, u32 y_buf_height, + u32 uv_stride, u32 uv_buf_height, + u32 y_md_stride, u32 y_md_height, + u32 uv_md_stride, u32 uv_md_height) +{ + u32 y_data_size, uv_data_size, y_md_size, uv_md_size; + + y_data_size = hfi_yuv420_tp10_ubwc_calc_y_buf_size(y_stride, y_buf_height); + uv_data_size = hfi_yuv420_tp10_ubwc_calc_uv_buf_size(uv_stride, uv_buf_height); + y_md_size = hfi_ubwc_metadata_plane_buffer_size(y_md_stride, y_md_height); + uv_md_size = hfi_ubwc_metadata_plane_buffer_size(uv_md_stride, uv_md_height); + + return y_data_size + uv_data_size + y_md_size + uv_md_size; +} + +int iris_int_buf_size_iris3(struct iris_inst *inst, + enum iris_buffer_type buffer_type); + +#endif From patchwork Mon Dec 18 11:32:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755546 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8613020319; Mon, 18 Dec 2023 11:40:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="YQL+b2L3" Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsJhd018720; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=vLWVflzGpI+Jo2k1Eg3SqmsgzB7LOhhG3o6SoZkBRRc=; b=YQ L+b2L3uhBRPdq2Mdrt5CiiRJRt46aYK/rs2islRK6YGKapMIGrjDmXIfYGizvhCi OvrAraXLZO2KFe7QxHePQl2alm9bWF4XIRmk1eI4V6soNYFiJ8uR1d5Cux99Ag58 5MQkD4FEGznSqav64f/6/evTki7jLMGa5xB4Npbeq8j8d+31OWAMHig0hrg3AwRS rYsNy5FW2SbEgIlr324YCbX9RLqVcucbf8gksX1iNhjOerLG1rIAr1D8zrBCw9LX YKS+GR4QJ5k99pS18OnuJjFdWjN+KOg8PWKgQRWFQxLQp/fnqfXlicO9e8pelE3F vfJJGOEOMOfBabROKx1w== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mfe04np-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0Hl004853; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ae-1; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX72r029995; Mon, 18 Dec 2023 11:33:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6II029972; Mon, 18 Dec 2023 11:33:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 5377E2314; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 21/34] media: iris: implement internal buffer management Date: Mon, 18 Dec 2023 17:02:16 +0530 Message-Id: <1702899149-21321-22-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: rx914hXB-zqkaMxTFQopuNafJYIhPCPb X-Proofpoint-GUID: rx914hXB-zqkaMxTFQopuNafJYIhPCPb X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1015 suspectscore=0 malwarescore=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 adultscore=0 impostorscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement functions for creating, queueing, releasing and destroying buffers for internal usage. During any bitstream parameter change, these internal buffers needs to be reallocated and requeued. To optimize the same, driver checks for reusability of any buffer and skip re-allocation/re-queue. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 + .../media/platform/qcom/vcodec/iris/iris_buffer.c | 312 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_buffer.h | 12 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 3 + 5 files changed, 333 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index c044f78..ce4eaff 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -53,6 +53,8 @@ #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 +#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193 + #define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 struct hfi_debug_header { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c index 6d4e722..2a3989c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -6,8 +6,22 @@ #include "../buffers.h" #include "iris_buffer.h" #include "iris_helpers.h" +#include "iris_hfi.h" +#include "hfi_defines.h" +#include "iris_hfi_packet.h" #include "iris_instance.h" +static const u32 dec_ip_int_buf_type[] = { + BUF_BIN, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, +}; + +static const u32 dec_op_int_buf_type[] = { + BUF_DPB, +}; + static int input_min_count(struct iris_inst *inst) { return MIN_BUFFERS; @@ -225,3 +239,301 @@ int iris_free_buffers(struct iris_inst *inst, return 0; } + +static int iris_get_internal_buf_info(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + struct iris_buffers *buffers; + struct iris_core *core; + u32 buf_count; + u32 buf_size; + + core = inst->core; + + buf_size = call_session_op(core, int_buf_size, + inst, buffer_type); + + buf_count = iris_get_buf_min_count(inst, buffer_type); + + buffers = iris_get_buffer_list(inst, buffer_type); + if (!buffers) + return -EINVAL; + + if (buf_size && buf_size <= buffers->size && + buf_count && buf_count <= buffers->min_count) { + buffers->reuse = true; + } else { + buffers->reuse = false; + buffers->size = buf_size; + buffers->min_count = buf_count; + } + + return 0; +} + +int iris_get_internal_buffers(struct iris_inst *inst, + u32 plane) +{ + int ret = 0; + u32 i = 0; + + if (plane == INPUT_MPLANE) { + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } + } else { + return iris_get_internal_buf_info(inst, BUF_DPB); + } + + return ret; +} + +static int iris_create_internal_buffer(struct iris_inst *inst, + enum iris_buffer_type buffer_type, u32 index) +{ + struct iris_buffers *buffers; + struct iris_buffer *buffer; + struct iris_core *core; + + core = inst->core; + + buffers = iris_get_buffer_list(inst, buffer_type); + if (!buffers) + return -EINVAL; + + if (!buffers->size) + return 0; + + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL); + if (!buffer) + return -ENOMEM; + + INIT_LIST_HEAD(&buffer->list); + buffer->type = buffer_type; + buffer->index = index; + buffer->buffer_size = buffers->size; + buffer->dma_attrs = DMA_ATTR_WRITE_COMBINE | DMA_ATTR_NO_KERNEL_MAPPING; + list_add_tail(&buffer->list, &buffers->list); + + buffer->kvaddr = dma_alloc_attrs(core->dev, buffer->buffer_size, + &buffer->device_addr, GFP_KERNEL, buffer->dma_attrs); + + if (!buffer->kvaddr) + return -ENOMEM; + + return 0; +} + +static int iris_create_internal_buffers(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + struct iris_buffers *buffers; + int ret = 0; + int i; + + buffers = iris_get_buffer_list(inst, buffer_type); + if (!buffers) + return -EINVAL; + + if (buffers->reuse) + return 0; + + for (i = 0; i < buffers->min_count; i++) { + ret = iris_create_internal_buffer(inst, buffer_type, i); + if (ret) + return ret; + } + + return ret; +} + +int iris_create_input_internal_buffers(struct iris_inst *inst) +{ + int ret = 0; + u32 i = 0; + + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } + + return ret; +} + +int iris_create_output_internal_buffers(struct iris_inst *inst) +{ + return iris_create_internal_buffers(inst, BUF_DPB); +} + +static int set_num_comv(struct iris_inst *inst) +{ + u32 num_comv; + + num_comv = inst->cap[NUM_COMV].value; + + return iris_hfi_set_property(inst, + HFI_PROP_COMV_BUFFER_COUNT, + HFI_HOST_FLAGS_NONE, + HFI_PORT_BITSTREAM, + HFI_PAYLOAD_U32, + &num_comv, + sizeof(u32)); +} + +static int iris_queue_internal_buffers(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + struct iris_buffer *buffer, *dummy; + struct iris_buffers *buffers; + int ret = 0; + + if (buffer_type == BUF_COMV) { + ret = set_num_comv(inst); + if (ret) + return ret; + } + + buffers = iris_get_buffer_list(inst, buffer_type); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buffer, dummy, &buffers->list, list) { + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + continue; + if (buffer->attr & BUF_ATTR_QUEUED) + continue; + ret = iris_hfi_queue_buffer(inst, buffer); + if (ret) + return ret; + buffer->attr |= BUF_ATTR_QUEUED; + } + + return ret; +} + +int iris_queue_input_internal_buffers(struct iris_inst *inst) +{ + int ret = 0; + u32 i; + + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } + + return ret; +} + +int iris_queue_output_internal_buffers(struct iris_inst *inst) +{ + return iris_queue_internal_buffers(inst, BUF_DPB); +} + +int iris_destroy_internal_buffer(struct iris_inst *inst, + struct iris_buffer *buffer) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + struct iris_core *core; + + core = inst->core; + + buffers = iris_get_buffer_list(inst, buffer->type); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + if (buf->device_addr == buffer->device_addr) { + list_del(&buf->list); + dma_free_attrs(core->dev, buf->buffer_size, buf->kvaddr, + buf->device_addr, buf->dma_attrs); + buf->kvaddr = NULL; + buf->device_addr = 0; + kfree(buf); + break; + } + } + + return 0; +} + +int iris_destroy_internal_buffers(struct iris_inst *inst, + u32 plane) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + const u32 *internal_buf_type; + int ret = 0; + u32 i, len; + + if (plane == INPUT_MPLANE) { + internal_buf_type = dec_ip_int_buf_type; + len = ARRAY_SIZE(dec_ip_int_buf_type); + } else { + internal_buf_type = dec_op_int_buf_type; + len = ARRAY_SIZE(dec_op_int_buf_type); + } + + for (i = 0; i < len; i++) { + buffers = iris_get_buffer_list(inst, internal_buf_type[i]); + if (!buffers) + return -EINVAL; + + if (buffers->reuse) + continue; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + ret = iris_destroy_internal_buffer(inst, buf); + if (ret) + return ret; + } + } + + return ret; +} + +static int iris_release_internal_buffers(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + struct iris_buffer *buffer, *dummy; + struct iris_buffers *buffers; + int ret = 0; + + buffers = iris_get_buffer_list(inst, buffer_type); + if (!buffers) + return -EINVAL; + + if (buffers->reuse) + return 0; + + list_for_each_entry_safe(buffer, dummy, &buffers->list, list) { + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + continue; + if (!(buffer->attr & BUF_ATTR_QUEUED)) + continue; + ret = iris_hfi_release_buffer(inst, buffer); + if (ret) + return ret; + buffer->attr |= BUF_ATTR_PENDING_RELEASE; + } + + return ret; +} + +int iris_release_input_internal_buffers(struct iris_inst *inst) +{ + int ret = 0; + u32 i = 0; + + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h index 1cd76a9..ece894e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -46,4 +46,16 @@ int iris_allocate_buffers(struct iris_inst *inst, int iris_free_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type); +int iris_get_internal_buffers(struct iris_inst *inst, + u32 plane); +int iris_create_input_internal_buffers(struct iris_inst *inst); +int iris_create_output_internal_buffers(struct iris_inst *inst); +int iris_queue_input_internal_buffers(struct iris_inst *inst); +int iris_queue_output_internal_buffers(struct iris_inst *inst); +int iris_destroy_internal_buffer(struct iris_inst *inst, + struct iris_buffer *buffer); +int iris_destroy_internal_buffers(struct iris_inst *inst, + u32 plane); +int iris_release_input_internal_buffers(struct iris_inst *inst); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index 4aefdc4..ca2339e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -16,6 +16,10 @@ int iris_hfi_session_close(struct iris_inst *inst); int iris_hfi_set_property(struct iris_inst *inst, u32 packet_type, u32 flag, u32 plane, u32 payload_type, void *payload, u32 payload_size); +int iris_hfi_queue_buffer(struct iris_inst *inst, + struct iris_buffer *buffer); +int iris_hfi_release_buffer(struct iris_inst *inst, + struct iris_buffer *buffer); irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index b131a50..566048e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -39,6 +39,9 @@ void vdec_inst_init(struct iris_inst *inst) f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT); + inst->buffers.output.actual_count = inst->buffers.output.min_count; + inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage; inst->fw_min_count = 0; } From patchwork Mon Dec 18 11:32:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757241 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C21AA1C698; Mon, 18 Dec 2023 11:40:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="TGLaTVKb" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArvvC003521; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=qhSUy+6TOTvWGOHRrcWbcV5KaOQ6iVEmTghe6XRp+Lg=; b=TG LaTVKb/KVPbaFMXKQ6a+OKqzri/hylahTK5DtWB5wMNVO60nN9C4IwxKM3eA3Zun 9ki6bCuCcvsd+kuhbjN3aOe8d6EngcJQDnnR0QHecys6h6uWF0QHyHupPUj28M4a b3aEXt+kd3GLpBO5XCWMWCAOHu2Fe7Adb3BReTfQYIvmTrUgKTcAsI/eNRFSDupG oCAgeuk2thV8REtKHBKzPIibAeTTLqNrwmtxC47Wuj79UDC23RVseYUyX23HpYqr QgliLZcYSk6vuW1AVjztrTodgoTcjntwibV8dmlp6ziG3wKPuN+XBoHom5cat916 H7nUlExdye6CW+43sdfw== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4xy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0ww004854; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ag-2; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7pR030023; Mon, 18 Dec 2023 11:33:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6B7029967; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 55BF1232E; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 22/34] media: iris: introduce instance states Date: Mon, 18 Dec 2023 17:02:17 +0530 Message-Id: <1702899149-21321-23-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: X7JjZwzpVHK3NnlkLrxR0-h3_3ESkyBr X-Proofpoint-GUID: X7JjZwzpVHK3NnlkLrxR0-h3_3ESkyBr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=944 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Introduce different states for instances. State transitions are defined, based on which they would be allowed/rejected/ignored. IRIS_INST_OPEN - video instance is opened. IRIS_INST_INPUT_STREAMING - stream on is completed on output plane. IRIS_INST_OUTPUT_STREAMING - stream on is completed on capture plane. IRIS_INST_STREAMING - stream on is completed on both output and capture planes. IRIS_INST_CLOSE - video instance is closed. IRIS_INST_ERROR - error state. | v ------------- +---------| OPEN |--------- + | ------------- | | ^ ^ | | / \ | | / \ | | v v | | ----------- ----------- | | | INPUT OUTPUT | | |---| STREAMING STREAMING |---| | ----------- ----------- | | ^ ^ | | \ / | | \ / | | v v | | ------------- | |--------| STREAMING |-----------| | ------------- | | | | | | | | v | | ----------- | +-------->| CLOSE |<----------+ | ----------- | | | | | | | | v | | ---------- | +-------->| ERROR |<-----------+ ---------- Signed-off-by: Dikshita Agarwal --- .../platform/qcom/vcodec/iris/iris_instance.h | 2 + .../media/platform/qcom/vcodec/iris/iris_state.c | 86 ++++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 3 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 5 ++ 5 files changed, 118 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 5d4c856..275efa8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -37,6 +37,7 @@ * @codec: codec type * @buffers: structure of buffer info * @fw_min_count: minimnum count of buffers needed by fw + * @state: instance state */ struct iris_inst { @@ -60,6 +61,7 @@ struct iris_inst { enum codec_type codec; struct iris_buffers_info buffers; u32 fw_min_count; + enum iris_inst_state state; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 83bbc6b..9bf79a0 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -69,3 +69,89 @@ int iris_change_core_state(struct iris_core *core, return ret; } + +struct iris_inst_state_change_allow { + enum iris_inst_state from; + enum iris_inst_state to; + enum state_change allow; +}; + +static enum state_change iris_allow_inst_state_change(struct iris_inst *inst, + enum iris_inst_state req_state) +{ + enum state_change allow = STATE_CHANGE_ALLOW; + int cnt; + static struct iris_inst_state_change_allow state[] = { + /* from, to, allow */ + {IRIS_INST_OPEN, IRIS_INST_OPEN, STATE_CHANGE_IGNORE }, + {IRIS_INST_OPEN, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_OPEN, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_OPEN, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_OPEN, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_OPEN, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_INPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_CLOSE, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE }, + {IRIS_INST_CLOSE, IRIS_INST_ERROR, STATE_CHANGE_IGNORE }, + + {IRIS_INST_ERROR, IRIS_INST_OPEN, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_ERROR, STATE_CHANGE_IGNORE }, + }; + + for (cnt = 0; cnt < ARRAY_SIZE(state); cnt++) { + if (state[cnt].from == inst->state && state[cnt].to == req_state) { + allow = state[cnt].allow; + break; + } + } + + return allow; +} + +int iris_inst_change_state(struct iris_inst *inst, + enum iris_inst_state request_state) +{ + enum state_change allow; + + if (IS_SESSION_ERROR(inst)) + return 0; + + if (inst->state == request_state) + return 0; + + allow = iris_allow_inst_state_change(inst, request_state); + if (allow != STATE_CHANGE_ALLOW) + return (allow == STATE_CHANGE_DISALLOW ? -EINVAL : 0); + + inst->state = request_state; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h index ee20842..6db95a1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -7,6 +7,7 @@ #define _IRIS_STATE_H_ struct iris_core; +struct iris_inst; enum iris_core_state { IRIS_CORE_DEINIT, @@ -15,8 +16,29 @@ enum iris_core_state { IRIS_CORE_ERROR, }; +enum iris_inst_state { + IRIS_INST_OPEN, + IRIS_INST_INPUT_STREAMING, + IRIS_INST_OUTPUT_STREAMING, + IRIS_INST_STREAMING, + IRIS_INST_CLOSE, + IRIS_INST_ERROR, +}; + +enum state_change { + STATE_CHANGE_ALLOW, + STATE_CHANGE_DISALLOW, + STATE_CHANGE_IGNORE, +}; + +#define IS_SESSION_ERROR(inst) \ +((inst)->state == IRIS_INST_ERROR) + bool core_in_valid_state(struct iris_core *core); int iris_change_core_state(struct iris_core *core, enum iris_core_state request_state); +int iris_inst_change_state(struct iris_inst *inst, + enum iris_inst_state request_state); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index a57b5fb..8f499a9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -34,6 +34,9 @@ int iris_vb2_queue_setup(struct vb2_queue *q, else f = inst->fmt_dst; + if (inst->state == IRIS_INST_STREAMING) + return -EINVAL; + if (*num_planes) { if (*num_planes != f->fmt.pix_mp.num_planes || sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 124333a..68ba75f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -168,6 +168,7 @@ int vidc_open(struct file *filp) inst->core = core; inst->session_id = hash32_ptr(inst); + iris_inst_change_state(inst, IRIS_INST_OPEN); mutex_init(&inst->ctx_q_lock); ret = vidc_add_session(inst); @@ -244,6 +245,7 @@ int vidc_close(struct file *filp) v4l2_ctrl_handler_free(&inst->ctrl_handler); vdec_inst_deinit(inst); close_session(inst); + iris_inst_change_state(inst, IRIS_INST_CLOSE); vidc_vb2_queue_deinit(inst); vidc_v4l2_fh_deinit(inst); vidc_remove_session(inst); @@ -295,6 +297,9 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt) if (!inst) return EPOLLERR; + if (IS_SESSION_ERROR(inst)) + return EPOLLERR; + poll_wait(filp, &inst->fh.wait, pt); poll_wait(filp, &inst->vb2q_src->done_wq, pt); poll_wait(filp, &inst->vb2q_dst->done_wq, pt); From patchwork Mon Dec 18 11:32:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755540 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B151E37D1F; Mon, 18 Dec 2023 11:40:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="QKcqsOTS" Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAtqox020050; Mon, 18 Dec 2023 11:40:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=5yHKjL9hcT4dzps/o4iYETybsLwHUpVhtq32vBP6USo=; b=QK cqsOTSZ302Uky6Q5w1SmfPJfkJzE2+mMlsaYcxYo7BvJOj3Gh4YeX4OknVJ/Zk38 suoDJn2ygSeMAkOMWIsrXl1+a46yemSlJNzbdvib9Y+YB+SuTbY3tFpdh8ygzN/g oFJsuncQe15vMxbdkUF63nzRAlb2r1WOBqWNs8QDIk8AN5A7uN/8hb9sQ/yC0OGb k6xqZFHJiDsSEX2+oeq6F4vexQpH8c/fn2Ig0BOYCl0p9kPsVrayN8liq4y4Hd/2 /nbV2KvNXUzaO0DyVd68F6sr47n2uuiowOZkRhsfMW/cZl1WA2kuCwKX2T6rf53A x17FdPdxsRk50+I8bmlA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v156dm7qn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:06 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Ka029944; Mon, 18 Dec 2023 11:40:03 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00af-5; Mon, 18 Dec 2023 11:40:03 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7Xd029998; Mon, 18 Dec 2023 11:36:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX62Z029968; Mon, 18 Dec 2023 11:36:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 583722342; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 23/34] media: iris: implement iris v4l2 ioctl ops supported by decoder Date: Mon, 18 Dec 2023 17:02:18 +0530 Message-Id: <1702899149-21321-24-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: Zph83nyaocpHd_tKYFiHWcRy7M7ufymH X-Proofpoint-GUID: Zph83nyaocpHd_tKYFiHWcRy7M7ufymH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 lowpriorityscore=0 bulkscore=0 impostorscore=0 mlxscore=0 spamscore=0 priorityscore=1501 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Implement all iris v4l2 ioctls ops supported by decoder. Add state checks to ensure ioctl are allowed in valid instance state only. Codec format can be changed by client during s_fmt. Update the v4l2 control values according to the updated codec format. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/iris_buffer.c | 23 + .../media/platform/qcom/vcodec/iris/iris_buffer.h | 1 + .../media/platform/qcom/vcodec/iris/iris_common.h | 2 + .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 48 +- .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 6 +- .../media/platform/qcom/vcodec/iris/iris_helpers.c | 212 +++++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 8 + .../platform/qcom/vcodec/iris/iris_instance.h | 4 + .../media/platform/qcom/vcodec/iris/iris_state.c | 38 ++ .../media/platform/qcom/vcodec/iris/iris_state.h | 6 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 17 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 281 ++++++++- .../media/platform/qcom/vcodec/iris/iris_vdec.h | 6 +- .../media/platform/qcom/vcodec/iris/iris_vidc.c | 632 ++++++++++++++++++++- .../platform/qcom/vcodec/iris/platform_common.h | 18 + .../platform/qcom/vcodec/iris/platform_sm8550.c | 42 ++ 16 files changed, 1323 insertions(+), 21 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c index 2a3989c..1ee840e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -55,6 +55,29 @@ static int output_min_count(struct iris_inst *inst) return output_min_count; } +int update_buffer_count(struct iris_inst *inst, u32 plane) +{ + switch (plane) { + case INPUT_MPLANE: + inst->buffers.input.min_count = input_min_count(inst); + if (inst->buffers.input.actual_count < inst->buffers.input.min_count) + inst->buffers.input.actual_count = inst->buffers.input.min_count; + + break; + case OUTPUT_MPLANE: + if (!inst->vb2q_src->streaming) + inst->buffers.output.min_count = output_min_count(inst); + if (inst->buffers.output.actual_count < inst->buffers.output.min_count) + inst->buffers.output.actual_count = inst->buffers.output.min_count; + + break; + default: + return -EINVAL; + } + + return 0; +} + static u32 internal_buffer_count(struct iris_inst *inst, enum iris_buffer_type buffer_type) { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h index ece894e..bdef15f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -34,6 +34,7 @@ struct iris_buffers_info { struct iris_buffers vpss; }; +int update_buffer_count(struct iris_inst *inst, u32 plane); int iris_get_buf_min_count(struct iris_inst *inst, enum iris_buffer_type buffer_type); int iris_get_buffer_size(struct iris_inst *inst, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index a83d1c1..6b771f8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -17,6 +17,8 @@ #define DEFAULT_HEIGHT 240 #define DEFAULT_BSE_VPP_DELAY 2 +#define MAX_EVENTS 30 + #define MB_IN_PIXEL (16 * 16) #define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c index 6b7aeaa..28977e8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -209,15 +209,15 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl) struct iris_inst *inst = NULL; int ret = 0; + inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler); switch (ctrl->id) { case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE: - ctrl->val = MIN_CAPTURE_BUFFERS; + ctrl->val = inst->buffers.output.min_count; break; case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT: - ctrl->val = MIN_OUTPUT_BUFFERS; + ctrl->val = inst->buffers.input.min_count; break; default: - inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler); cap_id = get_cap_id(inst, ctrl->id); if (is_valid_cap_id(cap_id)) ctrl->val = inst->cap[cap_id].value; @@ -232,12 +232,17 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) { struct cap_entry *entry = NULL, *temp = NULL; struct list_head children_list, firmware_list; + struct ctrl_data *priv_ctrl_data; enum plat_inst_cap_type cap_id; bool cap_present[INST_CAP_MAX]; struct plat_inst_cap *cap; struct iris_inst *inst; int ret = 0; + priv_ctrl_data = ctrl->priv ? ctrl->priv : NULL; + if (priv_ctrl_data && priv_ctrl_data->skip_s_ctrl) + return 0; + inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler); cap = &inst->cap[0]; @@ -249,6 +254,9 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) if (!is_valid_cap_id(cap_id)) return -EINVAL; + if (!allow_s_ctrl(inst, cap_id)) + return -EBUSY; + cap[cap_id].flags |= CAP_FLAG_CLIENT_SET; if (!inst->vb2q_src->streaming) { @@ -282,11 +290,12 @@ static const struct v4l2_ctrl_ops ctrl_ops = { .g_volatile_ctrl = vdec_op_g_volatile_ctrl, }; -int ctrls_init(struct iris_inst *inst) +int ctrls_init(struct iris_inst *inst, bool init) { int num_ctrls = 0, ctrl_idx = 0; struct plat_inst_cap *cap; struct iris_core *core; + u64 step_or_mask; int idx = 0; int ret = 0; @@ -300,10 +309,12 @@ int ctrls_init(struct iris_inst *inst) if (!num_ctrls) return -EINVAL; - ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, - INST_CAP_MAX * core->dec_codecs_count); - if (ret) - return ret; + if (init) { + ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, + INST_CAP_MAX * core->dec_codecs_count); + if (ret) + return ret; + } for (idx = 0; idx < INST_CAP_MAX; idx++) { struct v4l2_ctrl *ctrl; @@ -316,6 +327,27 @@ int ctrls_init(struct iris_inst *inst) goto error; } + if (!init) { + struct ctrl_data ctrl_priv_data; + + ctrl = v4l2_ctrl_find(&inst->ctrl_handler, cap[idx].v4l2_id); + if (ctrl) { + step_or_mask = (cap[idx].flags & CAP_FLAG_MENU) ? + ~(cap[idx].step_or_mask) : + cap[idx].step_or_mask; + memset(&ctrl_priv_data, 0, sizeof(ctrl_priv_data)); + ctrl_priv_data.skip_s_ctrl = true; + ctrl->priv = &ctrl_priv_data; + v4l2_ctrl_modify_range(ctrl, + cap[idx].min, + cap[idx].max, + step_or_mask, + cap[idx].value); + ctrl->priv = NULL; + continue; + } + } + if (cap[idx].flags & CAP_FLAG_MENU) { ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &ctrl_ops, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h index 0f67f4f..22ee6c4b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h @@ -13,6 +13,10 @@ struct cap_entry { enum plat_inst_cap_type cap_id; }; +struct ctrl_data { + bool skip_s_ctrl; +}; + int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id); int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id); int set_pipe(struct iris_inst *inst, enum plat_inst_cap_type cap_id); @@ -25,6 +29,6 @@ int iris_init_instance_caps(struct iris_core *core); int iris_init_core_caps(struct iris_core *core); int get_inst_capability(struct iris_inst *inst); int adjust_v4l2_properties(struct iris_inst *inst); -int ctrls_init(struct iris_inst *inst); +int ctrls_init(struct iris_inst *inst, bool init); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 335885f..ff44cda 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -79,6 +79,112 @@ bool is_split_mode_enabled(struct iris_inst *inst) return false; } +inline bool is_10bit_colorformat(enum colorformat_type colorformat) +{ + return colorformat == FMT_TP10C; +} + +inline bool is_8bit_colorformat(enum colorformat_type colorformat) +{ + return colorformat == FMT_NV12 || + colorformat == FMT_NV12C || + colorformat == FMT_NV21; +} + +u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec) +{ + const struct codec_info *codec_info; + struct iris_core *core; + u32 v4l2_codec = 0; + u32 i, size; + + core = inst->core; + codec_info = core->platform_data->format_data->codec_info; + size = core->platform_data->format_data->codec_info_size; + + for (i = 0; i < size; i++) { + if (codec_info[i].codec == codec) + return codec_info[i].v4l2_codec; + } + + return v4l2_codec; +} + +enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec) +{ + const struct codec_info *codec_info; + enum codec_type codec = 0; + struct iris_core *core; + u32 i, size; + + core = inst->core; + codec_info = core->platform_data->format_data->codec_info; + size = core->platform_data->format_data->codec_info_size; + + for (i = 0; i < size; i++) { + if (codec_info[i].v4l2_codec == v4l2_codec) + return codec_info[i].codec; + } + + return codec; +} + +u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat) +{ + const struct color_format_info *color_format_info; + u32 v4l2_colorformat = 0; + struct iris_core *core; + u32 i, size; + + core = inst->core; + color_format_info = core->platform_data->format_data->color_format_info; + size = core->platform_data->format_data->color_format_info_size; + + for (i = 0; i < size; i++) { + if (color_format_info[i].color_format == colorformat) + return color_format_info[i].v4l2_color_format; + } + + return v4l2_colorformat; +} + +enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat) +{ + const struct color_format_info *color_format_info; + enum colorformat_type colorformat = 0; + struct iris_core *core; + u32 i, size; + + core = inst->core; + color_format_info = core->platform_data->format_data->color_format_info; + size = core->platform_data->format_data->color_format_info_size; + + for (i = 0; i < size; i++) { + if (color_format_info[i].v4l2_color_format == v4l2_colorformat) + return color_format_info[i].color_format; + } + + return colorformat; +} + +struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type) +{ + struct vb2_queue *vb2q = NULL; + + switch (type) { + case INPUT_MPLANE: + vb2q = inst->vb2q_src; + break; + case OUTPUT_MPLANE: + vb2q = inst->vb2q_dst; + break; + default: + return NULL; + } + + return vb2q; +} + static int process_inst_timeout(struct iris_inst *inst) { struct iris_inst *instance; @@ -128,13 +234,119 @@ int close_session(struct iris_inst *inst) inst->packet = NULL; if (wait_for_response) { + mutex_unlock(&inst->lock); ret = wait_for_completion_timeout(&inst->completions[SIGNAL_CMD_CLOSE], msecs_to_jiffies(hw_response_timeout_val)); if (!ret) { ret = -ETIMEDOUT; process_inst_timeout(inst); } + mutex_lock(&inst->lock); } return ret; } + +static int check_core_mbps_mbpf(struct iris_inst *inst) +{ + u32 mbpf = 0, mbps = 0, total_mbpf = 0, total_mbps = 0; + struct iris_core *core; + struct iris_inst *instance; + u32 fps; + + core = inst->core; + + mutex_lock(&core->lock); + list_for_each_entry(instance, &core->instances, list) { + fps = inst->cap[QUEUED_RATE].value >> 16; + mbpf = get_mbpf(inst); + mbps = mbpf * fps; + total_mbpf += mbpf; + total_mbps += mbps; + } + mutex_unlock(&core->lock); + + if (total_mbps > core->cap[MAX_MBPS].value || + total_mbpf > core->cap[MAX_MBPF].value) + return -ENOMEM; + + return 0; +} + +static int check_inst_mbpf(struct iris_inst *inst) +{ + u32 mbpf = 0, max_mbpf = 0; + + max_mbpf = inst->cap[MBPF].max; + mbpf = get_mbpf(inst); + if (mbpf > max_mbpf) + return -ENOMEM; + + return 0; +} + +static int check_resolution_supported(struct iris_inst *inst) +{ + u32 width = 0, height = 0, min_width, min_height, + max_width, max_height; + + width = inst->fmt_src->fmt.pix_mp.width; + height = inst->fmt_src->fmt.pix_mp.height; + + min_width = inst->cap[FRAME_WIDTH].min; + max_width = inst->cap[FRAME_WIDTH].max; + min_height = inst->cap[FRAME_HEIGHT].min; + max_height = inst->cap[FRAME_HEIGHT].max; + + if (!(min_width <= width && width <= max_width) || + !(min_height <= height && height <= max_height)) + return -EINVAL; + + return 0; +} + +static int check_max_sessions(struct iris_inst *inst) +{ + struct iris_core *core; + u32 num_sessions = 0; + struct iris_inst *i; + + core = inst->core; + mutex_lock(&core->lock); + list_for_each_entry(i, &core->instances, list) { + num_sessions++; + } + mutex_unlock(&core->lock); + + if (num_sessions > core->cap[MAX_SESSION_COUNT].value) + return -ENOMEM; + + return 0; +} + +int check_session_supported(struct iris_inst *inst) +{ + int ret; + + ret = check_core_mbps_mbpf(inst); + if (ret) + goto exit; + + ret = check_inst_mbpf(inst); + if (ret) + goto exit; + + ret = check_resolution_supported(inst); + if (ret) + goto exit; + + ret = check_max_sessions(inst); + if (ret) + goto exit; + + return ret; +exit: + dev_err(inst->core->dev, "current session not supported(%d)\n", ret); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 9e85510..fe85d23 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -28,6 +28,14 @@ int get_mbpf(struct iris_inst *inst); int close_session(struct iris_inst *inst); bool is_linear_colorformat(u32 colorformat); +bool is_10bit_colorformat(enum colorformat_type colorformat); +bool is_8bit_colorformat(enum colorformat_type colorformat); bool is_split_mode_enabled(struct iris_inst *inst); +u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec); +enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec); +u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat); +enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat); +struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type); +int check_session_supported(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 275efa8..365f844 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -23,6 +23,7 @@ * @vb2q_src: source vb2 queue * @vb2q_dst: destination vb2 queue * @ctx_q_lock: lock to serialize queues related ioctls + * @lock: lock to seralise forward and reverse threads * @fh: reference of v4l2 file handler * @fmt_src: structure of v4l2_format for source * @fmt_dst: structure of v4l2_format for destination @@ -38,6 +39,7 @@ * @buffers: structure of buffer info * @fw_min_count: minimnum count of buffers needed by fw * @state: instance state + * @ipsc_properties_set: boolean to set ipsc properties to fw */ struct iris_inst { @@ -47,6 +49,7 @@ struct iris_inst { struct vb2_queue *vb2q_src; struct vb2_queue *vb2q_dst; struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */ + struct mutex lock; struct v4l2_fh fh; struct v4l2_format *fmt_src; struct v4l2_format *fmt_dst; @@ -62,6 +65,7 @@ struct iris_inst { struct iris_buffers_info buffers; u32 fw_min_count; enum iris_inst_state state; + bool ipsc_properties_set; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 9bf79a0..4cf6b69 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -155,3 +155,41 @@ int iris_inst_change_state(struct iris_inst *inst, return 0; } + +bool allow_s_fmt(struct iris_inst *inst, u32 type) +{ + return (inst->state == IRIS_INST_OPEN) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) || + (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING); +} + +bool allow_reqbufs(struct iris_inst *inst, u32 type) +{ + return (inst->state == IRIS_INST_OPEN) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) || + (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING); +} + +bool allow_streamon(struct iris_inst *inst, u32 type) +{ + return (type == INPUT_MPLANE && inst->state == IRIS_INST_OPEN) || + (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OPEN) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING); +} + +bool allow_streamoff(struct iris_inst *inst, u32 type) +{ + return (type == INPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) || + (type == INPUT_MPLANE && inst->state == IRIS_INST_STREAMING) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_STREAMING); +} + +bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id) +{ + return ((inst->state == IRIS_INST_OPEN) || + ((inst->cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED) && + (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_STREAMING))); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h index 6db95a1..35263e8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -41,4 +41,10 @@ int iris_change_core_state(struct iris_core *core, int iris_inst_change_state(struct iris_inst *inst, enum iris_inst_state request_state); +bool allow_s_fmt(struct iris_inst *inst, u32 type); +bool allow_reqbufs(struct iris_inst *inst, u32 type); +bool allow_streamon(struct iris_inst *inst, u32 type); +bool allow_streamoff(struct iris_inst *inst, u32 type); +bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index 8f499a9..66b5295 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -4,6 +4,7 @@ */ #include "iris_buffer.h" +#include "iris_ctrls.h" #include "iris_core.h" #include "iris_helpers.h" #include "iris_instance.h" @@ -47,6 +48,22 @@ int iris_vb2_queue_setup(struct vb2_queue *q, if (!buffer_type) return -EINVAL; + if (list_empty(&inst->caps_list)) { + ret = prepare_dependency_list(inst); + if (ret) + return ret; + } + + if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + ret = adjust_v4l2_properties(inst); + if (ret) + return ret; + } + + ret = check_session_supported(inst); + if (ret) + return ret; + ret = iris_free_buffers(inst, buffer_type); if (ret) return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 566048e..64067d5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -3,13 +3,51 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_buffer.h" #include "iris_common.h" +#include "iris_ctrls.h" +#include "iris_helpers.h" #include "iris_vdec.h" -void vdec_inst_init(struct iris_inst *inst) +static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec) +{ + bool session_init = false; + int ret; + + if (!inst->codec) + session_init = true; + + if (inst->codec && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec) + return 0; + + inst->codec = v4l2_codec_to_driver(inst, v4l2_codec); + if (!inst->codec) + return -EINVAL; + + inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec; + ret = get_inst_capability(inst); + if (ret) + return ret; + + ret = ctrls_init(inst, session_init); + if (ret) + return ret; + + ret = update_buffer_count(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = update_buffer_count(inst, OUTPUT_MPLANE); + + return ret; +} + +int vdec_inst_init(struct iris_inst *inst) { struct v4l2_format *f; + int ret; inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL); inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL); @@ -43,6 +81,10 @@ void vdec_inst_init(struct iris_inst *inst) inst->buffers.output.actual_count = inst->buffers.output.min_count; inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage; inst->fw_min_count = 0; + + ret = vdec_codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat); + + return ret; } void vdec_inst_deinit(struct iris_inst *inst) @@ -50,3 +92,240 @@ void vdec_inst_deinit(struct iris_inst *inst) kfree(inst->fmt_dst); kfree(inst->fmt_src); } + +static int vdec_check_colorformat_supported(struct iris_inst *inst, + enum colorformat_type colorformat) +{ + bool supported = true; + + if (!inst->vb2q_src->streaming) + return true; + + if (inst->cap[BIT_DEPTH].value == BIT_DEPTH_8 && + !is_8bit_colorformat(colorformat)) + supported = false; + if (inst->cap[BIT_DEPTH].value == BIT_DEPTH_10 && + !is_10bit_colorformat(colorformat)) + supported = false; + if (inst->cap[CODED_FRAMES].value == CODED_FRAMES_INTERLACE) + supported = false; + + return supported; +} + +int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f) +{ + struct iris_core *core; + u32 array[32] = {0}; + u32 i = 0; + + if (f->index >= ARRAY_SIZE(array)) + return -EINVAL; + + core = inst->core; + if (f->type == INPUT_MPLANE) { + u32 codecs = core->cap[DEC_CODECS].value; + u32 codecs_count = hweight32(codecs); + u32 idx = 0; + + for (i = 0; i <= codecs_count; i++) { + if (codecs & BIT(i)) { + if (idx >= ARRAY_SIZE(array)) + break; + array[idx] = codecs & BIT(i); + idx++; + } + } + if (!array[f->index]) + return -EINVAL; + f->pixelformat = v4l2_codec_from_driver(inst, array[f->index]); + f->flags = V4L2_FMT_FLAG_COMPRESSED; + strscpy(f->description, "codec", sizeof(f->description)); + } else if (f->type == OUTPUT_MPLANE) { + u32 formats = inst->cap[PIX_FMTS].step_or_mask; + u32 idx = 0; + + for (i = 0; i <= 31; i++) { + if (formats & BIT(i)) { + if (idx >= ARRAY_SIZE(array)) + break; + if (vdec_check_colorformat_supported(inst, formats & BIT(i))) { + array[idx] = formats & BIT(i); + idx++; + } + } + } + if (!array[f->index]) + return -EINVAL; + f->pixelformat = v4l2_colorformat_from_driver(inst, array[f->index]); + strscpy(f->description, "colorformat", sizeof(f->description)); + } + + if (!f->pixelformat) + return -EINVAL; + + memset(f->reserved, 0, sizeof(f->reserved)); + + return 0; +} + +int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; + struct v4l2_format *f_inst; + u32 pix_fmt; + + memset(pixmp->reserved, 0, sizeof(pixmp->reserved)); + if (f->type == INPUT_MPLANE) { + pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat); + if (!pix_fmt) { + f_inst = inst->fmt_src; + f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width; + f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height; + f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat; + pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat); + } + } else if (f->type == OUTPUT_MPLANE) { + pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat); + if (!pix_fmt) { + f_inst = inst->fmt_dst; + f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat; + f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width; + f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height; + } + if (inst->vb2q_src->streaming) { + f_inst = inst->fmt_src; + f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height; + f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width; + } + } else { + return -EINVAL; + } + + if (pixmp->field == V4L2_FIELD_ANY) + pixmp->field = V4L2_FIELD_NONE; + + pixmp->num_planes = 1; + + return 0; +} + +int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f) +{ + struct v4l2_format *fmt, *output_fmt; + enum colorformat_type colorformat; + u32 codec_align, stride = 0; + int ret = 0; + + vdec_try_fmt(inst, f); + + if (f->type == INPUT_MPLANE) { + if (inst->fmt_src->fmt.pix_mp.pixelformat != + f->fmt.pix_mp.pixelformat) { + ret = vdec_codec_change(inst, f->fmt.pix_mp.pixelformat); + if (ret) + return ret; + } + + fmt = inst->fmt_src; + fmt->type = INPUT_MPLANE; + + codec_align = inst->fmt_src->fmt.pix_mp.pixelformat == + V4L2_PIX_FMT_HEVC ? 32 : 16; + fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align); + fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align); + fmt->fmt.pix_mp.num_planes = 1; + fmt->fmt.pix_mp.plane_fmt[0].bytesperline = 0; + fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT); + inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT); + if (inst->buffers.input.actual_count < + inst->buffers.input.min_count) { + inst->buffers.input.actual_count = + inst->buffers.input.min_count; + } + inst->buffers.input.size = + fmt->fmt.pix_mp.plane_fmt[0].sizeimage; + + fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace; + fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func; + fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + + output_fmt = inst->fmt_dst; + output_fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace; + output_fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func; + output_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + output_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + + inst->crop.left = 0; + inst->crop.top = 0; + inst->crop.width = f->fmt.pix_mp.width; + inst->crop.height = f->fmt.pix_mp.height; + } else if (f->type == OUTPUT_MPLANE) { + fmt = inst->fmt_dst; + fmt->type = OUTPUT_MPLANE; + if (inst->vb2q_src->streaming) { + f->fmt.pix_mp.height = inst->fmt_src->fmt.pix_mp.height; + f->fmt.pix_mp.width = inst->fmt_src->fmt.pix_mp.width; + } + fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat; + codec_align = f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C ? 192 : 128; + fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align); + codec_align = f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C ? 16 : 32; + fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align); + fmt->fmt.pix_mp.num_planes = 1; + if (f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C) { + stride = ALIGN(f->fmt.pix_mp.width, 192); + fmt->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(stride * 4 / 3, 256); + } else { + fmt->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(f->fmt.pix_mp.width, 128); + } + fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); + + if (!inst->vb2q_src->streaming) + inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT); + if (inst->buffers.output.actual_count < + inst->buffers.output.min_count) { + inst->buffers.output.actual_count = + inst->buffers.output.min_count; + } + + colorformat = v4l2_colorformat_to_driver(inst, fmt->fmt.pix_mp.pixelformat); + inst->buffers.output.size = + fmt->fmt.pix_mp.plane_fmt[0].sizeimage; + inst->cap[PIX_FMTS].value = colorformat; + + if (!inst->vb2q_src->streaming) { + inst->crop.top = 0; + inst->crop.left = 0; + inst->crop.width = f->fmt.pix_mp.width; + inst->crop.height = f->fmt.pix_mp.height; + } + } else { + return -EINVAL; + } + memcpy(f, fmt, sizeof(*fmt)); + + return ret; +} + +int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub) +{ + int ret = 0; + + switch (sub->type) { + case V4L2_EVENT_EOS: + ret = v4l2_event_subscribe(&inst->fh, sub, MAX_EVENTS, NULL); + break; + case V4L2_EVENT_SOURCE_CHANGE: + ret = v4l2_src_change_event_subscribe(&inst->fh, sub); + break; + case V4L2_EVENT_CTRL: + ret = v4l2_ctrl_subscribe_event(&inst->fh, sub); + break; + default: + return -EINVAL; + } + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index dc8f43f..a2f159d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -8,7 +8,11 @@ #include "iris_instance.h" -void vdec_inst_init(struct iris_inst *inst); +int vdec_inst_init(struct iris_inst *inst); void vdec_inst_deinit(struct iris_inst *inst); +int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); +int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); +int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); +int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 68ba75f..5c76821 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -3,6 +3,11 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include +#include +#include + +#include "iris_buffer.h" #include "iris_common.h" #include "iris_helpers.h" #include "iris_hfi.h" @@ -12,6 +17,9 @@ #include "iris_ctrls.h" #include "iris_vb2.h" +#define VIDC_DRV_NAME "iris_driver" +#define VIDC_BUS_NAME "platform:iris_bus" + static int vidc_v4l2_fh_init(struct iris_inst *inst) { struct iris_core *core; @@ -169,6 +177,7 @@ int vidc_open(struct file *filp) inst->core = core; inst->session_id = hash32_ptr(inst); iris_inst_change_state(inst, IRIS_INST_OPEN); + mutex_init(&inst->lock); mutex_init(&inst->ctx_q_lock); ret = vidc_add_session(inst); @@ -200,14 +209,6 @@ int vidc_open(struct file *filp) if (ret) goto fail_inst_deinit; - ret = get_inst_capability(inst); - if (ret) - goto fail_queue_deinit; - - ret = ctrls_init(inst); - if (ret) - goto fail_queue_deinit; - ret = iris_hfi_session_open(inst); if (ret) { dev_err(core->dev, "%s: session open failed\n", __func__); @@ -220,7 +221,6 @@ int vidc_open(struct file *filp) fail_core_deinit: v4l2_ctrl_handler_free(&inst->ctrl_handler); iris_core_deinit(core); -fail_queue_deinit: vidc_vb2_queue_deinit(inst); fail_inst_deinit: vdec_inst_deinit(inst); @@ -229,6 +229,7 @@ int vidc_open(struct file *filp) vidc_remove_session(inst); fail_free_inst: mutex_destroy(&inst->ctx_q_lock); + mutex_destroy(&inst->lock); kfree(inst); return ret; @@ -244,14 +245,16 @@ int vidc_close(struct file *filp) v4l2_ctrl_handler_free(&inst->ctrl_handler); vdec_inst_deinit(inst); + mutex_lock(&inst->lock); close_session(inst); iris_inst_change_state(inst, IRIS_INST_CLOSE); vidc_vb2_queue_deinit(inst); vidc_v4l2_fh_deinit(inst); vidc_remove_session(inst); + mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock); + mutex_destroy(&inst->lock); kfree(inst); - filp->private_data = NULL; return 0; @@ -313,6 +316,588 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt) return poll; } +static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f) +{ + struct iris_inst *inst; + int ret; + + inst = get_vidc_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + ret = vdec_enum_fmt(inst, f); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f) +{ + struct iris_inst *inst; + int ret; + + inst = get_vidc_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (!allow_s_fmt(inst, f->type)) { + ret = -EBUSY; + goto unlock; + } + + ret = vdec_try_fmt(inst, f); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f) +{ + struct iris_inst *inst; + int ret; + + inst = get_vidc_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (!allow_s_fmt(inst, f->type)) { + ret = -EBUSY; + goto unlock; + } + + ret = vdec_s_fmt(inst, f); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_g_fmt(struct file *filp, void *fh, struct v4l2_format *f) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (f->type == INPUT_MPLANE) + memcpy(f, inst->fmt_src, sizeof(*f)); + else if (f->type == OUTPUT_MPLANE) + memcpy(f, inst->fmt_dst, sizeof(*f)); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_enum_framesizes(struct file *filp, void *fh, + struct v4l2_frmsizeenum *fsize) +{ + enum colorformat_type colorfmt; + struct iris_inst *inst; + enum codec_type codec; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !fsize) + return -EINVAL; + + if (fsize->index) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + codec = v4l2_codec_to_driver(inst, fsize->pixel_format); + if (!codec) { + colorfmt = v4l2_colorformat_to_driver(inst, fsize->pixel_format); + if (colorfmt == FMT_NONE) { + ret = -EINVAL; + goto unlock; + } + } + + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; + fsize->stepwise.min_width = inst->cap[FRAME_WIDTH].min; + fsize->stepwise.max_width = inst->cap[FRAME_WIDTH].max; + fsize->stepwise.step_width = inst->cap[FRAME_WIDTH].step_or_mask; + fsize->stepwise.min_height = inst->cap[FRAME_HEIGHT].min; + fsize->stepwise.max_height = inst->cap[FRAME_HEIGHT].max; + fsize->stepwise.step_height = inst->cap[FRAME_HEIGHT].step_or_mask; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_reqbufs(struct file *filp, void *fh, struct v4l2_requestbuffers *b) +{ + struct vb2_queue *vb2q = NULL; + struct iris_inst *inst; + int ret; + + inst = get_vidc_inst(filp, fh); + if (!inst || !b) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (!allow_reqbufs(inst, b->type)) { + ret = -EBUSY; + goto unlock; + } + + vb2q = get_vb2q(inst, b->type); + if (!vb2q) { + ret = -EINVAL; + goto unlock; + } + + ret = vb2_reqbufs(vb2q, b); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_querybuf(struct file *filp, void *fh, struct v4l2_buffer *b) +{ + struct vb2_queue *vb2q = NULL; + struct iris_inst *inst; + int ret; + + inst = get_vidc_inst(filp, fh); + if (!inst || !b) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + vb2q = get_vb2q(inst, b->type); + if (!vb2q) { + ret = -EINVAL; + goto unlock; + } + + ret = vb2_querybuf(vb2q, b); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_create_bufs(struct file *filp, void *fh, struct v4l2_create_buffers *b) +{ + struct iris_inst *inst; + struct vb2_queue *vb2q; + struct v4l2_format *f; + int ret; + + inst = get_vidc_inst(filp, fh); + if (!inst || !b) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + f = &b->format; + vb2q = get_vb2q(inst, f->type); + if (!vb2q) { + ret = -EINVAL; + goto unlock; + } + + ret = vb2_create_bufs(vb2q, b); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_prepare_buf(struct file *filp, void *fh, struct v4l2_buffer *b) +{ + struct video_device *vdev; + struct iris_inst *inst; + struct vb2_queue *vb2q; + int ret; + + inst = get_vidc_inst(filp, fh); + vdev = video_devdata(filp); + if (!inst || !vdev || !vdev->v4l2_dev->mdev) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + vb2q = get_vb2q(inst, b->type); + if (!vb2q) { + ret = -EINVAL; + goto unlock; + } + + ret = vb2_prepare_buf(vb2q, vdev->v4l2_dev->mdev, b); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_qbuf(struct file *filp, void *fh, struct v4l2_buffer *b) +{ + struct video_device *vdev; + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + vdev = video_devdata(filp); + if (!inst || !b) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (b->type == INPUT_MPLANE) + ret = vb2_qbuf(inst->vb2q_src, vdev->v4l2_dev->mdev, b); + else if (b->type == OUTPUT_MPLANE) + ret = vb2_qbuf(inst->vb2q_dst, vdev->v4l2_dev->mdev, b); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_dqbuf(struct file *filp, void *fh, struct v4l2_buffer *b) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !b) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EINVAL; + goto unlock; + } + + if (b->type == INPUT_MPLANE) + ret = vb2_dqbuf(inst->vb2q_src, b, true); + else if (b->type == OUTPUT_MPLANE) + ret = vb2_dqbuf(inst->vb2q_dst, b, true); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_streamon(struct file *filp, void *fh, enum v4l2_buf_type type) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EINVAL; + goto unlock; + } + + if (!allow_streamon(inst, type)) { + ret = -EBUSY; + goto unlock; + } + + if (type == INPUT_MPLANE) + ret = vb2_streamon(inst->vb2q_src, type); + else if (type == OUTPUT_MPLANE) + ret = vb2_streamon(inst->vb2q_dst, type); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_streamoff(struct file *filp, void *fh, enum v4l2_buf_type type) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EINVAL; + goto unlock; + } + + if (!allow_streamoff(inst, type)) { + ret = -EBUSY; + goto unlock; + } + + if (type == INPUT_MPLANE) + ret = vb2_streamoff(inst->vb2q_src, type); + else if (type == OUTPUT_MPLANE) + ret = vb2_streamoff(inst->vb2q_dst, type); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_querycap(struct file *filp, void *fh, struct v4l2_capability *cap) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + strscpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver)); + strscpy(cap->bus_info, VIDC_BUS_NAME, sizeof(cap->bus_info)); + memset(cap->reserved, 0, sizeof(cap->reserved)); + strscpy(cap->card, "iris_decoder", sizeof(cap->card)); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_queryctrl(struct file *filp, void *fh, struct v4l2_queryctrl *q_ctrl) +{ + struct v4l2_ctrl *ctrl; + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !q_ctrl) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + ctrl = v4l2_ctrl_find(&inst->ctrl_handler, q_ctrl->id); + if (!ctrl) { + ret = -EINVAL; + goto unlock; + } + + q_ctrl->minimum = ctrl->minimum; + q_ctrl->maximum = ctrl->maximum; + q_ctrl->default_value = ctrl->default_value; + q_ctrl->flags = 0; + q_ctrl->step = ctrl->step; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qmenu) +{ + struct v4l2_ctrl *ctrl; + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !qmenu) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + ctrl = v4l2_ctrl_find(&inst->ctrl_handler, qmenu->id); + if (!ctrl) { + ret = -EINVAL; + goto unlock; + } + + if (ctrl->type != V4L2_CTRL_TYPE_MENU) { + ret = -EINVAL; + goto unlock; + } + + if (qmenu->index < ctrl->minimum || qmenu->index > ctrl->maximum) { + ret = -EINVAL; + goto unlock; + } + + if (ctrl->menu_skip_mask & (1 << qmenu->index)) { + ret = -EINVAL; + goto unlock; + } + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub) +{ + struct iris_inst *inst; + int ret; + + inst = container_of(fh, struct iris_inst, fh); + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + ret = vdec_subscribe_event(inst, sub); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_unsubscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub) +{ + struct iris_inst *inst; + int ret; + + inst = container_of(fh, struct iris_inst, fh); + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + ret = v4l2_event_unsubscribe(&inst->fh, sub); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection *s) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !s) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) { + ret = -EINVAL; + goto unlock; + } + + switch (s->target) { + case V4L2_SEL_TGT_CROP_BOUNDS: + case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP: + case V4L2_SEL_TGT_COMPOSE_BOUNDS: + case V4L2_SEL_TGT_COMPOSE_PADDED: + case V4L2_SEL_TGT_COMPOSE_DEFAULT: + case V4L2_SEL_TGT_COMPOSE: + s->r.left = inst->crop.left; + s->r.top = inst->crop.top; + s->r.width = inst->crop.width; + s->r.height = inst->crop.height; + break; + default: + ret = -EINVAL; + } + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static const struct v4l2_file_operations v4l2_file_ops = { .owner = THIS_MODULE, .open = vidc_open, @@ -335,11 +920,38 @@ static struct vb2_mem_ops iris_vb2_mem_ops = { .unmap_dmabuf = iris_vb2_unmap_dmabuf, }; +static const struct v4l2_ioctl_ops v4l2_ioctl_ops = { + .vidioc_enum_fmt_vid_cap = vidc_enum_fmt, + .vidioc_enum_fmt_vid_out = vidc_enum_fmt, + .vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt, + .vidioc_try_fmt_vid_out_mplane = vidc_try_fmt, + .vidioc_s_fmt_vid_cap_mplane = vidc_s_fmt, + .vidioc_s_fmt_vid_out_mplane = vidc_s_fmt, + .vidioc_g_fmt_vid_cap_mplane = vidc_g_fmt, + .vidioc_g_fmt_vid_out_mplane = vidc_g_fmt, + .vidioc_enum_framesizes = vidc_enum_framesizes, + .vidioc_reqbufs = vidc_reqbufs, + .vidioc_querybuf = vidc_querybuf, + .vidioc_create_bufs = vidc_create_bufs, + .vidioc_prepare_buf = vidc_prepare_buf, + .vidioc_qbuf = vidc_qbuf, + .vidioc_dqbuf = vidc_dqbuf, + .vidioc_streamon = vidc_streamon, + .vidioc_streamoff = vidc_streamoff, + .vidioc_querycap = vidc_querycap, + .vidioc_queryctrl = vidc_queryctrl, + .vidioc_querymenu = vidc_querymenu, + .vidioc_subscribe_event = vidc_subscribe_event, + .vidioc_unsubscribe_event = vidc_unsubscribe_event, + .vidioc_g_selection = vidc_g_selection, +}; + int init_ops(struct iris_core *core) { core->v4l2_file_ops = &v4l2_file_ops; core->vb2_ops = &iris_vb2_ops; core->vb2_mem_ops = &iris_vb2_mem_ops; + core->v4l2_ioctl_ops = &v4l2_ioctl_ops; return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index e242614..abd11fa 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -166,6 +166,23 @@ struct plat_inst_caps { struct plat_inst_cap cap[INST_CAP_MAX + 1]; }; +struct codec_info { + u32 v4l2_codec; + enum codec_type codec; +}; + +struct color_format_info { + u32 v4l2_color_format; + enum colorformat_type color_format; +}; + +struct format_capability { + struct codec_info *codec_info; + u32 codec_info_size; + struct color_format_info *color_format_info; + u32 color_format_info_size; +}; + struct platform_data { const struct bus_info *bus_tbl; unsigned int bus_tbl_size; @@ -182,6 +199,7 @@ struct platform_data { const struct reg_preset_info *reg_prst_tbl; unsigned int reg_prst_tbl_size; struct ubwc_config_data *ubwc_config; + struct format_capability *format_data; const char *fwname; u32 pas_id; struct plat_core_cap *core_data; diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 0759ac5..85bc677 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -19,6 +19,40 @@ #define MINIMUM_FPS 1 #define MAXIMUM_FPS 480 +static struct codec_info codec_data_sm8550[] = { + { + .v4l2_codec = V4L2_PIX_FMT_H264, + .codec = H264, + }, + { + .v4l2_codec = V4L2_PIX_FMT_HEVC, + .codec = HEVC, + }, + { + .v4l2_codec = V4L2_PIX_FMT_VP9, + .codec = VP9, + }, +}; + +static struct color_format_info color_format_data_sm8550[] = { + { + .v4l2_color_format = V4L2_PIX_FMT_NV12, + .color_format = FMT_NV12, + }, + { + .v4l2_color_format = V4L2_PIX_FMT_NV21, + .color_format = FMT_NV21, + }, + { + .v4l2_color_format = V4L2_PIX_FMT_QC08C, + .color_format = FMT_NV12C, + }, + { + .v4l2_color_format = V4L2_PIX_FMT_QC10C, + .color_format = FMT_TP10C, + }, +}; + static struct plat_core_cap core_data_sm8550[] = { {DEC_CODECS, H264 | HEVC | VP9}, {MAX_SESSION_COUNT, 16}, @@ -341,6 +375,13 @@ static struct ubwc_config_data ubwc_config_sm8550[] = { UBWC_CONFIG(8, 32, 16, 0, 1, 1, 1), }; +static struct format_capability format_data_sm8550 = { + .codec_info = codec_data_sm8550, + .codec_info_size = ARRAY_SIZE(codec_data_sm8550), + .color_format_info = color_format_data_sm8550, + .color_format_info_size = ARRAY_SIZE(color_format_data_sm8550), +}; + struct platform_data sm8550_data = { .bus_tbl = sm8550_bus_table, .bus_tbl_size = ARRAY_SIZE(sm8550_bus_table), @@ -367,4 +408,5 @@ struct platform_data sm8550_data = { .inst_cap_data = instance_cap_data_sm8550, .inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550), .ubwc_config = ubwc_config_sm8550, + .format_data = &format_data_sm8550, }; From patchwork Mon Dec 18 11:32:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755549 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DC261B272; Mon, 18 Dec 2023 11:36:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="o7WcZUgd" Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBDcoC018040; Mon, 18 Dec 2023 11:36:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Q03KspE6e+LrgwjDhfZcLcqLHb5nu3RQ3Bq6dmATH3A=; b=o7 WcZUgdsmECrQYua8jt+o82o2lOwgf5OxfMCF/Nt+3MlAxc0e3TbELuHnjLuFsOba 9d4DJnICVVvwRD8M/E+xDW0W206PQ+GQYZMzQrqbfKp0/CaW9XC93mhLTN3EeoDB qntx/IktP86E5kZj6+9W6A3Y3JyB+JjLJbmeplx5JUeC7GTIM1PuXsN8WMqSRDG/ irb0Th6zn/6zqSCIGLU8R1QEvoqflRBbAN3vMrcmSsU9ET7wE7Izw9BdK8oQb+WD hByaLabj7+SmlHF0enS8S76v+pPTk9xUq8fU2kH37cpLKitxVmD8Yt34p419mXh/ aJwLSMUaHE2YN5wHD1MQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2n1781gj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:13 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5R2029941; Mon, 18 Dec 2023 11:36:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyva-1; Mon, 18 Dec 2023 11:36:09 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7pT030023; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8SW030061; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 5A85B2346; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 24/34] media: iris: subscribe input and output properties to firmware Date: Mon, 18 Dec 2023 17:02:19 +0530 Message-Id: <1702899149-21321-25-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: YOXUHpHcWpeRvUFeQefdpbGqd7L_sRey X-Proofpoint-GUID: YOXUHpHcWpeRvUFeQefdpbGqd7L_sRey X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 adultscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 malwarescore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Driver can subscribe to different properties for which it expects a response from firmware. Different SOCs firmware can support different properties which can be subscribed by driver. HFI_CMD_SUBSCRIBE_MODE with HFI_MODE_PROPERTY - to subscribe any property to firmware. Also, set below mandatory properties on capture plane: HFI_PROP_COLOR_FORMAT HFI_PROP_LINEAR_STRIDE_SCANLINE HFI_PROP_UBWC_STRIDE_SCANLINE. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 31 +++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 38 +++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 42 ++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 3 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 154 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 2 + .../platform/qcom/vcodec/iris/platform_common.h | 8 ++ .../platform/qcom/vcodec/iris/platform_sm8550.c | 29 ++++ 9 files changed, 310 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index ce4eaff..a6078a5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -25,6 +25,29 @@ #define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008 #define HFI_PROP_UBWC_BANK_SPREADING 0x03000009 +enum hfi_property_mode_type { + HFI_MODE_NONE = 0x00000000, + HFI_MODE_PORT_SETTINGS_CHANGE = 0x00000001, + HFI_MODE_PROPERTY = 0x00000002, +}; + +#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B + +enum hfi_color_format { + HFI_COLOR_FMT_OPAQUE = 0, + HFI_COLOR_FMT_NV12 = 1, + HFI_COLOR_FMT_NV12_UBWC = 2, + HFI_COLOR_FMT_P010 = 3, + HFI_COLOR_FMT_TP10_UBWC = 4, + HFI_COLOR_FMT_RGBA8888 = 5, + HFI_COLOR_FMT_RGBA8888_UBWC = 6, + HFI_COLOR_FMT_NV21 = 7, +}; + +#define HFI_PROP_COLOR_FORMAT 0x03000101 + +#define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104 + #define HFI_PROP_PROFILE 0x03000107 #define HFI_PROP_LEVEL 0x03000108 @@ -49,10 +72,18 @@ #define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b +#define HFI_PROP_PICTURE_TYPE 0x03000162 + #define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 +#define HFI_PROP_NO_OUTPUT 0x0300016a + +#define HFI_PROP_DPB_LIST 0x0300017A + +#define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190 + #define HFI_PROP_COMV_BUFFER_COUNT 0x03000193 #define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 8f1e456..d15ce5a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -219,6 +219,44 @@ int iris_hfi_session_open(struct iris_inst *inst) return ret; } +int iris_hfi_session_subscribe_mode(struct iris_inst *inst, + u32 cmd, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + struct iris_core *core; + int ret; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + ret = hfi_packet_session_command(inst, + cmd, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + get_hfi_port(plane), + inst->session_id, + payload_type, + payload, + payload_size); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + int iris_hfi_session_close(struct iris_inst *inst) { struct iris_core *core; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index ca2339e..d6b3fca 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -13,6 +13,9 @@ int iris_hfi_core_init(struct iris_core *core); int iris_hfi_core_deinit(struct iris_core *core); int iris_hfi_session_open(struct iris_inst *inst); int iris_hfi_session_close(struct iris_inst *inst); +int iris_hfi_session_subscribe_mode(struct iris_inst *inst, + u32 cmd, u32 plane, u32 payload_type, + void *payload, u32 payload_size); int iris_hfi_set_property(struct iris_inst *inst, u32 packet_type, u32 flag, u32 plane, u32 payload_type, void *payload, u32 payload_size); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index a3544d8..d4cdfcf 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -34,6 +34,24 @@ u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type) return hfi_port; } +u32 get_hfi_port(u32 plane) +{ + u32 hfi_port = HFI_PORT_NONE; + + switch (plane) { + case INPUT_MPLANE: + hfi_port = HFI_PORT_BITSTREAM; + break; + case OUTPUT_MPLANE: + hfi_port = HFI_PORT_RAW; + break; + default: + break; + } + + return hfi_port; +} + static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) { switch (buffer_type) { @@ -58,6 +76,30 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) } } +u32 get_hfi_colorformat(u32 colorformat) +{ + u32 hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC; + + switch (colorformat) { + case V4L2_PIX_FMT_NV12: + hfi_colorformat = HFI_COLOR_FMT_NV12; + break; + case V4L2_PIX_FMT_QC08C: + hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC; + break; + case V4L2_PIX_FMT_QC10C: + hfi_colorformat = HFI_COLOR_FMT_TP10_UBWC; + break; + case V4L2_PIX_FMT_NV21: + hfi_colorformat = HFI_COLOR_FMT_NV21; + break; + default: + break; + } + + return hfi_colorformat; +} + int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) { memset(buf, 0, sizeof(*buf)); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index bf18553..4276d6d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -81,6 +81,9 @@ enum hfi_packet_port_type { u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type); int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); +u32 get_hfi_colorformat(u32 colorformat); +u32 get_hfi_port(u32 plane); + int hfi_packet_sys_init(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_image_version(struct iris_core *core, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 64067d5..7d16c96 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -5,10 +5,13 @@ #include +#include "hfi_defines.h" #include "iris_buffer.h" #include "iris_common.h" #include "iris_ctrls.h" #include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_hfi_packet.h" #include "iris_vdec.h" static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec) @@ -329,3 +332,154 @@ int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscri return ret; } + +int vdec_subscribe_property(struct iris_inst *inst, u32 plane) +{ + const u32 *subcribe_prop = NULL; + u32 subscribe_prop_size = 0; + struct iris_core *core; + u32 payload[32] = {0}; + u32 i; + + core = inst->core; + + payload[0] = HFI_MODE_PROPERTY; + + if (plane == INPUT_MPLANE) { + subscribe_prop_size = core->platform_data->dec_input_prop_size; + subcribe_prop = core->platform_data->dec_input_prop; + } else if (plane == OUTPUT_MPLANE) { + if (inst->codec == H264) { + subscribe_prop_size = core->platform_data->dec_output_prop_size_avc; + subcribe_prop = core->platform_data->dec_output_prop_avc; + } else if (inst->codec == HEVC) { + subscribe_prop_size = core->platform_data->dec_output_prop_size_hevc; + subcribe_prop = core->platform_data->dec_output_prop_hevc; + } else if (inst->codec == VP9) { + subscribe_prop_size = core->platform_data->dec_output_prop_size_vp9; + subcribe_prop = core->platform_data->dec_output_prop_vp9; + } else { + return -EINVAL; + } + } else { + return -EINVAL; + } + + for (i = 0; i < subscribe_prop_size; i++) + payload[i + 1] = subcribe_prop[i]; + + return iris_hfi_session_subscribe_mode(inst, + HFI_CMD_SUBSCRIBE_MODE, + plane, + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + (subscribe_prop_size + 1) * sizeof(u32)); +} + +static int vdec_set_colorformat(struct iris_inst *inst) +{ + u32 hfi_colorformat; + u32 pixelformat; + + pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + hfi_colorformat = get_hfi_colorformat(pixelformat); + + return iris_hfi_set_property(inst, + HFI_PROP_COLOR_FORMAT, + HFI_HOST_FLAGS_NONE, + get_hfi_port(OUTPUT_MPLANE), + HFI_PAYLOAD_U32, + &hfi_colorformat, + sizeof(u32)); +} + +static int vdec_set_linear_stride_scanline(struct iris_inst *inst) +{ + u32 stride_y, scanline_y, stride_uv, scanline_uv; + u32 pixelformat; + u32 payload[2]; + + pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + + if (!is_linear_colorformat(pixelformat)) + return 0; + + stride_y = inst->fmt_dst->fmt.pix_mp.width; + scanline_y = inst->fmt_dst->fmt.pix_mp.height; + stride_uv = stride_y; + scanline_uv = scanline_y / 2; + + payload[0] = stride_y << 16 | scanline_y; + payload[1] = stride_uv << 16 | scanline_uv; + + return iris_hfi_set_property(inst, + HFI_PROP_LINEAR_STRIDE_SCANLINE, + HFI_HOST_FLAGS_NONE, + get_hfi_port(OUTPUT_MPLANE), + HFI_PAYLOAD_U64, + &payload, + sizeof(u64)); +} + +static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst) +{ + u32 meta_stride_y, meta_scanline_y, meta_stride_uv, meta_scanline_uv; + u32 stride_y, scanline_y, stride_uv, scanline_uv; + u32 pix_fmt, width, height; + u32 payload[4]; + + pix_fmt = inst->fmt_dst->fmt.pix_mp.pixelformat; + width = inst->fmt_dst->fmt.pix_mp.width; + height = inst->fmt_dst->fmt.pix_mp.height; + + if (is_linear_colorformat(pix_fmt)) + return 0; + + if (pix_fmt == V4L2_PIX_FMT_QC08C) { + stride_y = ALIGN(width, 128); + scanline_y = ALIGN(height, 32); + stride_uv = ALIGN(width, 128); + scanline_uv = ALIGN((height + 1) >> 1, 32); + meta_stride_y = ALIGN(DIV_ROUND_UP(width, 32), 64); + meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 8), 16); + meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64); + meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16); + } else { + stride_y = ALIGN(ALIGN(width, 192) * 4 / 3, 256); + scanline_y = ALIGN(height, 16); + stride_uv = ALIGN(ALIGN(width, 192) * 4 / 3, 256); + scanline_uv = ALIGN((height + 1) >> 1, 16); + meta_stride_y = ALIGN(DIV_ROUND_UP(width, 48), 64); + meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 4), 16); + meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64); + meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16); + } + + payload[0] = stride_y << 16 | scanline_y; + payload[1] = stride_uv << 16 | scanline_uv; + payload[2] = meta_stride_y << 16 | meta_scanline_y; + payload[3] = meta_stride_uv << 16 | meta_scanline_uv; + + return iris_hfi_set_property(inst, + HFI_PROP_UBWC_STRIDE_SCANLINE, + HFI_HOST_FLAGS_NONE, + get_hfi_port(OUTPUT_MPLANE), + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + sizeof(u32) * 4); +} + +int vdec_set_output_property(struct iris_inst *inst) +{ + int ret; + + ret = vdec_set_colorformat(inst); + if (ret) + return ret; + + ret = vdec_set_linear_stride_scanline(inst); + if (ret) + return ret; + + return vdec_set_ubwc_stride_scanline(inst); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index a2f159d..6b0306c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -14,5 +14,7 @@ int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); +int vdec_subscribe_property(struct iris_inst *inst, u32 plane); +int vdec_set_output_property(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index abd11fa..fc12bde 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -206,6 +206,14 @@ struct platform_data { u32 core_data_size; struct plat_inst_cap *inst_cap_data; u32 inst_cap_data_size; + const u32 *dec_input_prop; + unsigned int dec_input_prop_size; + const u32 *dec_output_prop_avc; + unsigned int dec_output_prop_size_avc; + const u32 *dec_output_prop_hevc; + unsigned int dec_output_prop_size_hevc; + const u32 *dec_output_prop_vp9; + unsigned int dec_output_prop_size_vp9; }; int init_platform(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 85bc677..6a4bfa3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -382,6 +382,26 @@ static struct format_capability format_data_sm8550 = { .color_format_info_size = ARRAY_SIZE(color_format_data_sm8550), }; +static const u32 sm8550_vdec_input_properties[] = { + HFI_PROP_NO_OUTPUT, +}; + +static const u32 sm8550_vdec_output_properties_avc[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_DPB_LIST, + HFI_PROP_CABAC_SESSION, +}; + +static const u32 sm8550_vdec_output_properties_hevc[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_DPB_LIST, +}; + +static const u32 sm8550_vdec_output_properties_vp9[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_DPB_LIST, +}; + struct platform_data sm8550_data = { .bus_tbl = sm8550_bus_table, .bus_tbl_size = ARRAY_SIZE(sm8550_bus_table), @@ -409,4 +429,13 @@ struct platform_data sm8550_data = { .inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550), .ubwc_config = ubwc_config_sm8550, .format_data = &format_data_sm8550, + + .dec_input_prop = sm8550_vdec_input_properties, + .dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_input_properties), + .dec_output_prop_avc = sm8550_vdec_output_properties_avc, + .dec_output_prop_size_avc = ARRAY_SIZE(sm8550_vdec_output_properties_avc), + .dec_output_prop_hevc = sm8550_vdec_output_properties_hevc, + .dec_output_prop_size_hevc = ARRAY_SIZE(sm8550_vdec_output_properties_hevc), + .dec_output_prop_vp9 = sm8550_vdec_output_properties_vp9, + .dec_output_prop_size_vp9 = ARRAY_SIZE(sm8550_vdec_output_properties_vp9), }; From patchwork Mon Dec 18 11:32:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755543 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16BBD3B187; Mon, 18 Dec 2023 11:40:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="h7zOB8Js" Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBM4IU021485; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=lMKp0y/4t5au0RD/mod2lpTCZDH1CTSHKwpmYwCU3Ao=; b=h7 zOB8JsDFx1QjH1V3H7xjK8usXoh0toOPR37q2hD74GrwkF4qdDLtOIrF92CVElhO 2XBJmJmksjwWGXfw/rE3cJWQNTUQMP6uoCdhZbgkEsDSgXojFIE4OBefb7beCbmD 2SuLqwsMDZFqLGT0lvwqnZWPNKpsKN2GBz+AQd7VF01J3Vz+glUef54s9Kt4ORgz xOAsVafqehQ4zK//bjorxcTB72slCDgVxoTrMFaPXyz+HL9dbddzdnc3Az9yEy+c /e2PbTDsAhfTGkX3R9fqQg0bID4Aistfefe2lwWRRf+yRyNUWOeMStb1tk8giUA3 iqO+hj65XSbR47jKnVnQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2ghdgnby-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX6PN029956; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ad-3; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7Xf029998; Mon, 18 Dec 2023 11:36:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6Fm029971; Mon, 18 Dec 2023 11:36:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 5D1182348; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 25/34] media: iris: subscribe src change and handle firmware responses Date: Mon, 18 Dec 2023 17:02:20 +0530 Message-Id: <1702899149-21321-26-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: ZU2fDNi4bfxReJzlVZjn67Azg-RhH70l X-Proofpoint-ORIG-GUID: ZU2fDNi4bfxReJzlVZjn67Azg-RhH70l X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 impostorscore=0 phishscore=0 clxscore=1015 adultscore=0 suspectscore=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Driver can subscribe to different bitstream parameters for which it expects a response from firmware. These are vital bitstream parameters which requires reconfiguration of buffers or sending updated info to client. Incase of any change in these subscribed parameters, firmware sends a response, which is then sent to client as source change event. HFI_CMD_SUBSCRIBE_MODE with HFI_MODE_PORT_SETTINGS_CHANGE - to subscribe for different bitstream parameters. Below properties are set to firmware as part of this: HFI_PROP_BITSTREAM_RESOLUTION HFI_PROP_CROP_OFFSETS HFI_PROP_LUMA_CHROMA_BIT_DEPTH HFI_PROP_CODED_FRAMES HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT HFI_PROP_PIC_ORDER_CNT_TYPE HFI_PROP_SIGNAL_COLOR_INFO HFI_PROP_PROFILE HFI_PROP_LEVEL HFI_PROP_TIER Add the handling of different responses from firmware. Below are different types of responses: System responses: - System error - response in case of system error occurred. - System property - response for system level property - sys image version. - System init - response for video core init completion. Session responses: - Session error - response in case of any session level error. - Session property - response for all session properties. HFI_PROP_BITSTREAM_RESOLUTION HFI_PROP_CROP_OFFSETS HFI_PROP_LUMA_CHROMA_BIT_DEPTH HFI_PROP_CODED_FRAMES HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT HFI_PROP_PIC_ORDER_CNT_TYPE HFI_PROP_SIGNAL_COLOR_INFO HFI_PROP_PROFILE HFI_PROP_LEVEL HFI_PROP_TIER - Session command - response for session level commands. HFI_CMD_OPEN - response for open competition. HFI_CMD_CLOSE - response for open competition. HFI_CMD_SETTINGS_CHANGE - response for change in subscribe parameters. HFI_CMD_SUBSCRIBE_MODE - response for info on subscribed properties. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 114 +++- .../media/platform/qcom/vcodec/iris/iris_common.h | 25 +- .../media/platform/qcom/vcodec/iris/iris_core.h | 2 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 29 +- .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 195 +++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 11 +- .../platform/qcom/vcodec/iris/iris_hfi_response.c | 364 ++++++++++++- .../platform/qcom/vcodec/iris/iris_instance.h | 8 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 580 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 4 + .../platform/qcom/vcodec/iris/platform_common.h | 6 + .../platform/qcom/vcodec/iris/platform_sm8550.c | 44 ++ 13 files changed, 1375 insertions(+), 11 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index a6078a5..0ef6bad 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -10,6 +10,7 @@ #define HFI_VIDEO_ARCH_LX 0x1 +#define HFI_CMD_BEGIN 0x01000000 #define HFI_CMD_INIT 0x01000001 #define HFI_CMD_OPEN 0x01000003 #define HFI_CMD_CLOSE 0x01000004 @@ -31,7 +32,18 @@ enum hfi_property_mode_type { HFI_MODE_PROPERTY = 0x00000002, }; -#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B +#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B +#define HFI_CMD_SETTINGS_CHANGE 0x0100000C +#define HFI_CMD_END 0x01FFFFFF + +#define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000 +#define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff + +#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 + +#define HFI_PROP_BEGIN 0x03000000 + +#define HFI_PROP_IMAGE_VERSION 0x03000001 enum hfi_color_format { HFI_COLOR_FMT_OPAQUE = 0, @@ -46,8 +58,12 @@ enum hfi_color_format { #define HFI_PROP_COLOR_FORMAT 0x03000101 +#define HFI_PROP_BITSTREAM_RESOLUTION 0x03000103 + #define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104 +#define HFI_PROP_CROP_OFFSETS 0x03000105 + #define HFI_PROP_PROFILE 0x03000107 #define HFI_PROP_LEVEL 0x03000108 @@ -68,13 +84,83 @@ enum hfi_color_format { #define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123 +#define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124 + #define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128 +#define HFI_PROP_QUALITY_MODE 0x03000148 + +enum hfi_color_primaries { + HFI_PRIMARIES_RESERVED = 0, + HFI_PRIMARIES_BT709 = 1, + HFI_PRIMARIES_UNSPECIFIED = 2, + HFI_PRIMARIES_BT470_SYSTEM_M = 4, + HFI_PRIMARIES_BT470_SYSTEM_BG = 5, + HFI_PRIMARIES_BT601_525 = 6, + HFI_PRIMARIES_SMPTE_ST240M = 7, + HFI_PRIMARIES_GENERIC_FILM = 8, + HFI_PRIMARIES_BT2020 = 9, + HFI_PRIMARIES_SMPTE_ST428_1 = 10, + HFI_PRIMARIES_SMPTE_RP431_2 = 11, + HFI_PRIMARIES_SMPTE_EG431_1 = 12, + HFI_PRIMARIES_SMPTE_EBU_TECH = 22, +}; + +enum hfi_transfer_characteristics { + HFI_TRANSFER_RESERVED = 0, + HFI_TRANSFER_BT709 = 1, + HFI_TRANSFER_UNSPECIFIED = 2, + HFI_TRANSFER_BT470_SYSTEM_M = 4, + HFI_TRANSFER_BT470_SYSTEM_BG = 5, + HFI_TRANSFER_BT601_525_OR_625 = 6, + HFI_TRANSFER_SMPTE_ST240M = 7, + HFI_TRANSFER_LINEAR = 8, + HFI_TRANSFER_LOG_100_1 = 9, + HFI_TRANSFER_LOG_SQRT = 10, + HFI_TRANSFER_XVYCC = 11, + HFI_TRANSFER_BT1361_0 = 12, + HFI_TRANSFER_SRGB_SYCC = 13, + HFI_TRANSFER_BT2020_14 = 14, + HFI_TRANSFER_BT2020_15 = 15, + HFI_TRANSFER_SMPTE_ST2084_PQ = 16, + HFI_TRANSFER_SMPTE_ST428_1 = 17, + HFI_TRANSFER_BT2100_2_HLG = 18, +}; + +enum hfi_matrix_coefficients { + HFI_MATRIX_COEFF_SRGB_SMPTE_ST428_1 = 0, + HFI_MATRIX_COEFF_BT709 = 1, + HFI_MATRIX_COEFF_UNSPECIFIED = 2, + HFI_MATRIX_COEFF_RESERVED = 3, + HFI_MATRIX_COEFF_FCC_TITLE_47 = 4, + HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625 = 5, + HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625 = 6, + HFI_MATRIX_COEFF_SMPTE_ST240 = 7, + HFI_MATRIX_COEFF_YCGCO = 8, + HFI_MATRIX_COEFF_BT2020_NON_CONSTANT = 9, + HFI_MATRIX_COEFF_BT2020_CONSTANT = 10, + HFI_MATRIX_COEFF_SMPTE_ST2085 = 11, + HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_NON_CONSTANT = 12, + HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_CONSTANT = 13, + HFI_MATRIX_COEFF_BT2100 = 14, +}; + +#define HFI_PROP_SIGNAL_COLOR_INFO 0x03000155 + #define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b +enum hfi_picture_type { + HFI_PICTURE_IDR = 0x00000001, + HFI_PICTURE_P = 0x00000002, + HFI_PICTURE_B = 0x00000004, + HFI_PICTURE_I = 0x00000008, + HFI_PICTURE_CRA = 0x00000010, + HFI_PICTURE_BLA = 0x00000020, +}; + #define HFI_PROP_PICTURE_TYPE 0x03000162 -#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 +#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 @@ -86,7 +172,29 @@ enum hfi_color_format { #define HFI_PROP_COMV_BUFFER_COUNT 0x03000193 -#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 +#define HFI_PROP_END 0x03FFFFFF + +#define HFI_SESSION_ERROR_BEGIN 0x04000000 + +#define HFI_ERROR_UNKNOWN_SESSION 0x04000001 + +#define HFI_ERROR_MAX_SESSIONS 0x04000002 + +#define HFI_ERROR_FATAL 0x04000003 + +#define HFI_ERROR_INVALID_STATE 0x04000004 + +#define HFI_ERROR_INSUFFICIENT_RESOURCES 0x04000005 + +#define HFI_ERROR_BUFFER_NOT_SET 0x04000006 + +#define HFI_SESSION_ERROR_END 0x04FFFFFF + +#define HFI_SYSTEM_ERROR_BEGIN 0x05000000 + +#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 + +#define HFI_SYSTEM_ERROR_END 0x05FFFFFF struct hfi_debug_header { u32 size; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 6b771f8..0fbd139 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -8,7 +8,6 @@ #include #include #include -#include #include #define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE @@ -16,6 +15,7 @@ #define DEFAULT_WIDTH 320 #define DEFAULT_HEIGHT 240 #define DEFAULT_BSE_VPP_DELAY 2 +#define IRIS_VERSION_LENGTH 128 #define MAX_EVENTS 30 @@ -23,6 +23,9 @@ #define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) +#define MAX_DPB_LIST_ARRAY_SIZE (16 * 4) +#define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4) + enum codec_type { H264 = BIT(0), HEVC = BIT(1), @@ -95,4 +98,24 @@ struct iris_buffer { struct dma_buf_attachment *attach; }; +struct subscription_params { + u32 bitstream_resolution; + u32 crop_offsets[2]; + u32 bit_depth; + u32 coded_frames; + u32 fw_min_count; + u32 pic_order_cnt; + u32 color_info; + u32 profile; + u32 level; + u32 tier; +}; + +struct iris_hfi_frame_info { + u32 picture_type; + u32 no_output; + u32 data_corrupt; + u32 overflow; +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 154991c..6452ea7 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -57,6 +57,7 @@ * @inst_caps: a pointer to supported instance capabilities * @instances: a list_head of all instances * @intr_status: interrupt status + * @fw_version: firmware version */ struct iris_core { @@ -99,6 +100,7 @@ struct iris_core { struct plat_inst_caps *inst_caps; struct list_head instances; u32 intr_status; + char fw_version[IRIS_VERSION_LENGTH]; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index ff44cda..4cad673 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include "hfi_defines.h" +#include "iris_core.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" @@ -66,7 +68,7 @@ int get_mbpf(struct iris_inst *inst) return NUM_MBS_PER_FRAME(height, width); } -bool is_linear_colorformat(u32 colorformat) +inline bool is_linear_colorformat(u32 colorformat) { return colorformat == V4L2_PIX_FMT_NV12 || colorformat == V4L2_PIX_FMT_NV21; } @@ -350,3 +352,28 @@ int check_session_supported(struct iris_inst *inst) return ret; } + +int signal_session_msg_receipt(struct iris_inst *inst, + enum signal_session_response cmd) +{ + if (cmd < MAX_SIGNAL) + complete(&inst->completions[cmd]); + + return 0; +} + +struct iris_inst *to_instance(struct iris_core *core, u32 session_id) +{ + struct iris_inst *inst = NULL; + + mutex_lock(&core->lock); + list_for_each_entry(inst, &core->instances, list) { + if (inst->session_id == session_id) { + mutex_unlock(&core->lock); + return inst; + } + } + mutex_unlock(&core->lock); + + return NULL; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index fe85d23..cb22adf 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -31,6 +31,10 @@ bool is_linear_colorformat(u32 colorformat); bool is_10bit_colorformat(enum colorformat_type colorformat); bool is_8bit_colorformat(enum colorformat_type colorformat); bool is_split_mode_enabled(struct iris_inst *inst); +int signal_session_msg_receipt(struct iris_inst *inst, + enum signal_session_response cmd); +struct iris_inst *to_instance(struct iris_core *core, u32 session_id); + u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec); enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec); u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index d4cdfcf..dc7157d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -100,6 +100,201 @@ u32 get_hfi_colorformat(u32 colorformat) return hfi_colorformat; } +u32 get_hfi_color_primaries(u32 primaries) +{ + u32 hfi_primaries = HFI_PRIMARIES_RESERVED; + + switch (primaries) { + case V4L2_COLORSPACE_DEFAULT: + hfi_primaries = HFI_PRIMARIES_RESERVED; + break; + case V4L2_COLORSPACE_REC709: + hfi_primaries = HFI_PRIMARIES_BT709; + break; + case V4L2_COLORSPACE_470_SYSTEM_M: + hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_M; + break; + case V4L2_COLORSPACE_470_SYSTEM_BG: + hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_BG; + break; + case V4L2_COLORSPACE_SMPTE170M: + hfi_primaries = HFI_PRIMARIES_BT601_525; + break; + case V4L2_COLORSPACE_SMPTE240M: + hfi_primaries = HFI_PRIMARIES_SMPTE_ST240M; + break; + case V4L2_COLORSPACE_BT2020: + hfi_primaries = HFI_PRIMARIES_BT2020; + break; + case V4L2_COLORSPACE_DCI_P3: + hfi_primaries = HFI_PRIMARIES_SMPTE_RP431_2; + break; + default: + break; + } + + return hfi_primaries; +} + +u32 get_hfi_transfer_char(u32 characterstics) +{ + u32 hfi_characterstics = HFI_TRANSFER_RESERVED; + + switch (characterstics) { + case V4L2_XFER_FUNC_DEFAULT: + hfi_characterstics = HFI_TRANSFER_RESERVED; + break; + case V4L2_XFER_FUNC_709: + hfi_characterstics = HFI_TRANSFER_BT709; + break; + case V4L2_XFER_FUNC_SMPTE240M: + hfi_characterstics = HFI_TRANSFER_SMPTE_ST240M; + break; + case V4L2_XFER_FUNC_SRGB: + hfi_characterstics = HFI_TRANSFER_SRGB_SYCC; + break; + case V4L2_XFER_FUNC_SMPTE2084: + hfi_characterstics = HFI_TRANSFER_SMPTE_ST2084_PQ; + break; + default: + break; + } + + return hfi_characterstics; +} + +u32 get_hfi_matrix_coefficients(u32 coefficients) +{ + u32 hfi_coefficients = HFI_MATRIX_COEFF_RESERVED; + + switch (coefficients) { + case V4L2_YCBCR_ENC_DEFAULT: + hfi_coefficients = HFI_MATRIX_COEFF_RESERVED; + break; + case V4L2_YCBCR_ENC_709: + hfi_coefficients = HFI_MATRIX_COEFF_BT709; + break; + case V4L2_YCBCR_ENC_XV709: + hfi_coefficients = HFI_MATRIX_COEFF_BT709; + break; + case V4L2_YCBCR_ENC_XV601: + hfi_coefficients = HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625; + break; + case V4L2_YCBCR_ENC_601: + hfi_coefficients = HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625; + break; + case V4L2_YCBCR_ENC_SMPTE240M: + hfi_coefficients = HFI_MATRIX_COEFF_SMPTE_ST240; + break; + case V4L2_YCBCR_ENC_BT2020: + hfi_coefficients = HFI_MATRIX_COEFF_BT2020_NON_CONSTANT; + break; + case V4L2_YCBCR_ENC_BT2020_CONST_LUM: + hfi_coefficients = HFI_MATRIX_COEFF_BT2020_CONSTANT; + break; + default: + break; + } + + return hfi_coefficients; +} + +u32 get_v4l2_color_primaries(u32 hfi_primaries) +{ + u32 primaries = V4L2_COLORSPACE_DEFAULT; + + switch (hfi_primaries) { + case HFI_PRIMARIES_RESERVED: + primaries = V4L2_COLORSPACE_DEFAULT; + break; + case HFI_PRIMARIES_BT709: + primaries = V4L2_COLORSPACE_REC709; + break; + case HFI_PRIMARIES_BT470_SYSTEM_M: + primaries = V4L2_COLORSPACE_470_SYSTEM_M; + break; + case HFI_PRIMARIES_BT470_SYSTEM_BG: + primaries = V4L2_COLORSPACE_470_SYSTEM_BG; + break; + case HFI_PRIMARIES_BT601_525: + primaries = V4L2_COLORSPACE_SMPTE170M; + break; + case HFI_PRIMARIES_SMPTE_ST240M: + primaries = V4L2_COLORSPACE_SMPTE240M; + break; + case HFI_PRIMARIES_BT2020: + primaries = V4L2_COLORSPACE_BT2020; + break; + case V4L2_COLORSPACE_DCI_P3: + primaries = HFI_PRIMARIES_SMPTE_RP431_2; + break; + default: + break; + } + + return primaries; +} + +u32 get_v4l2_transfer_char(u32 hfi_characterstics) +{ + u32 characterstics = V4L2_XFER_FUNC_DEFAULT; + + switch (hfi_characterstics) { + case HFI_TRANSFER_RESERVED: + characterstics = V4L2_XFER_FUNC_DEFAULT; + break; + case HFI_TRANSFER_BT709: + characterstics = V4L2_XFER_FUNC_709; + break; + case HFI_TRANSFER_SMPTE_ST240M: + characterstics = V4L2_XFER_FUNC_SMPTE240M; + break; + case HFI_TRANSFER_SRGB_SYCC: + characterstics = V4L2_XFER_FUNC_SRGB; + break; + case HFI_TRANSFER_SMPTE_ST2084_PQ: + characterstics = V4L2_XFER_FUNC_SMPTE2084; + break; + default: + break; + } + + return characterstics; +} + +u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients) +{ + u32 coefficients = V4L2_YCBCR_ENC_DEFAULT; + + switch (hfi_coefficients) { + case HFI_MATRIX_COEFF_RESERVED: + coefficients = V4L2_YCBCR_ENC_DEFAULT; + break; + case HFI_MATRIX_COEFF_BT709: + coefficients = V4L2_YCBCR_ENC_709; + break; + case HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625: + coefficients = V4L2_YCBCR_ENC_XV601; + break; + case HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625: + coefficients = V4L2_YCBCR_ENC_601; + break; + case HFI_MATRIX_COEFF_SMPTE_ST240: + coefficients = V4L2_YCBCR_ENC_SMPTE240M; + break; + case HFI_MATRIX_COEFF_BT2020_NON_CONSTANT: + coefficients = V4L2_YCBCR_ENC_BT2020; + break; + case HFI_MATRIX_COEFF_BT2020_CONSTANT: + coefficients = V4L2_YCBCR_ENC_BT2020_CONST_LUM; + break; + default: + break; + } + + return coefficients; +} + int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) { memset(buf, 0, sizeof(*buf)); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index 4276d6d..f813116 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -80,10 +80,17 @@ enum hfi_packet_port_type { }; u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type); -int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); -u32 get_hfi_colorformat(u32 colorformat); u32 get_hfi_port(u32 plane); +u32 get_hfi_colorformat(u32 colorformat); +u32 get_hfi_color_primaries(u32 primaries); +u32 get_hfi_transfer_char(u32 characterstics); +u32 get_hfi_matrix_coefficients(u32 coefficients); +u32 get_v4l2_color_primaries(u32 hfi_primaries); +u32 get_v4l2_transfer_char(u32 hfi_characterstics); +u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients); +int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); + int hfi_packet_sys_init(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_image_version(struct iris_core *core, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index 829f3f6..4ca9314 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -4,8 +4,27 @@ */ #include "hfi_defines.h" +#include "iris_helpers.h" #include "iris_hfi_packet.h" #include "iris_hfi_response.h" +#include "iris_vdec.h" + +struct iris_core_hfi_range { + u32 begin; + u32 end; + int (*handle)(struct iris_core *core, struct hfi_packet *pkt); +}; + +struct iris_inst_hfi_range { + u32 begin; + u32 end; + int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt); +}; + +struct iris_hfi_packet_handle { + enum hfi_buffer_type type; + int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt); +}; static void print_sfr_message(struct iris_core *core) { @@ -25,8 +44,7 @@ static void print_sfr_message(struct iris_core *core) } } -static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, - u32 core_resp_pkt_size) +static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, u32 core_resp_pkt_size) { u32 response_pkt_size = 0; u8 *response_limit; @@ -52,8 +70,7 @@ static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, return 0; } -static int validate_hdr_packet(struct iris_core *core, - struct hfi_header *hdr) +static int validate_hdr_packet(struct iris_core *core, struct hfi_header *hdr) { struct hfi_packet *packet; int i, ret = 0; @@ -76,6 +93,45 @@ static int validate_hdr_packet(struct iris_core *core, return ret; } +static int handle_session_error(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + struct iris_core *core; + char *error; + + core = inst->core; + + switch (pkt->type) { + case HFI_ERROR_MAX_SESSIONS: + error = "exceeded max sessions"; + break; + case HFI_ERROR_UNKNOWN_SESSION: + error = "unknown session id"; + break; + case HFI_ERROR_INVALID_STATE: + error = "invalid operation for current state"; + break; + case HFI_ERROR_INSUFFICIENT_RESOURCES: + error = "insufficient resources"; + break; + case HFI_ERROR_BUFFER_NOT_SET: + error = "internal buffers not set"; + break; + case HFI_ERROR_FATAL: + error = "fatal error"; + break; + default: + error = "unknown"; + break; + } + + dev_err(core->dev, "session error received %#x: %s\n", + pkt->type, error); + iris_inst_change_state(inst, IRIS_INST_ERROR); + + return 0; +} + static int handle_system_error(struct iris_core *core, struct hfi_packet *pkt) { @@ -86,6 +142,301 @@ static int handle_system_error(struct iris_core *core, return 0; } +static int handle_system_init(struct iris_core *core, + struct hfi_packet *pkt) +{ + if (!(pkt->flags & HFI_FW_FLAGS_SUCCESS)) + return 0; + + mutex_lock(&core->lock); + if (pkt->packet_id != core->sys_init_id) + goto unlock; + + iris_change_core_state(core, IRIS_CORE_INIT); + +unlock: + mutex_unlock(&core->lock); + + return 0; +} + +static int handle_session_close(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + signal_session_msg_receipt(inst, SIGNAL_CMD_CLOSE); + + return 0; +} + +static int handle_src_change(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + int ret = 0; + + if (pkt->port == HFI_PORT_BITSTREAM) + ret = vdec_src_change(inst); + else if (pkt->port == HFI_PORT_RAW) + ret = 0; + + if (ret) + iris_inst_change_state(inst, IRIS_INST_ERROR); + + return ret; +} + +static int handle_session_command(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + int i, ret = 0; + static const struct iris_hfi_packet_handle hfi_pkt_handle[] = { + {HFI_CMD_OPEN, NULL }, + {HFI_CMD_CLOSE, handle_session_close }, + {HFI_CMD_SETTINGS_CHANGE, handle_src_change }, + {HFI_CMD_SUBSCRIBE_MODE, NULL }, + }; + + for (i = 0; i < ARRAY_SIZE(hfi_pkt_handle); i++) { + if (hfi_pkt_handle[i].type == pkt->type) { + if (hfi_pkt_handle[i].handle) { + ret = hfi_pkt_handle[i].handle(inst, pkt); + if (ret) + return ret; + } + break; + } + } + + if (i == ARRAY_SIZE(hfi_pkt_handle)) + return -EINVAL; + + return ret; +} + +static int handle_dpb_list_property(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + u8 *payload_start; + u32 payload_size; + + payload_size = pkt->size - sizeof(*pkt); + payload_start = (u8 *)((u8 *)pkt + sizeof(*pkt)); + memset(inst->dpb_list_payload, 0, MAX_DPB_LIST_ARRAY_SIZE); + + if (payload_size > MAX_DPB_LIST_PAYLOAD_SIZE) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + return -EINVAL; + } + + memcpy(inst->dpb_list_payload, payload_start, payload_size); + + return 0; +} + +static int handle_session_property(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + u32 *payload_ptr = NULL; + int ret = 0; + + if (pkt->port != HFI_PORT_BITSTREAM) + return 0; + + if (pkt->flags & HFI_FW_FLAGS_INFORMATION) + return 0; + + payload_ptr = (u32 *)((u8 *)pkt + sizeof(*pkt)); + if (!payload_ptr) + return -EINVAL; + + switch (pkt->type) { + case HFI_PROP_BITSTREAM_RESOLUTION: + inst->src_subcr_params.bitstream_resolution = payload_ptr[0]; + break; + case HFI_PROP_CROP_OFFSETS: + inst->src_subcr_params.crop_offsets[0] = payload_ptr[0]; + inst->src_subcr_params.crop_offsets[1] = payload_ptr[1]; + break; + case HFI_PROP_LUMA_CHROMA_BIT_DEPTH: + inst->src_subcr_params.bit_depth = payload_ptr[0]; + break; + case HFI_PROP_CODED_FRAMES: + inst->src_subcr_params.coded_frames = payload_ptr[0]; + break; + case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT: + inst->src_subcr_params.fw_min_count = payload_ptr[0]; + break; + case HFI_PROP_PIC_ORDER_CNT_TYPE: + inst->src_subcr_params.pic_order_cnt = payload_ptr[0]; + break; + case HFI_PROP_SIGNAL_COLOR_INFO: + inst->src_subcr_params.color_info = payload_ptr[0]; + break; + case HFI_PROP_PROFILE: + inst->src_subcr_params.profile = payload_ptr[0]; + break; + case HFI_PROP_LEVEL: + inst->src_subcr_params.level = payload_ptr[0]; + break; + case HFI_PROP_TIER: + inst->src_subcr_params.tier = payload_ptr[0]; + break; + case HFI_PROP_PICTURE_TYPE: + inst->hfi_frame_info.picture_type = payload_ptr[0]; + break; + case HFI_PROP_DPB_LIST: + ret = handle_dpb_list_property(inst, pkt); + if (ret) + break; + break; + case HFI_PROP_NO_OUTPUT: + inst->hfi_frame_info.no_output = 1; + break; + case HFI_PROP_QUALITY_MODE: + case HFI_PROP_STAGE: + case HFI_PROP_PIPE: + break; + default: + break; + } + + return ret; +} + +static int handle_image_version_property(struct iris_core *core, + struct hfi_packet *pkt) +{ + u8 *str_image_version; + u32 req_bytes; + u32 i = 0; + + req_bytes = pkt->size - sizeof(*pkt); + if (req_bytes < IRIS_VERSION_LENGTH - 1) + return -EINVAL; + + str_image_version = (u8 *)pkt + sizeof(*pkt); + + for (i = 0; i < IRIS_VERSION_LENGTH - 1; i++) { + if (str_image_version[i] != '\0') + core->fw_version[i] = str_image_version[i]; + else + core->fw_version[i] = ' '; + } + core->fw_version[i] = '\0'; + + return 0; +} + +static int handle_system_property(struct iris_core *core, + struct hfi_packet *pkt) +{ + int ret = 0; + + switch (pkt->type) { + case HFI_PROP_IMAGE_VERSION: + ret = handle_image_version_property(core, pkt); + break; + default: + break; + } + + return ret; +} + +static int handle_system_response(struct iris_core *core, + struct hfi_header *hdr) +{ + struct hfi_packet *packet; + u8 *pkt, *start_pkt; + int ret = 0; + int i, j; + static const struct iris_core_hfi_range be[] = { + {HFI_SYSTEM_ERROR_BEGIN, HFI_SYSTEM_ERROR_END, handle_system_error }, + {HFI_PROP_BEGIN, HFI_PROP_END, handle_system_property }, + {HFI_CMD_BEGIN, HFI_CMD_END, handle_system_init }, + }; + + start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + for (i = 0; i < ARRAY_SIZE(be); i++) { + pkt = start_pkt; + for (j = 0; j < hdr->num_packets; j++) { + packet = (struct hfi_packet *)pkt; + if (packet->flags & HFI_FW_FLAGS_SYSTEM_ERROR) { + ret = handle_system_error(core, packet); + return ret; + } + + if (packet->type > be[i].begin && packet->type < be[i].end) { + ret = be[i].handle(core, packet); + if (ret) + return ret; + + if (packet->type > HFI_SYSTEM_ERROR_BEGIN && + packet->type < HFI_SYSTEM_ERROR_END) + return 0; + } + pkt += packet->size; + } + } + + return ret; +} + +static int handle_session_response(struct iris_core *core, + struct hfi_header *hdr) +{ + struct hfi_packet *packet; + struct iris_inst *inst; + u8 *pkt, *start_pkt; + int ret = 0; + int i, j; + static const struct iris_inst_hfi_range be[] = { + {HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, handle_session_error }, + {HFI_PROP_BEGIN, HFI_PROP_END, handle_session_property }, + {HFI_CMD_BEGIN, HFI_CMD_END, handle_session_command }, + }; + + inst = to_instance(core, hdr->session_id); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info)); + + pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + for (i = 0; i < hdr->num_packets; i++) { + packet = (struct hfi_packet *)pkt; + if (packet->type == HFI_CMD_SETTINGS_CHANGE) { + if (packet->port == HFI_PORT_BITSTREAM) { + vdec_init_src_change_param(inst); + break; + } + } + pkt += packet->size; + } + + start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + for (i = 0; i < ARRAY_SIZE(be); i++) { + pkt = start_pkt; + for (j = 0; j < hdr->num_packets; j++) { + packet = (struct hfi_packet *)pkt; + if (packet->flags & HFI_FW_FLAGS_SESSION_ERROR) + handle_session_error(inst, packet); + + if (packet->type > be[i].begin && packet->type < be[i].end) { + ret = be[i].handle(inst, packet); + if (ret) + iris_inst_change_state(inst, IRIS_INST_ERROR); + } + pkt += packet->size; + } + } + + memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info)); + mutex_unlock(&inst->lock); + + return ret; +} + static int handle_response(struct iris_core *core, void *response) { struct hfi_header *hdr; @@ -96,6 +447,11 @@ static int handle_response(struct iris_core *core, void *response) if (ret) return handle_system_error(core, NULL); + if (!hdr->session_id) + return handle_system_response(core, hdr); + else + return handle_session_response(core, hdr); + return ret; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 365f844..4f51d68 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -40,6 +40,10 @@ * @fw_min_count: minimnum count of buffers needed by fw * @state: instance state * @ipsc_properties_set: boolean to set ipsc properties to fw + * @hfi_frame_info: structure of frame info + * @src_subcr_params: subscription params to fw on input port + * @dst_subcr_params: subscription params to fw on output port + * @dpb_list_payload: array of dpb buffers */ struct iris_inst { @@ -66,6 +70,10 @@ struct iris_inst { u32 fw_min_count; enum iris_inst_state state; bool ipsc_properties_set; + struct iris_hfi_frame_info hfi_frame_info; + struct subscription_params src_subcr_params; + struct subscription_params dst_subcr_params; + u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE]; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 7d16c96..ac47fc0 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -14,6 +14,13 @@ #include "iris_hfi_packet.h" #include "iris_vdec.h" +#define UNSPECIFIED_COLOR_FORMAT 5 + +struct vdec_prop_type_handle { + u32 type; + int (*handle)(struct iris_inst *inst); +}; + static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec) { bool session_init = false; @@ -376,6 +383,455 @@ int vdec_subscribe_property(struct iris_inst *inst, u32 plane) (subscribe_prop_size + 1) * sizeof(u32)); } +static int vdec_set_bitstream_resolution(struct iris_inst *inst) +{ + u32 resolution; + + resolution = inst->fmt_src->fmt.pix_mp.width << 16 | + inst->fmt_src->fmt.pix_mp.height; + inst->src_subcr_params.bitstream_resolution = resolution; + + return iris_hfi_set_property(inst, + HFI_PROP_BITSTREAM_RESOLUTION, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32, + &resolution, + sizeof(u32)); +} + +static int vdec_set_crop_offsets(struct iris_inst *inst) +{ + u32 left_offset, top_offset, right_offset, bottom_offset; + u32 payload[2] = {0}; + + left_offset = inst->crop.left; + top_offset = inst->crop.top; + right_offset = (inst->fmt_src->fmt.pix_mp.width - + inst->crop.width); + bottom_offset = (inst->fmt_src->fmt.pix_mp.height - + inst->crop.height); + + payload[0] = left_offset << 16 | top_offset; + payload[1] = right_offset << 16 | bottom_offset; + inst->src_subcr_params.crop_offsets[0] = payload[0]; + inst->src_subcr_params.crop_offsets[1] = payload[1]; + + return iris_hfi_set_property(inst, + HFI_PROP_CROP_OFFSETS, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_64_PACKED, + &payload, + sizeof(u64)); +} + +static int vdec_set_bit_depth(struct iris_inst *inst) +{ + u32 bitdepth = 8 << 16 | 8; + u32 pix_fmt; + + pix_fmt = inst->fmt_dst->fmt.pix_mp.pixelformat; + if (is_10bit_colorformat(pix_fmt)) + bitdepth = 10 << 16 | 10; + + inst->src_subcr_params.bit_depth = bitdepth; + inst->cap[BIT_DEPTH].value = bitdepth; + + return iris_hfi_set_property(inst, + HFI_PROP_LUMA_CHROMA_BIT_DEPTH, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32, + &bitdepth, + sizeof(u32)); +} + +static int vdec_set_coded_frames(struct iris_inst *inst) +{ + u32 coded_frames = 0; + + if (inst->cap[CODED_FRAMES].value == CODED_FRAMES_PROGRESSIVE) + coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG; + inst->src_subcr_params.coded_frames = coded_frames; + + return iris_hfi_set_property(inst, + HFI_PROP_CODED_FRAMES, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32, + &coded_frames, + sizeof(u32)); +} + +static int vdec_set_min_output_count(struct iris_inst *inst) +{ + u32 min_output; + + min_output = inst->buffers.output.min_count; + inst->src_subcr_params.fw_min_count = min_output; + + return iris_hfi_set_property(inst, + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32, + &min_output, + sizeof(u32)); +} + +static int vdec_set_picture_order_count(struct iris_inst *inst) +{ + u32 poc = 0; + + inst->src_subcr_params.pic_order_cnt = poc; + + return iris_hfi_set_property(inst, + HFI_PROP_PIC_ORDER_CNT_TYPE, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32, + &poc, + sizeof(u32)); +} + +static int vdec_set_colorspace(struct iris_inst *inst) +{ + u32 video_signal_type_present_flag = 0, color_info = 0; + u32 matrix_coeff = HFI_MATRIX_COEFF_RESERVED; + u32 video_format = UNSPECIFIED_COLOR_FORMAT; + struct v4l2_pix_format_mplane *pixmp = NULL; + u32 full_range = V4L2_QUANTIZATION_DEFAULT; + u32 transfer_char = HFI_TRANSFER_RESERVED; + u32 colour_description_present_flag = 0; + u32 primaries = HFI_PRIMARIES_RESERVED; + + int ret; + + if (inst->codec == VP9) + return 0; + + pixmp = &inst->fmt_src->fmt.pix_mp; + if (pixmp->colorspace != V4L2_COLORSPACE_DEFAULT || + pixmp->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT || + pixmp->xfer_func != V4L2_XFER_FUNC_DEFAULT) { + colour_description_present_flag = 1; + video_signal_type_present_flag = 1; + primaries = get_hfi_color_primaries(pixmp->colorspace); + matrix_coeff = get_hfi_matrix_coefficients(pixmp->ycbcr_enc); + transfer_char = get_hfi_transfer_char(pixmp->xfer_func); + } + + if (pixmp->quantization != V4L2_QUANTIZATION_DEFAULT) { + video_signal_type_present_flag = 1; + full_range = pixmp->quantization == + V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0; + } + + color_info = (matrix_coeff & 0xFF) | + ((transfer_char << 8) & 0xFF00) | + ((primaries << 16) & 0xFF0000) | + ((colour_description_present_flag << 24) & 0x1000000) | + ((full_range << 25) & 0x2000000) | + ((video_format << 26) & 0x1C000000) | + ((video_signal_type_present_flag << 29) & 0x20000000); + + inst->src_subcr_params.color_info = color_info; + + ret = iris_hfi_set_property(inst, + HFI_PROP_SIGNAL_COLOR_INFO, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_32_PACKED, + &color_info, + sizeof(u32)); + + return ret; +} + +static int vdec_set_profile(struct iris_inst *inst) +{ + u32 profile; + + profile = inst->cap[PROFILE].value; + inst->src_subcr_params.profile = profile; + + return iris_hfi_set_property(inst, + HFI_PROP_PROFILE, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32_ENUM, + &profile, + sizeof(u32)); +} + +static int vdec_set_level(struct iris_inst *inst) +{ + u32 level; + + level = inst->cap[LEVEL].value; + inst->src_subcr_params.level = level; + + return iris_hfi_set_property(inst, + HFI_PROP_LEVEL, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32_ENUM, + &level, + sizeof(u32)); +} + +static int vdec_set_tier(struct iris_inst *inst) +{ + u32 tier; + + tier = inst->cap[HEVC_TIER].value; + inst->src_subcr_params.tier = tier; + + return iris_hfi_set_property(inst, + HFI_PROP_TIER, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32_ENUM, + &tier, + sizeof(u32)); +} + +int vdec_subscribe_src_change_param(struct iris_inst *inst) +{ + const u32 *src_change_param; + u32 src_change_param_size; + struct iris_core *core; + u32 payload[32] = {0}; + int ret; + u32 i, j; + + static const struct vdec_prop_type_handle prop_type_handle_arr[] = { + {HFI_PROP_BITSTREAM_RESOLUTION, vdec_set_bitstream_resolution }, + {HFI_PROP_CROP_OFFSETS, vdec_set_crop_offsets }, + {HFI_PROP_LUMA_CHROMA_BIT_DEPTH, vdec_set_bit_depth }, + {HFI_PROP_CODED_FRAMES, vdec_set_coded_frames }, + {HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, vdec_set_min_output_count }, + {HFI_PROP_PIC_ORDER_CNT_TYPE, vdec_set_picture_order_count }, + {HFI_PROP_SIGNAL_COLOR_INFO, vdec_set_colorspace }, + {HFI_PROP_PROFILE, vdec_set_profile }, + {HFI_PROP_LEVEL, vdec_set_level }, + {HFI_PROP_TIER, vdec_set_tier }, + }; + + core = inst->core; + + payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE; + if (inst->codec == H264) { + src_change_param_size = core->platform_data->avc_subscribe_param_size; + src_change_param = core->platform_data->avc_subscribe_param; + } else if (inst->codec == HEVC) { + src_change_param_size = core->platform_data->hevc_subscribe_param_size; + src_change_param = core->platform_data->hevc_subscribe_param; + } else if (inst->codec == VP9) { + src_change_param_size = core->platform_data->vp9_subscribe_param_size; + src_change_param = core->platform_data->vp9_subscribe_param; + } else { + src_change_param = NULL; + return -EINVAL; + } + + if (!src_change_param || !src_change_param_size) + return -EINVAL; + + for (i = 0; i < src_change_param_size; i++) + payload[i + 1] = src_change_param[i]; + + ret = iris_hfi_session_subscribe_mode(inst, + HFI_CMD_SUBSCRIBE_MODE, + INPUT_MPLANE, + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + ((src_change_param_size + 1) * sizeof(u32))); + if (ret) + return ret; + + for (i = 0; i < src_change_param_size; i++) { + for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) { + if (prop_type_handle_arr[j].type == src_change_param[i]) { + ret = prop_type_handle_arr[j].handle(inst); + if (ret) + return ret; + break; + } + } + } + + return ret; +} + +int vdec_init_src_change_param(struct iris_inst *inst) +{ + u32 left_offset, top_offset, right_offset, bottom_offset; + struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op; + u32 primaries, matrix_coeff, transfer_char; + struct subscription_params *subsc_params; + u32 colour_description_present_flag = 0; + u32 video_signal_type_present_flag = 0; + u32 full_range = 0, video_format = 0; + + subsc_params = &inst->src_subcr_params; + pixmp_ip = &inst->fmt_src->fmt.pix_mp; + pixmp_op = &inst->fmt_dst->fmt.pix_mp; + + subsc_params->bitstream_resolution = + pixmp_ip->width << 16 | pixmp_ip->height; + + left_offset = inst->crop.left; + top_offset = inst->crop.top; + right_offset = (pixmp_ip->width - inst->crop.width); + bottom_offset = (pixmp_ip->height - inst->crop.height); + subsc_params->crop_offsets[0] = + left_offset << 16 | top_offset; + subsc_params->crop_offsets[1] = + right_offset << 16 | bottom_offset; + + subsc_params->fw_min_count = inst->buffers.output.min_count; + + primaries = get_hfi_color_primaries(pixmp_op->colorspace); + matrix_coeff = get_hfi_matrix_coefficients(pixmp_op->ycbcr_enc); + transfer_char = get_hfi_transfer_char(pixmp_op->xfer_func); + full_range = pixmp_op->quantization == V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0; + subsc_params->color_info = + (matrix_coeff & 0xFF) | + ((transfer_char << 8) & 0xFF00) | + ((primaries << 16) & 0xFF0000) | + ((colour_description_present_flag << 24) & 0x1000000) | + ((full_range << 25) & 0x2000000) | + ((video_format << 26) & 0x1C000000) | + ((video_signal_type_present_flag << 29) & 0x20000000); + + subsc_params->profile = inst->cap[PROFILE].value; + subsc_params->level = inst->cap[LEVEL].value; + subsc_params->tier = inst->cap[HEVC_TIER].value; + subsc_params->pic_order_cnt = inst->cap[POC].value; + subsc_params->bit_depth = inst->cap[BIT_DEPTH].value; + if (inst->cap[CODED_FRAMES].value == + CODED_FRAMES_PROGRESSIVE) + subsc_params->coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG; + else + subsc_params->coded_frames = 0; + + return 0; +} + +static int vdec_read_input_subcr_params(struct iris_inst *inst) +{ + struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op; + u32 primaries, matrix_coeff, transfer_char; + struct subscription_params subsc_params; + u32 colour_description_present_flag = 0; + u32 video_signal_type_present_flag = 0; + u32 full_range = 0; + u32 width, height; + + subsc_params = inst->src_subcr_params; + pixmp_ip = &inst->fmt_src->fmt.pix_mp; + pixmp_op = &inst->fmt_dst->fmt.pix_mp; + width = (subsc_params.bitstream_resolution & + HFI_BITMASK_BITSTREAM_WIDTH) >> 16; + height = subsc_params.bitstream_resolution & + HFI_BITMASK_BITSTREAM_HEIGHT; + + pixmp_ip->width = width; + pixmp_ip->height = height; + + pixmp_op->width = pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ? + ALIGN(width, 192) : ALIGN(width, 128); + pixmp_op->height = pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ? + ALIGN(height, 16) : ALIGN(height, 32); + pixmp_op->plane_fmt[0].bytesperline = + pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ? + ALIGN(ALIGN(width, 192) * 4 / 3, 256) : + ALIGN(width, 128); + pixmp_op->plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); + + matrix_coeff = subsc_params.color_info & 0xFF; + transfer_char = (subsc_params.color_info & 0xFF00) >> 8; + primaries = (subsc_params.color_info & 0xFF0000) >> 16; + colour_description_present_flag = + (subsc_params.color_info & 0x1000000) >> 24; + full_range = (subsc_params.color_info & 0x2000000) >> 25; + video_signal_type_present_flag = + (subsc_params.color_info & 0x20000000) >> 29; + + pixmp_op->colorspace = V4L2_COLORSPACE_DEFAULT; + pixmp_op->xfer_func = V4L2_XFER_FUNC_DEFAULT; + pixmp_op->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + pixmp_op->quantization = V4L2_QUANTIZATION_DEFAULT; + + if (video_signal_type_present_flag) { + pixmp_op->quantization = + full_range ? + V4L2_QUANTIZATION_FULL_RANGE : + V4L2_QUANTIZATION_LIM_RANGE; + if (colour_description_present_flag) { + pixmp_op->colorspace = + get_v4l2_color_primaries(primaries); + pixmp_op->xfer_func = + get_v4l2_transfer_char(transfer_char); + pixmp_op->ycbcr_enc = + get_v4l2_matrix_coefficients(matrix_coeff); + } + } + + pixmp_ip->colorspace = pixmp_op->colorspace; + pixmp_ip->xfer_func = pixmp_op->xfer_func; + pixmp_ip->ycbcr_enc = pixmp_op->ycbcr_enc; + pixmp_ip->quantization = pixmp_op->quantization; + + inst->crop.top = subsc_params.crop_offsets[0] & 0xFFFF; + inst->crop.left = (subsc_params.crop_offsets[0] >> 16) & 0xFFFF; + inst->crop.height = pixmp_ip->height - + (subsc_params.crop_offsets[1] & 0xFFFF) - inst->crop.top; + inst->crop.width = pixmp_ip->width - + ((subsc_params.crop_offsets[1] >> 16) & 0xFFFF) - inst->crop.left; + + inst->cap[PROFILE].value = subsc_params.profile; + inst->cap[LEVEL].value = subsc_params.level; + inst->cap[HEVC_TIER].value = subsc_params.tier; + inst->cap[POC].value = subsc_params.pic_order_cnt; + + if (subsc_params.bit_depth == BIT_DEPTH_8) + inst->cap[BIT_DEPTH].value = BIT_DEPTH_8; + else + inst->cap[BIT_DEPTH].value = BIT_DEPTH_10; + + if (subsc_params.coded_frames & HFI_BITMASK_FRAME_MBS_ONLY_FLAG) + inst->cap[CODED_FRAMES].value = CODED_FRAMES_PROGRESSIVE; + else + inst->cap[CODED_FRAMES].value = CODED_FRAMES_INTERLACE; + + inst->fw_min_count = subsc_params.fw_min_count; + inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT); + + return 0; +} + +int vdec_src_change(struct iris_inst *inst) +{ + struct v4l2_event event = {0}; + u32 ret; + + if (!inst->vb2q_src->streaming) + return 0; + + ret = vdec_read_input_subcr_params(inst); + if (ret) + return ret; + + event.type = V4L2_EVENT_SOURCE_CHANGE; + event.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION; + v4l2_event_queue_fh(&inst->fh, &event); + + return ret; +} + static int vdec_set_colorformat(struct iris_inst *inst) { u32 hfi_colorformat; @@ -483,3 +939,127 @@ int vdec_set_output_property(struct iris_inst *inst) return vdec_set_ubwc_stride_scanline(inst); } + +int vdec_subscribe_dst_change_param(struct iris_inst *inst) +{ + u32 prop_type, payload_size, payload_type; + struct subscription_params subsc_params; + const u32 *dst_change_param = NULL; + u32 dst_change_param_size = 0; + struct iris_core *core; + u32 payload[32] = {0}; + int ret; + u32 i; + + core = inst->core; + + payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE; + if (inst->codec == H264) { + dst_change_param_size = core->platform_data->avc_subscribe_param_size; + dst_change_param = core->platform_data->avc_subscribe_param; + } else if (inst->codec == HEVC) { + dst_change_param_size = core->platform_data->hevc_subscribe_param_size; + dst_change_param = core->platform_data->hevc_subscribe_param; + } else if (inst->codec == VP9) { + dst_change_param_size = core->platform_data->vp9_subscribe_param_size; + dst_change_param = core->platform_data->vp9_subscribe_param; + } else { + dst_change_param = NULL; + return -EINVAL; + } + + if (!dst_change_param || !dst_change_param_size) + return -EINVAL; + + payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE; + for (i = 0; i < dst_change_param_size; i++) + payload[i + 1] = dst_change_param[i]; + + ret = iris_hfi_session_subscribe_mode(inst, + HFI_CMD_SUBSCRIBE_MODE, + OUTPUT_MPLANE, + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + ((dst_change_param_size + 1) * sizeof(u32))); + if (ret) + return ret; + + subsc_params = inst->dst_subcr_params; + for (i = 0; i < dst_change_param_size; i++) { + payload[0] = 0; + payload[1] = 0; + payload_size = 0; + payload_type = 0; + prop_type = dst_change_param[i]; + switch (prop_type) { + case HFI_PROP_BITSTREAM_RESOLUTION: + payload[0] = subsc_params.bitstream_resolution; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_CROP_OFFSETS: + payload[0] = subsc_params.crop_offsets[0]; + payload[1] = subsc_params.crop_offsets[1]; + payload_size = sizeof(u64); + payload_type = HFI_PAYLOAD_64_PACKED; + break; + case HFI_PROP_LUMA_CHROMA_BIT_DEPTH: + payload[0] = subsc_params.bit_depth; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_CODED_FRAMES: + payload[0] = subsc_params.coded_frames; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT: + payload[0] = subsc_params.fw_min_count; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_PIC_ORDER_CNT_TYPE: + payload[0] = subsc_params.pic_order_cnt; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_SIGNAL_COLOR_INFO: + payload[0] = subsc_params.color_info; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_PROFILE: + payload[0] = subsc_params.profile; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_LEVEL: + payload[0] = subsc_params.level; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_TIER: + payload[0] = subsc_params.tier; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + default: + prop_type = 0; + ret = -EINVAL; + break; + } + if (prop_type) { + ret = iris_hfi_set_property(inst, + prop_type, + HFI_HOST_FLAGS_NONE, + get_hfi_port(OUTPUT_MPLANE), + payload_type, + &payload, + payload_size); + if (ret) + return ret; + } + } + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index 6b0306c..e0db653 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -16,5 +16,9 @@ int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); int vdec_subscribe_property(struct iris_inst *inst, u32 plane); int vdec_set_output_property(struct iris_inst *inst); +int vdec_init_src_change_param(struct iris_inst *inst); +int vdec_src_change(struct iris_inst *inst); +int vdec_subscribe_src_change_param(struct iris_inst *inst); +int vdec_subscribe_dst_change_param(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index fc12bde..22a8f5b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -206,6 +206,12 @@ struct platform_data { u32 core_data_size; struct plat_inst_cap *inst_cap_data; u32 inst_cap_data_size; + const u32 *avc_subscribe_param; + unsigned int avc_subscribe_param_size; + const u32 *hevc_subscribe_param; + unsigned int hevc_subscribe_param_size; + const u32 *vp9_subscribe_param; + unsigned int vp9_subscribe_param_size; const u32 *dec_input_prop; unsigned int dec_input_prop_size; const u32 *dec_output_prop_avc; diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 6a4bfa3..7ae9715 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -382,6 +382,38 @@ static struct format_capability format_data_sm8550 = { .color_format_info_size = ARRAY_SIZE(color_format_data_sm8550), }; +static const u32 sm8550_vdec_src_change_param_avc[] = { + HFI_PROP_BITSTREAM_RESOLUTION, + HFI_PROP_CROP_OFFSETS, + HFI_PROP_CODED_FRAMES, + HFI_PROP_LUMA_CHROMA_BIT_DEPTH, + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, + HFI_PROP_PIC_ORDER_CNT_TYPE, + HFI_PROP_PROFILE, + HFI_PROP_LEVEL, + HFI_PROP_SIGNAL_COLOR_INFO, +}; + +static const u32 sm8550_vdec_src_change_param_hevc[] = { + HFI_PROP_BITSTREAM_RESOLUTION, + HFI_PROP_CROP_OFFSETS, + HFI_PROP_LUMA_CHROMA_BIT_DEPTH, + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, + HFI_PROP_PROFILE, + HFI_PROP_LEVEL, + HFI_PROP_TIER, + HFI_PROP_SIGNAL_COLOR_INFO, +}; + +static const u32 sm8550_vdec_src_change_param_vp9[] = { + HFI_PROP_BITSTREAM_RESOLUTION, + HFI_PROP_CROP_OFFSETS, + HFI_PROP_LUMA_CHROMA_BIT_DEPTH, + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, + HFI_PROP_PROFILE, + HFI_PROP_LEVEL, +}; + static const u32 sm8550_vdec_input_properties[] = { HFI_PROP_NO_OUTPUT, }; @@ -430,6 +462,18 @@ struct platform_data sm8550_data = { .ubwc_config = ubwc_config_sm8550, .format_data = &format_data_sm8550, + .avc_subscribe_param = + sm8550_vdec_src_change_param_avc, + .avc_subscribe_param_size = + ARRAY_SIZE(sm8550_vdec_src_change_param_avc), + .hevc_subscribe_param = + sm8550_vdec_src_change_param_hevc, + .hevc_subscribe_param_size = + ARRAY_SIZE(sm8550_vdec_src_change_param_hevc), + .vp9_subscribe_param = + sm8550_vdec_src_change_param_vp9, + .vp9_subscribe_param_size = + ARRAY_SIZE(sm8550_vdec_src_change_param_vp9), .dec_input_prop = sm8550_vdec_input_properties, .dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_input_properties), .dec_output_prop_avc = sm8550_vdec_output_properties_avc, From patchwork Mon Dec 18 11:32:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757244 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 436BB1A59B; Mon, 18 Dec 2023 11:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="M9u23Pn6" Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArs70008055; Mon, 18 Dec 2023 11:36:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=inJAjKri0vli8p2oISrlOgpqbAM5ecoVH66wMnG6Rqw=; b=M9 u23Pn6j0G+GxeMfPPrUV5BjvK6tuodz84LROAyRv/AHqBSpR+z6BBMDAdINBuokR 4gOSYZPreW+LPyifNyR11oTIVtNdZlkOdZfrbbyfXWjKDU4tBZ8di+9mq0ViZjMV Tr79CBIDgeY7F1uBTNXzn4V3uYGu3WiqUFsX+h+amzke4HyP9pLBqjcRSmGtLOmI p7dZkjMhTlu0a81IoxUkXVYiOUzKmLLE4tKrsjPBcOGEzw2szlFTxcJaZXhvpBDk JwjgvVjSsT5b41Oo6fX9R2MCfq0IULRja9mlAvMi+6aj0JG/GDn3JUuLVPjk417b Nbx9QMVKtxjW79z+rMbg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2m49g5sg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:12 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7qK030018; Mon, 18 Dec 2023 11:36:08 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyjh-1; Mon, 18 Dec 2023 11:33:09 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX54a029912; Mon, 18 Dec 2023 11:33:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6sI029966; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 5F827234B; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 26/34] media: iris: implement vb2 streaming ops on capture and output planes Date: Mon, 18 Dec 2023 17:02:21 +0530 Message-Id: <1702899149-21321-27-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: k_TfUofMAz_uHiWPsvyal0H5DAGv77kg X-Proofpoint-GUID: k_TfUofMAz_uHiWPsvyal0H5DAGv77kg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=999 impostorscore=0 malwarescore=0 clxscore=1015 mlxscore=0 spamscore=0 suspectscore=0 bulkscore=0 priorityscore=1501 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: During stream on, set some mandatory properties to firmware to start a session. Subscribe for different bitstream parameters to get notified for change in any of the subscribed parameters. Set all v4l2 properties set by client, to firmware prepared with the dependency graph. Also, configure the hardware internal buffers required for frame processing. Send HFI_CMD_START on capture and output planes to start the processing on respective planes. The instance state is changed accordingly. During stream off, send HFI_CMD_STOP to firmware which is a synchronous command. After the response is received from firmware, the session is closed on firmware and instance state is changed accordingly. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 12 ++ .../media/platform/qcom/vcodec/iris/iris_buffer.c | 21 +++ .../media/platform/qcom/vcodec/iris/iris_buffer.h | 3 + .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 28 ++++ .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 1 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 51 ++++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 1 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 173 +++++++++++++++++++-- drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 7 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 14 ++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 2 +- .../platform/qcom/vcodec/iris/iris_instance.h | 4 + .../media/platform/qcom/vcodec/iris/iris_state.c | 38 +++++ .../media/platform/qcom/vcodec/iris/iris_state.h | 3 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 78 ++++++++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 2 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 138 +++++++++++++++- .../media/platform/qcom/vcodec/iris/iris_vdec.h | 6 +- .../media/platform/qcom/vcodec/iris/iris_vidc.c | 4 + 19 files changed, 566 insertions(+), 20 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index 0ef6bad..dfd1a4c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -14,6 +14,8 @@ #define HFI_CMD_INIT 0x01000001 #define HFI_CMD_OPEN 0x01000003 #define HFI_CMD_CLOSE 0x01000004 +#define HFI_CMD_START 0x01000005 +#define HFI_CMD_STOP 0x01000006 #define HFI_CMD_BUFFER 0x01000009 #define HFI_PROP_IMAGE_VERSION 0x03000001 @@ -45,6 +47,16 @@ enum hfi_property_mode_type { #define HFI_PROP_IMAGE_VERSION 0x03000001 +enum hfi_codec_type { + HFI_CODEC_DECODE_AVC = 1, + HFI_CODEC_ENCODE_AVC = 2, + HFI_CODEC_DECODE_HEVC = 3, + HFI_CODEC_ENCODE_HEVC = 4, + HFI_CODEC_DECODE_VP9 = 5, +}; + +#define HFI_PROP_CODEC 0x03000100 + enum hfi_color_format { HFI_COLOR_FMT_OPAQUE = 0, HFI_COLOR_FMT_NV12 = 1, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c index 1ee840e..ab25026 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -560,3 +560,24 @@ int iris_release_input_internal_buffers(struct iris_inst *inst) return ret; } + +int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + int ret; + + if (buffer_type != BUF_PERSIST) + return -EINVAL; + + ret = iris_get_internal_buf_info(inst, buffer_type); + if (ret) + return ret; + + ret = iris_create_internal_buffers(inst, buffer_type); + if (ret) + return ret; + + ret = iris_queue_internal_buffers(inst, buffer_type); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h index bdef15f..8769c3d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -59,4 +59,7 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); int iris_release_input_internal_buffers(struct iris_inst *inst); +int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst, + enum iris_buffer_type buffer_type); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c index 28977e8..94fff74 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -113,6 +113,20 @@ static int adjust_cap(struct iris_inst *inst, return cap->adjust(inst, ctrl); } +static int set_cap(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + struct plat_inst_cap *cap; + + cap = &inst->cap[cap_id]; + if (!inst->cap[cap_id].cap_id) + return 0; + + if (!cap->set) + return 0; + + return cap->set(inst, cap_id); +} + static int adjust_dynamic_property(struct iris_inst *inst, enum plat_inst_cap_type cap_id, struct v4l2_ctrl *ctrl, @@ -657,6 +671,20 @@ int set_pipe(struct iris_inst *inst, &work_route, sizeof(u32)); } +int set_v4l2_properties(struct iris_inst *inst) +{ + struct cap_entry *entry = NULL, *temp = NULL; + int ret = 0; + + list_for_each_entry_safe(entry, temp, &inst->caps_list, list) { + ret = set_cap(inst, entry->cap_id); + if (ret) + return ret; + } + + return ret; +} + int adjust_v4l2_properties(struct iris_inst *inst) { struct cap_entry *entry = NULL, *temp = NULL; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h index 22ee6c4b..28ce767 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h @@ -28,6 +28,7 @@ int prepare_dependency_list(struct iris_inst *inst); int iris_init_instance_caps(struct iris_core *core); int iris_init_core_caps(struct iris_core *core); int get_inst_capability(struct iris_inst *inst); +int set_v4l2_properties(struct iris_inst *inst); int adjust_v4l2_properties(struct iris_inst *inst); int ctrls_init(struct iris_inst *inst, bool init); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 4cad673..7868566 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -9,6 +9,7 @@ #include "iris_hfi.h" #include "iris_hfi_packet.h" #include "iris_instance.h" +#include "iris_vidc.h" int check_core_lock(struct iris_core *core) { @@ -377,3 +378,53 @@ struct iris_inst *to_instance(struct iris_core *core, u32 session_id) return NULL; } + +static int kill_session(struct iris_inst *inst) +{ + if (!inst->session_id) + return 0; + + close_session(inst); + iris_inst_change_state(inst, IRIS_INST_ERROR); + + return 0; +} + +int session_streamoff(struct iris_inst *inst, u32 plane) +{ + enum signal_session_response signal_type; + u32 hw_response_timeout_val; + struct iris_core *core; + int ret; + + ret = iris_hfi_stop(inst, plane); + if (ret) + goto error; + + core = inst->core; + hw_response_timeout_val = core->cap[HW_RESPONSE_TIMEOUT].value; + mutex_unlock(&inst->lock); + ret = wait_for_completion_timeout(&inst->completions[signal_type], + msecs_to_jiffies(hw_response_timeout_val)); + if (!ret) { + ret = -ETIMEDOUT; + process_inst_timeout(inst); + } else { + ret = 0; + } + mutex_lock(&inst->lock); + + if (ret) + goto error; + + ret = iris_inst_state_change_streamoff(inst, plane); + if (ret) + goto error; + + return 0; + +error: + kill_session(inst); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index cb22adf..47a017d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -34,6 +34,7 @@ bool is_split_mode_enabled(struct iris_inst *inst); int signal_session_msg_receipt(struct iris_inst *inst, enum signal_session_response cmd); struct iris_inst *to_instance(struct iris_core *core, u32 session_id); +int session_streamoff(struct iris_inst *inst, u32 plane); u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec); enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index d15ce5a..e0da3be 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -219,9 +219,7 @@ int iris_hfi_session_open(struct iris_inst *inst) return ret; } -int iris_hfi_session_subscribe_mode(struct iris_inst *inst, - u32 cmd, u32 plane, u32 payload_type, - void *payload, u32 payload_size) +int iris_hfi_session_close(struct iris_inst *inst) { struct iris_core *core; int ret; @@ -238,14 +236,126 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst, } ret = hfi_packet_session_command(inst, - cmd, + HFI_CMD_CLOSE, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + HFI_PORT_NONE, + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_session_set_codec(struct iris_inst *inst) +{ + struct iris_core *core; + int ret; + u32 codec; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + codec = get_hfi_codec(inst); + ret = hfi_packet_session_property(inst, + HFI_PROP_CODEC, + HFI_HOST_FLAGS_NONE, + HFI_PORT_NONE, + HFI_PAYLOAD_U32_ENUM, + &codec, + sizeof(u32)); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_session_set_default_header(struct iris_inst *inst) +{ + struct iris_core *core; + u32 default_header = false; + int ret; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + default_header = inst->cap[DEFAULT_HEADER].value; + ret = hfi_packet_session_property(inst, + HFI_PROP_DEC_DEFAULT_HEADER, + HFI_HOST_FLAGS_NONE, + get_hfi_port(INPUT_MPLANE), + HFI_PAYLOAD_U32, + &default_header, + sizeof(u32)); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_start(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core; + int ret = 0; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE) + goto unlock; + + ret = hfi_packet_session_command(inst, + HFI_CMD_START, (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED), get_hfi_port(plane), inst->session_id, - payload_type, - payload, - payload_size); + HFI_PAYLOAD_NONE, + NULL, + 0); if (ret) goto unlock; @@ -257,10 +367,10 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst, return ret; } -int iris_hfi_session_close(struct iris_inst *inst) +int iris_hfi_stop(struct iris_inst *inst, u32 plane) { struct iris_core *core; - int ret; + int ret = 0; if (!inst->packet) return -EINVAL; @@ -273,12 +383,15 @@ int iris_hfi_session_close(struct iris_inst *inst) goto unlock; } + if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE) + goto unlock; + ret = hfi_packet_session_command(inst, - HFI_CMD_CLOSE, + HFI_CMD_STOP, (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED | HFI_HOST_FLAGS_NON_DISCARDABLE), - HFI_PORT_NONE, + get_hfi_port(plane), inst->session_id, HFI_PAYLOAD_NONE, NULL, @@ -294,6 +407,44 @@ int iris_hfi_session_close(struct iris_inst *inst) return ret; } +int iris_hfi_session_subscribe_mode(struct iris_inst *inst, + u32 cmd, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + struct iris_core *core; + int ret; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + ret = hfi_packet_session_command(inst, + cmd, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + get_hfi_port(plane), + inst->session_id, + payload_type, + payload, + payload_size); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + irqreturn_t iris_hfi_isr(int irq, void *data) { disable_irq_nosync(irq); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index d6b3fca..f054c2d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -19,6 +19,13 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst, int iris_hfi_set_property(struct iris_inst *inst, u32 packet_type, u32 flag, u32 plane, u32 payload_type, void *payload, u32 payload_size); + +int iris_hfi_session_set_codec(struct iris_inst *inst); +int iris_hfi_session_set_default_header(struct iris_inst *inst); + +int iris_hfi_start(struct iris_inst *inst, u32 plane); +int iris_hfi_stop(struct iris_inst *inst, u32 plane); + int iris_hfi_queue_buffer(struct iris_inst *inst, struct iris_buffer *buffer); int iris_hfi_release_buffer(struct iris_inst *inst, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index dc7157d..b8785a9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -76,6 +76,20 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) } } +u32 get_hfi_codec(struct iris_inst *inst) +{ + switch (inst->codec) { + case H264: + return HFI_CODEC_DECODE_AVC; + case HEVC: + return HFI_CODEC_DECODE_HEVC; + case VP9: + return HFI_CODEC_DECODE_VP9; + default: + return 0; + } +} + u32 get_hfi_colorformat(u32 colorformat) { u32 hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index f813116..cf0960b6 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -81,8 +81,8 @@ enum hfi_packet_port_type { u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type); u32 get_hfi_port(u32 plane); - u32 get_hfi_colorformat(u32 colorformat); +u32 get_hfi_codec(struct iris_inst *inst); u32 get_hfi_color_primaries(u32 primaries); u32 get_hfi_transfer_char(u32 characterstics); u32 get_hfi_matrix_coefficients(u32 coefficients); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 4f51d68..cddf143 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -40,10 +40,12 @@ * @fw_min_count: minimnum count of buffers needed by fw * @state: instance state * @ipsc_properties_set: boolean to set ipsc properties to fw + * @opsc_properties_set: boolean to set opsc properties to fw * @hfi_frame_info: structure of frame info * @src_subcr_params: subscription params to fw on input port * @dst_subcr_params: subscription params to fw on output port * @dpb_list_payload: array of dpb buffers + * @once_per_session_set: boolean to set once per session property */ struct iris_inst { @@ -70,10 +72,12 @@ struct iris_inst { u32 fw_min_count; enum iris_inst_state state; bool ipsc_properties_set; + bool opsc_properties_set; struct iris_hfi_frame_info hfi_frame_info; struct subscription_params src_subcr_params; struct subscription_params dst_subcr_params; u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE]; + bool once_per_session_set; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 4cf6b69..6553029 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -193,3 +193,41 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id) (inst->state == IRIS_INST_INPUT_STREAMING || inst->state == IRIS_INST_STREAMING))); } + +int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane) +{ + enum iris_inst_state new_state = IRIS_INST_ERROR; + + if (plane == INPUT_MPLANE) { + if (inst->state == IRIS_INST_OPEN) + new_state = IRIS_INST_INPUT_STREAMING; + else if (inst->state == IRIS_INST_OUTPUT_STREAMING) + new_state = IRIS_INST_STREAMING; + } else if (plane == OUTPUT_MPLANE) { + if (inst->state == IRIS_INST_OPEN) + new_state = IRIS_INST_OUTPUT_STREAMING; + else if (inst->state == IRIS_INST_INPUT_STREAMING) + new_state = IRIS_INST_STREAMING; + } + + return iris_inst_change_state(inst, new_state); +} + +int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane) +{ + enum iris_inst_state new_state = IRIS_INST_ERROR; + + if (plane == INPUT_MPLANE) { + if (inst->state == IRIS_INST_INPUT_STREAMING) + new_state = IRIS_INST_OPEN; + else if (inst->state == IRIS_INST_STREAMING) + new_state = IRIS_INST_OUTPUT_STREAMING; + } else if (plane == OUTPUT_MPLANE) { + if (inst->state == IRIS_INST_OUTPUT_STREAMING) + new_state = IRIS_INST_OPEN; + else if (inst->state == IRIS_INST_STREAMING) + new_state = IRIS_INST_INPUT_STREAMING; + } + + return iris_inst_change_state(inst, new_state); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h index 35263e8..28d5380 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -47,4 +47,7 @@ bool allow_streamon(struct iris_inst *inst, u32 type); bool allow_streamoff(struct iris_inst *inst, u32 type); bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id); +int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane); +int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index 66b5295..d599366 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -6,9 +6,12 @@ #include "iris_buffer.h" #include "iris_ctrls.h" #include "iris_core.h" +#include "iris_ctrls.h" #include "iris_helpers.h" +#include "iris_hfi.h" #include "iris_instance.h" #include "iris_vb2.h" +#include "iris_vdec.h" int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, @@ -90,6 +93,81 @@ int iris_vb2_queue_setup(struct vb2_queue *q, return ret; } +int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) +{ + struct iris_inst *inst; + int ret = 0; + + if (!q) + return -EINVAL; + + inst = vb2_get_drv_priv(q); + if (!inst || !inst->core) + return -EINVAL; + + if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE) { + ret = -EINVAL; + goto error; + } + + if (!inst->once_per_session_set) { + inst->once_per_session_set = true; + ret = iris_hfi_session_set_codec(inst); + if (ret) + goto error; + + ret = iris_hfi_session_set_default_header(inst); + if (ret) + goto error; + + ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST); + if (ret) + goto error; + } + + if (q->type == INPUT_MPLANE) + ret = vdec_streamon_input(inst); + else if (q->type == OUTPUT_MPLANE) + ret = vdec_streamon_output(inst); + if (ret) + goto error; + + return ret; +error: + iris_inst_change_state(inst, IRIS_INST_ERROR); + + return ret; +} + +void iris_vb2_stop_streaming(struct vb2_queue *q) +{ + struct iris_inst *inst; + int ret = 0; + + if (!q) + return; + + inst = vb2_get_drv_priv(q); + if (!inst) + return; + + if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE) + goto error; + + if (q->type == INPUT_MPLANE) + ret = session_streamoff(inst, INPUT_MPLANE); + else if (q->type == OUTPUT_MPLANE) + ret = session_streamoff(inst, OUTPUT_MPLANE); + + if (ret) + goto error; + + return; + +error: + iris_inst_change_state(inst, IRIS_INST_ERROR); +} + void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, struct dma_buf *dbuf, unsigned long size) { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h index 4342034..fc0e804 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h @@ -11,6 +11,8 @@ int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, unsigned int sizes[], struct device *alloc_devs[]); +int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count); +void iris_vb2_stop_streaming(struct vb2_queue *q); /* vb2_mem_ops */ void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index ac47fc0..0d8ca4b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -340,7 +340,7 @@ int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscri return ret; } -int vdec_subscribe_property(struct iris_inst *inst, u32 plane) +static int vdec_subscribe_property(struct iris_inst *inst, u32 plane) { const u32 *subcribe_prop = NULL; u32 subscribe_prop_size = 0; @@ -597,7 +597,7 @@ static int vdec_set_tier(struct iris_inst *inst) sizeof(u32)); } -int vdec_subscribe_src_change_param(struct iris_inst *inst) +static int vdec_subscribe_src_change_param(struct iris_inst *inst) { const u32 *src_change_param; u32 src_change_param_size; @@ -925,7 +925,7 @@ static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst) sizeof(u32) * 4); } -int vdec_set_output_property(struct iris_inst *inst) +static int vdec_set_output_property(struct iris_inst *inst) { int ret; @@ -940,7 +940,7 @@ int vdec_set_output_property(struct iris_inst *inst) return vdec_set_ubwc_stride_scanline(inst); } -int vdec_subscribe_dst_change_param(struct iris_inst *inst) +static int vdec_subscribe_dst_change_param(struct iris_inst *inst) { u32 prop_type, payload_size, payload_type; struct subscription_params subsc_params; @@ -1063,3 +1063,133 @@ int vdec_subscribe_dst_change_param(struct iris_inst *inst) return ret; } + +static int process_streamon_input(struct iris_inst *inst) +{ + int ret; + + ret = iris_hfi_start(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE); + if (ret) + return ret; + + return ret; +} + +int vdec_streamon_input(struct iris_inst *inst) +{ + int ret; + + ret = check_session_supported(inst); + if (ret) + return ret; + + ret = set_v4l2_properties(inst); + if (ret) + return ret; + + ret = iris_get_internal_buffers(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = iris_destroy_internal_buffers(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = iris_create_input_internal_buffers(inst); + if (ret) + return ret; + + ret = iris_queue_input_internal_buffers(inst); + if (ret) + return ret; + + if (!inst->ipsc_properties_set) { + ret = vdec_subscribe_src_change_param(inst); + if (ret) + return ret; + inst->ipsc_properties_set = true; + } + + ret = vdec_subscribe_property(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = process_streamon_input(inst); + if (ret) + return ret; + + return ret; +} + +static int process_streamon_output(struct iris_inst *inst) +{ + int ret; + + ret = iris_hfi_start(inst, OUTPUT_MPLANE); + if (ret) + return ret; + + ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE); + if (ret) + return ret; + + return ret; +} + +int vdec_streamon_output(struct iris_inst *inst) +{ + int ret; + + ret = check_session_supported(inst); + if (ret) + return ret; + + ret = vdec_set_output_property(inst); + if (ret) + goto error; + + if (!inst->opsc_properties_set) { + memcpy(&inst->dst_subcr_params, + &inst->src_subcr_params, + sizeof(inst->src_subcr_params)); + ret = vdec_subscribe_dst_change_param(inst); + if (ret) + goto error; + inst->opsc_properties_set = true; + } + + ret = vdec_subscribe_property(inst, OUTPUT_MPLANE); + if (ret) + goto error; + + ret = iris_get_internal_buffers(inst, OUTPUT_MPLANE); + if (ret) + goto error; + + ret = iris_destroy_internal_buffers(inst, OUTPUT_MPLANE); + if (ret) + goto error; + + ret = iris_create_output_internal_buffers(inst); + if (ret) + goto error; + + ret = process_streamon_output(inst); + if (ret) + goto error; + + ret = iris_queue_output_internal_buffers(inst); + if (ret) + goto error; + + return ret; + +error: + session_streamoff(inst, OUTPUT_MPLANE); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index e0db653..0722da1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -14,11 +14,9 @@ int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); -int vdec_subscribe_property(struct iris_inst *inst, u32 plane); -int vdec_set_output_property(struct iris_inst *inst); int vdec_init_src_change_param(struct iris_inst *inst); int vdec_src_change(struct iris_inst *inst); -int vdec_subscribe_src_change_param(struct iris_inst *inst); -int vdec_subscribe_dst_change_param(struct iris_inst *inst); +int vdec_streamon_input(struct iris_inst *inst); +int vdec_streamon_output(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 5c76821..14d0077 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -176,6 +176,8 @@ int vidc_open(struct file *filp) inst->core = core; inst->session_id = hash32_ptr(inst); + inst->ipsc_properties_set = false; + inst->opsc_properties_set = false; iris_inst_change_state(inst, IRIS_INST_OPEN); mutex_init(&inst->lock); mutex_init(&inst->ctx_q_lock); @@ -908,6 +910,8 @@ static const struct v4l2_file_operations v4l2_file_ops = { static const struct vb2_ops iris_vb2_ops = { .queue_setup = iris_vb2_queue_setup, + .start_streaming = iris_vb2_start_streaming, + .stop_streaming = iris_vb2_stop_streaming, }; static struct vb2_mem_ops iris_vb2_mem_ops = { From patchwork Mon Dec 18 11:32:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757237 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFD8F200DD; Mon, 18 Dec 2023 11:40:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="NaJLUqwO" Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBF2DJ020287; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=2B2H49iDz0W9d1l1UKpjzmSYJV0KYD6exsP+wHxqA5A=; b=Na JLUqwOqOT38WkcxXbRkTzJ3c4eb1DabZ/kkfrfqrYfNBXoEZfyrKY9F40Uj5fbGh iZQE6WRrVvPp26WL9eTPIbMHL+7yNP10FkZ9u1sBlUqiMwfciTgQaZg8TJIGfqEW ASUXXqlo1bhLE/OlvHnhStDJB8p1GhRaIL0Q+qR3sY1+/U+LVzFzDlt3V5W4bhHR OfIewkNo0OjEHkIYuChIsUJTXfwNLIOCuzRR51an3shAZ1AzkF9m+1mhDLN4mNEO vFAy+3erTwNN7FD2oRnGyFqoHUDx+5N/eGGqxClrvtmAGo8sOPM9t3+I2W5Q9Q1p EXRxkP0KNzLzoZIZVu0g== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2n1781tv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0Hn004853; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ae-3; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX77m029983; Mon, 18 Dec 2023 11:33:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6J0029963; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 620202358; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response Date: Mon, 18 Dec 2023 17:02:22 +0530 Message-Id: <1702899149-21321-28-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: gK5mxoCh8tKNMrmSfHTLRqXEaSkNTTdK X-Proofpoint-GUID: gK5mxoCh8tKNMrmSfHTLRqXEaSkNTTdK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 adultscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 malwarescore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement vb2 ops for buf queue. Below are the different buffer attributes: BUF_ATTR_DEFERRED - buffer queued by client but not submitted to firmware. BUF_ATTR_READ_ONLY - processed buffer received from firmware as read only. These buffers are held in firmware as reference for future frame processing. BUF_ATTR_PENDING_RELEASE - buffers requested to be released from firmware. BUF_ATTR_QUEUED - buffers submitted to firmware. BUF_ATTR_DEQUEUED - buffers received from firmware. BUF_ATTR_BUFFER_DONE - buffers sent back to vb2. Buffers are submitted and received via HFI_CMD_BUFFER. Firmware associates below flags during buffer response: HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware. HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware. Input buffers dequeued from firmware are sent directly to vb2. Output buffers if read only, are sent to vb2 and also maintained in read only list. If the same read only buffer is received form client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and submitted to firmware. Once the buffer is received from firmware as non read only, it is removed from read only list. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 6 + .../media/platform/qcom/vcodec/iris/iris_buffer.h | 1 - .../media/platform/qcom/vcodec/iris/iris_common.h | 9 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 369 +++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 11 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 2 + .../platform/qcom/vcodec/iris/iris_hfi_response.c | 392 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_state.c | 8 + .../media/platform/qcom/vcodec/iris/iris_state.h | 1 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 81 +++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 1 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 49 +++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 1 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 2 + 15 files changed, 959 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index dfd1a4c..104ef9c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -238,4 +238,10 @@ enum hfi_buffer_host_flags { HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200, }; +enum hfi_buffer_firmware_flags { + HFI_BUF_FW_FLAG_NONE = 0x00000000, + HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001, + HFI_BUF_FW_FLAG_READONLY = 0x00000010, +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h index 8769c3d..59c4a10 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -58,7 +58,6 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); int iris_release_input_internal_buffers(struct iris_inst *inst); - int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst, enum iris_buffer_type buffer_type); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 0fbd139..5fd96d3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -23,6 +23,8 @@ #define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) +#define MAX_DPB_COUNT 32 + #define MAX_DPB_LIST_ARRAY_SIZE (16 * 4) #define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4) @@ -69,6 +71,13 @@ enum iris_buffer_type { BUF_VPSS, }; +enum iris_buffer_flags { + BUF_FLAG_KEYFRAME = 0x00000008, + BUF_FLAG_PFRAME = 0x00000010, + BUF_FLAG_BFRAME = 0x00000020, + BUF_FLAG_ERROR = 0x00000040, +}; + enum iris_buffer_attributes { BUF_ATTR_DEFERRED = BIT(0), BUF_ATTR_READ_ONLY = BIT(1), diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 7868566..a5b8aef 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -57,6 +57,18 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type) } } +u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return INPUT_MPLANE; + case BUF_OUTPUT: + return OUTPUT_MPLANE; + default: + return 0; + } +} + int get_mbpf(struct iris_inst *inst) { int height = 0, width = 0; @@ -390,13 +402,359 @@ static int kill_session(struct iris_inst *inst) return 0; } +struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index) +{ + struct iris_buffer *iter = NULL; + struct iris_buffer *buf = NULL; + enum iris_buffer_type buf_type; + struct iris_buffers *buffers; + + bool found = false; + + buf_type = v4l2_type_to_driver(plane); + if (!buf_type) + return NULL; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return NULL; + + list_for_each_entry(iter, &buffers->list, list) { + if (iter->index == index) { + found = true; + buf = iter; + break; + } + } + + if (!found) + return NULL; + + return buf; +} + +static void process_requeued_readonly_buffers(struct iris_inst *inst, + struct iris_buffer *buf) +{ + struct iris_buffer *ro_buf, *dummy; + + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->device_addr != buf->device_addr) + continue; + if (ro_buf->attr & BUF_ATTR_READ_ONLY && + !(ro_buf->attr & BUF_ATTR_PENDING_RELEASE)) { + buf->attr |= BUF_ATTR_READ_ONLY; + + list_del_init(&ro_buf->list); + kfree(ro_buf); + break; + } + } +} + +int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf) +{ + int ret; + + if (buf->type == BUF_OUTPUT) + process_requeued_readonly_buffers(inst, buf); + + ret = iris_hfi_queue_buffer(inst, buf); + if (ret) + return ret; + + buf->attr &= ~BUF_ATTR_DEFERRED; + buf->attr |= BUF_ATTR_QUEUED; + + return ret; +} + +int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf; + int ret = 0; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return -EINVAL; + + list_for_each_entry(buf, &buffers->list, list) { + if (!(buf->attr & BUF_ATTR_DEFERRED)) + continue; + ret = queue_buffer(inst, buf); + if (ret) + return ret; + } + + return ret; +} + +int iris_release_nonref_buffers(struct iris_inst *inst) +{ + u32 fw_ro_count = 0, nonref_ro_count = 0; + struct iris_buffer *ro_buf; + bool found = false; + int ret = 0; + int i = 0; + + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + if (!(ro_buf->attr & BUF_ATTR_READ_ONLY)) + continue; + if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE) + continue; + fw_ro_count++; + } + + if (fw_ro_count <= MAX_DPB_COUNT) + return 0; + + /* + * Mark the read only buffers present in read_only list as + * non-reference if it's not part of dpb_list_payload. + * dpb_list_payload details: + * payload[0-1] : 64 bits base_address of DPB-1 + * payload[2] : 32 bits addr_offset of DPB-1 + * payload[3] : 32 bits data_offset of DPB-1 + */ + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + found = false; + if (!(ro_buf->attr & BUF_ATTR_READ_ONLY)) + continue; + if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE) + continue; + for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) { + if (ro_buf->device_addr == inst->dpb_list_payload[i] && + ro_buf->data_offset == inst->dpb_list_payload[i + 3]) { + found = true; + break; + } + } + if (!found) + nonref_ro_count++; + } + + if (nonref_ro_count <= inst->buffers.output.min_count) + return 0; + + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + found = false; + if (!(ro_buf->attr & BUF_ATTR_READ_ONLY)) + continue; + if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE) + continue; + for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) { + if (ro_buf->device_addr == inst->dpb_list_payload[i] && + ro_buf->data_offset == inst->dpb_list_payload[i + 3]) { + found = true; + break; + } + } + if (!found) { + ro_buf->attr |= BUF_ATTR_PENDING_RELEASE; + ret = iris_hfi_release_buffer(inst, ro_buf); + if (ret) + return ret; + } + } + + return ret; +} + +int iris_vb2_buffer_done(struct iris_inst *inst, + struct iris_buffer *buf) +{ + struct vb2_v4l2_buffer *vbuf; + struct vb2_queue *q = NULL; + struct vb2_buffer *iter; + struct vb2_buffer *vb2; + int type, state; + bool found; + + type = v4l2_type_from_driver(buf->type); + if (!type) + return -EINVAL; + + if (type == INPUT_MPLANE) + q = inst->vb2q_src; + else if (type == OUTPUT_MPLANE) + q = inst->vb2q_dst; + if (!q || !q->streaming) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &q->queued_list, queued_entry) { + if (iter->state != VB2_BUF_STATE_ACTIVE) + continue; + if (iter->index == buf->index) { + found = true; + vb2 = iter; + break; + } + } + if (!found) + return -EINVAL; + + if (buf->flags & BUF_FLAG_ERROR) + state = VB2_BUF_STATE_ERROR; + else + state = VB2_BUF_STATE_DONE; + + vbuf = to_vb2_v4l2_buffer(vb2); + vbuf->flags = buf->flags; + vb2->timestamp = buf->timestamp; + vb2->planes[0].bytesused = buf->data_size + vb2->planes[0].data_offset; + vb2_buffer_done(vb2, state); + + return 0; +} + +static int iris_flush_deferred_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + + buffers = iris_get_buffer_list(inst, type); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + if (buf->attr & BUF_ATTR_DEFERRED) { + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + buf->data_size = 0; + iris_vb2_buffer_done(inst, buf); + } + } + } + + return 0; +} + +static int iris_flush_read_only_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct iris_buffer *ro_buf, *dummy; + + if (type != BUF_OUTPUT) + return 0; + + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->attr & BUF_ATTR_READ_ONLY) + continue; + if (ro_buf->attach && ro_buf->sg_table) + dma_buf_unmap_attachment(ro_buf->attach, ro_buf->sg_table, + DMA_BIDIRECTIONAL); + if (ro_buf->attach && ro_buf->dmabuf) + dma_buf_detach(ro_buf->dmabuf, ro_buf->attach); + ro_buf->attach = NULL; + ro_buf->sg_table = NULL; + ro_buf->dmabuf = NULL; + ro_buf->device_addr = 0x0; + list_del_init(&ro_buf->list); + kfree(ro_buf); + } + + return 0; +} + +void iris_destroy_buffers(struct iris_inst *inst) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + + static const enum iris_buffer_type ext_buf_types[] = { + BUF_INPUT, + BUF_OUTPUT, + }; + static const enum iris_buffer_type internal_buf_types[] = { + BUF_BIN, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, + BUF_DPB, + BUF_PERSIST, + }; + int i; + + for (i = 0; i < ARRAY_SIZE(internal_buf_types); i++) { + buffers = iris_get_buffer_list(inst, internal_buf_types[i]); + if (!buffers) + continue; + list_for_each_entry_safe(buf, dummy, &buffers->list, list) + iris_destroy_internal_buffer(inst, buf); + } + + list_for_each_entry_safe(buf, dummy, &inst->buffers.read_only.list, list) { + if (buf->attach && buf->sg_table) + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + if (buf->attach && buf->dmabuf) + dma_buf_detach(buf->dmabuf, buf->attach); + list_del_init(&buf->list); + kfree(buf); + } + + for (i = 0; i < ARRAY_SIZE(ext_buf_types); i++) { + buffers = iris_get_buffer_list(inst, ext_buf_types[i]); + if (!buffers) + continue; + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + if (buf->attach && buf->sg_table) + dma_buf_unmap_attachment(buf->attach, buf->sg_table, + DMA_BIDIRECTIONAL); + if (buf->attach && buf->dmabuf) + dma_buf_detach(buf->dmabuf, buf->attach); + list_del_init(&buf->list); + kfree(buf); + } + } +} + +static int get_num_queued_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct iris_buffers *buffers; + struct iris_buffer *vbuf; + int count = 0; + + if (type == BUF_INPUT) + buffers = &inst->buffers.input; + else if (type == BUF_OUTPUT) + buffers = &inst->buffers.output; + else + return count; + + list_for_each_entry(vbuf, &buffers->list, list) { + if (vbuf->type != type) + continue; + if (!(vbuf->attr & BUF_ATTR_QUEUED)) + continue; + count++; + } + + return count; +} + int session_streamoff(struct iris_inst *inst, u32 plane) { enum signal_session_response signal_type; + enum iris_buffer_type buffer_type; u32 hw_response_timeout_val; struct iris_core *core; + int count = 0; int ret; + if (plane == INPUT_MPLANE) { + signal_type = SIGNAL_CMD_STOP_INPUT; + buffer_type = BUF_INPUT; + } else if (plane == OUTPUT_MPLANE) { + signal_type = SIGNAL_CMD_STOP_OUTPUT; + buffer_type = BUF_OUTPUT; + } else { + return -EINVAL; + } + ret = iris_hfi_stop(inst, plane); if (ret) goto error; @@ -417,14 +775,25 @@ int session_streamoff(struct iris_inst *inst, u32 plane) if (ret) goto error; + /* no more queued buffers after streamoff */ + count = get_num_queued_buffers(inst, buffer_type); + if (count) { + ret = -EINVAL; + goto error; + } + ret = iris_inst_state_change_streamoff(inst, plane); if (ret) goto error; + iris_flush_deferred_buffers(inst, buffer_type); + iris_flush_read_only_buffers(inst, buffer_type); return 0; error: kill_session(inst); + iris_flush_deferred_buffers(inst, buffer_type); + iris_flush_read_only_buffers(inst, buffer_type); return ret; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 47a017d..3a9889e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -24,6 +24,7 @@ bool res_is_less_than(u32 width, u32 height, u32 get_port_info(struct iris_inst *inst, enum plat_inst_cap_type cap_id); enum iris_buffer_type v4l2_type_to_driver(u32 type); +u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type); int get_mbpf(struct iris_inst *inst); int close_session(struct iris_inst *inst); @@ -34,7 +35,6 @@ bool is_split_mode_enabled(struct iris_inst *inst); int signal_session_msg_receipt(struct iris_inst *inst, enum signal_session_response cmd); struct iris_inst *to_instance(struct iris_core *core, u32 session_id); -int session_streamoff(struct iris_inst *inst, u32 plane); u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec); enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec); @@ -43,4 +43,13 @@ enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type); int check_session_supported(struct iris_inst *inst); +struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index); +int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); +int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type); +int iris_vb2_buffer_done(struct iris_inst *inst, + struct iris_buffer *buf); +int iris_release_nonref_buffers(struct iris_inst *inst); +void iris_destroy_buffers(struct iris_inst *inst); +int session_streamoff(struct iris_inst *inst, u32 plane); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index b8785a9..8dafd04 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -76,6 +76,34 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) } } +u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type) +{ + switch (buf_type) { + case HFI_BUFFER_BITSTREAM: + return BUF_INPUT; + case HFI_BUFFER_RAW: + return BUF_OUTPUT; + case HFI_BUFFER_BIN: + return BUF_BIN; + case HFI_BUFFER_ARP: + return BUF_ARP; + case HFI_BUFFER_COMV: + return BUF_COMV; + case HFI_BUFFER_NON_COMV: + return BUF_NON_COMV; + case HFI_BUFFER_LINE: + return BUF_LINE; + case HFI_BUFFER_DPB: + return BUF_DPB; + case HFI_BUFFER_PERSIST: + return BUF_PERSIST; + case HFI_BUFFER_VPSS: + return BUF_VPSS; + default: + return 0; + } +} + u32 get_hfi_codec(struct iris_inst *inst) { switch (inst->codec) { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index cf0960b6..8999c28 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -6,6 +6,7 @@ #ifndef _IRIS_HFI_PACKET_H_ #define _IRIS_HFI_PACKET_H_ +#include "hfi_defines.h" #include "iris_core.h" #include "iris_instance.h" @@ -89,6 +90,7 @@ u32 get_hfi_matrix_coefficients(u32 coefficients); u32 get_v4l2_color_primaries(u32 hfi_primaries); u32 get_v4l2_transfer_char(u32 hfi_characterstics); u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients); +u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type); int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); int hfi_packet_sys_init(struct iris_core *core, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index 4ca9314..b1236dd 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -4,6 +4,7 @@ */ #include "hfi_defines.h" +#include "iris_buffer.h" #include "iris_helpers.h" #include "iris_hfi_packet.h" #include "iris_hfi_response.h" @@ -21,6 +22,11 @@ struct iris_inst_hfi_range { int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt); }; +struct iris_hfi_buffer_handle { + enum hfi_buffer_type type; + int (*handle)(struct iris_inst *inst, struct hfi_buffer *buffer); +}; + struct iris_hfi_packet_handle { enum hfi_buffer_type type; int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt); @@ -44,6 +50,94 @@ static void print_sfr_message(struct iris_core *core) } } +static bool is_valid_hfi_buffer_type(u32 buffer_type) +{ + if (buffer_type != HFI_BUFFER_BITSTREAM && + buffer_type != HFI_BUFFER_RAW && + buffer_type != HFI_BUFFER_BIN && + buffer_type != HFI_BUFFER_ARP && + buffer_type != HFI_BUFFER_COMV && + buffer_type != HFI_BUFFER_NON_COMV && + buffer_type != HFI_BUFFER_LINE && + buffer_type != HFI_BUFFER_DPB && + buffer_type != HFI_BUFFER_PERSIST && + buffer_type != HFI_BUFFER_VPSS) { + return false; + } + + return true; +} + +static bool is_valid_hfi_port(u32 port, u32 buffer_type) +{ + if (port == HFI_PORT_NONE && + buffer_type != HFI_BUFFER_PERSIST) + return false; + + if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW) + return false; + + return true; +} + +static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags) +{ + u32 driver_flags = 0; + + if (inst->hfi_frame_info.picture_type & HFI_PICTURE_IDR) + driver_flags |= BUF_FLAG_KEYFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_P) + driver_flags |= BUF_FLAG_PFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_B) + driver_flags |= BUF_FLAG_BFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_I) + driver_flags |= BUF_FLAG_KEYFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_CRA) + driver_flags |= BUF_FLAG_KEYFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_BLA) + driver_flags |= BUF_FLAG_KEYFRAME; + + if (inst->hfi_frame_info.data_corrupt) + driver_flags |= BUF_FLAG_ERROR; + + if (inst->hfi_frame_info.overflow) + driver_flags |= BUF_FLAG_ERROR; + + return driver_flags; +} + +static bool validate_packet_payload(struct hfi_packet *pkt) +{ + u32 payload_size = 0; + + switch (pkt->payload_info) { + case HFI_PAYLOAD_U32: + case HFI_PAYLOAD_S32: + case HFI_PAYLOAD_Q16: + case HFI_PAYLOAD_U32_ENUM: + case HFI_PAYLOAD_32_PACKED: + payload_size = 4; + break; + case HFI_PAYLOAD_U64: + case HFI_PAYLOAD_S64: + case HFI_PAYLOAD_64_PACKED: + payload_size = 8; + break; + case HFI_PAYLOAD_STRUCTURE: + if (pkt->type == HFI_CMD_BUFFER) + payload_size = sizeof(struct hfi_buffer); + break; + default: + payload_size = 0; + break; + } + + if (pkt->size < sizeof(struct hfi_packet) + payload_size) + return false; + + return true; +} + static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, u32 core_resp_pkt_size) { u32 response_pkt_size = 0; @@ -168,6 +262,293 @@ static int handle_session_close(struct iris_inst *inst, return 0; } +static int handle_read_only_buffer(struct iris_inst *inst, + struct iris_buffer *buf) +{ + struct iris_buffer *ro_buf, *iter; + bool found = false; + + list_for_each_entry(iter, &inst->buffers.read_only.list, list) { + if (iter->device_addr == buf->device_addr) { + found = true; + ro_buf = iter; + break; + } + } + + if (!found) { + ro_buf = kzalloc(sizeof(*ro_buf), GFP_KERNEL); + if (!ro_buf) + return -ENOMEM; + ro_buf->index = -1; + ro_buf->inst = inst; + ro_buf->type = buf->type; + ro_buf->fd = buf->fd; + ro_buf->dmabuf = buf->dmabuf; + ro_buf->device_addr = buf->device_addr; + ro_buf->data_offset = buf->data_offset; + INIT_LIST_HEAD(&ro_buf->list); + list_add_tail(&ro_buf->list, &inst->buffers.read_only.list); + } + ro_buf->attr |= BUF_ATTR_READ_ONLY; + + return 0; +} + +static int handle_non_read_only_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffer *ro_buf; + + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + if (ro_buf->device_addr == buffer->base_address) { + ro_buf->attr &= ~BUF_ATTR_READ_ONLY; + break; + } + } + + return 0; +} + +static int handle_release_output_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffer *buf, *iter; + bool found = false; + + list_for_each_entry(iter, &inst->buffers.read_only.list, list) { + if (iter->device_addr == buffer->base_address && + iter->attr & BUF_ATTR_PENDING_RELEASE) { + found = true; + buf = iter; + break; + } + } + if (!found) + return -EINVAL; + + buf->attr &= ~BUF_ATTR_READ_ONLY; + buf->attr &= ~BUF_ATTR_PENDING_RELEASE; + + return 0; +} + +static int handle_input_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf, *iter; + bool found; + + buffers = iris_get_buffer_list(inst, BUF_INPUT); + if (!buffers) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (iter->index == buffer->index) { + found = true; + buf = iter; + break; + } + } + if (!found) + return -EINVAL; + + if (!(buf->attr & BUF_ATTR_QUEUED)) + return 0; + + buf->data_size = buffer->data_size; + buf->attr &= ~BUF_ATTR_QUEUED; + buf->attr |= BUF_ATTR_DEQUEUED; + + buf->flags = get_driver_buffer_flags(inst, buffer->flags); + + return 0; +} + +static int handle_output_buffer(struct iris_inst *inst, + struct hfi_buffer *hfi_buffer) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf, *iter; + bool found; + int ret = 0; + + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE) + return handle_release_output_buffer(inst, hfi_buffer); + + if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)) + ret = handle_non_read_only_buffer(inst, hfi_buffer); + + buffers = iris_get_buffer_list(inst, BUF_OUTPUT); + if (!buffers) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (!(iter->attr & BUF_ATTR_QUEUED)) + continue; + + found = (iter->index == hfi_buffer->index && + iter->device_addr == hfi_buffer->base_address && + iter->data_offset == hfi_buffer->data_offset); + + if (found) { + buf = iter; + break; + } + } + if (!found) + return 0; + + buf->data_offset = hfi_buffer->data_offset; + buf->data_size = hfi_buffer->data_size; + buf->timestamp = hfi_buffer->timestamp; + + buf->attr &= ~BUF_ATTR_QUEUED; + buf->attr |= BUF_ATTR_DEQUEUED; + + if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) + iris_inst_change_state(inst, IRIS_INST_ERROR); + + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) { + buf->attr |= BUF_ATTR_READ_ONLY; + ret = handle_read_only_buffer(inst, buf); + } else { + buf->attr &= ~BUF_ATTR_READ_ONLY; + } + + buf->flags = get_driver_buffer_flags(inst, hfi_buffer->flags); + + return ret; +} + +static int handle_dequeue_buffers(struct iris_inst *inst) +{ + struct iris_buffers *buffers; + struct iris_buffer *dummy; + struct iris_buffer *buf; + int ret = 0; + int i; + static const enum iris_buffer_type buffer_type[] = { + BUF_INPUT, + BUF_OUTPUT, + }; + + for (i = 0; i < ARRAY_SIZE(buffer_type); i++) { + buffers = iris_get_buffer_list(inst, buffer_type[i]); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + if (buf->attr & BUF_ATTR_DEQUEUED) { + buf->attr &= ~BUF_ATTR_DEQUEUED; + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + ret = iris_vb2_buffer_done(inst, buf); + if (ret) + ret = 0; + } + } + } + } + + return ret; +} + +static int handle_release_internal_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf, *iter; + int ret = 0; + bool found; + + buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(buffer->type)); + if (!buffers) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (iter->device_addr == buffer->base_address) { + found = true; + buf = iter; + break; + } + } + if (!found) + return -EINVAL; + + buf->attr &= ~BUF_ATTR_QUEUED; + + if (buf->attr & BUF_ATTR_PENDING_RELEASE) + ret = iris_destroy_internal_buffer(inst, buf); + + return ret; +} + +static int handle_session_buffer(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + struct hfi_buffer *buffer; + u32 hfi_handle_size = 0; + int i, ret = 0; + const struct iris_hfi_buffer_handle *hfi_handle_arr = NULL; + static const struct iris_hfi_buffer_handle input_hfi_handle[] = { + {HFI_BUFFER_BITSTREAM, handle_input_buffer }, + {HFI_BUFFER_BIN, handle_release_internal_buffer }, + {HFI_BUFFER_COMV, handle_release_internal_buffer }, + {HFI_BUFFER_NON_COMV, handle_release_internal_buffer }, + {HFI_BUFFER_LINE, handle_release_internal_buffer }, + {HFI_BUFFER_PERSIST, handle_release_internal_buffer }, + }; + static const struct iris_hfi_buffer_handle output_hfi_handle[] = { + {HFI_BUFFER_RAW, handle_output_buffer }, + {HFI_BUFFER_DPB, handle_release_internal_buffer }, + }; + + if (pkt->payload_info == HFI_PAYLOAD_NONE) + return 0; + + if (!validate_packet_payload(pkt)) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + return 0; + } + + buffer = (struct hfi_buffer *)((u8 *)pkt + sizeof(*pkt)); + if (!is_valid_hfi_buffer_type(buffer->type)) + return 0; + + if (!is_valid_hfi_port(pkt->port, buffer->type)) + return 0; + + if (pkt->port == HFI_PORT_BITSTREAM) { + hfi_handle_size = ARRAY_SIZE(input_hfi_handle); + hfi_handle_arr = input_hfi_handle; + } else if (pkt->port == HFI_PORT_RAW) { + hfi_handle_size = ARRAY_SIZE(output_hfi_handle); + hfi_handle_arr = output_hfi_handle; + } + + if (!hfi_handle_arr || !hfi_handle_size) + return -EINVAL; + + for (i = 0; i < hfi_handle_size; i++) { + if (hfi_handle_arr[i].type == buffer->type) { + ret = hfi_handle_arr[i].handle(inst, buffer); + if (ret) + return ret; + break; + } + } + + if (i == hfi_handle_size) + return -EINVAL; + + return ret; +} + static int handle_src_change(struct iris_inst *inst, struct hfi_packet *pkt) { @@ -191,6 +572,7 @@ static int handle_session_command(struct iris_inst *inst, static const struct iris_hfi_packet_handle hfi_pkt_handle[] = { {HFI_CMD_OPEN, NULL }, {HFI_CMD_CLOSE, handle_session_close }, + {HFI_CMD_BUFFER, handle_session_buffer }, {HFI_CMD_SETTINGS_CHANGE, handle_src_change }, {HFI_CMD_SUBSCRIBE_MODE, NULL }, }; @@ -386,6 +768,7 @@ static int handle_session_response(struct iris_core *core, { struct hfi_packet *packet; struct iris_inst *inst; + bool dequeue = false; u8 *pkt, *start_pkt; int ret = 0; int i, j; @@ -423,6 +806,7 @@ static int handle_session_response(struct iris_core *core, handle_session_error(inst, packet); if (packet->type > be[i].begin && packet->type < be[i].end) { + dequeue |= (packet->type == HFI_CMD_BUFFER); ret = be[i].handle(inst, packet); if (ret) iris_inst_change_state(inst, IRIS_INST_ERROR); @@ -431,7 +815,15 @@ static int handle_session_response(struct iris_core *core, } } + if (dequeue) { + ret = handle_dequeue_buffers(inst); + if (ret) + goto unlock; + } + memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info)); + +unlock: mutex_unlock(&inst->lock); return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 6553029..04e7dc8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -170,6 +170,14 @@ bool allow_reqbufs(struct iris_inst *inst, u32 type) (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING); } +bool allow_qbuf(struct iris_inst *inst, u32 type) +{ + return (type == INPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) || + (type == INPUT_MPLANE && inst->state == IRIS_INST_STREAMING) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_STREAMING); +} + bool allow_streamon(struct iris_inst *inst, u32 type) { return (type == INPUT_MPLANE && inst->state == IRIS_INST_OPEN) || diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h index 28d5380..47558ed 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -43,6 +43,7 @@ int iris_inst_change_state(struct iris_inst *inst, bool allow_s_fmt(struct iris_inst *inst, u32 type); bool allow_reqbufs(struct iris_inst *inst, u32 type); +bool allow_qbuf(struct iris_inst *inst, u32 type); bool allow_streamon(struct iris_inst *inst, u32 type); bool allow_streamoff(struct iris_inst *inst, u32 type); bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index d599366..f34434fc 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -95,6 +95,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q, int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) { + enum iris_buffer_type buf_type; struct iris_inst *inst; int ret = 0; @@ -132,6 +133,16 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) if (ret) goto error; + buf_type = v4l2_type_to_driver(q->type); + if (!buf_type) { + ret = -EINVAL; + goto error; + } + + ret = queue_deferred_buffers(inst, buf_type); + if (ret) + goto error; + return ret; error: iris_inst_change_state(inst, IRIS_INST_ERROR); @@ -168,9 +179,33 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) iris_inst_change_state(inst, IRIS_INST_ERROR); } +void iris_vb2_buf_queue(struct vb2_buffer *vb2) +{ + struct iris_inst *inst; + int ret; + + inst = vb2_get_drv_priv(vb2->vb2_queue); + if (!inst || !inst->core) + return; + + if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) { + ret = -EINVAL; + goto exit; + } + + ret = vdec_qbuf(inst, vb2); + +exit: + if (ret) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + vb2_buffer_done(vb2, VB2_BUF_STATE_ERROR); + } +} + void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, struct dma_buf *dbuf, unsigned long size) { + struct iris_buffer *ro_buf, *dummy; enum iris_buffer_type buf_type; struct iris_buffers *buffers; struct iris_buffer *iter; @@ -203,6 +238,16 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, buf->inst = inst; buf->dmabuf = dbuf; + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + buf->attach = ro_buf->attach; + ro_buf->attach = NULL; + return buf; + } + } + buf->attach = dma_buf_attach(dbuf, dev); if (IS_ERR(buf->attach)) { buf->attach = NULL; @@ -214,6 +259,7 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, int iris_vb2_map_dmabuf(void *buf_priv) { + struct iris_buffer *ro_buf, *dummy; struct iris_buffer *buf = buf_priv; struct iris_core *core; struct iris_inst *inst; @@ -229,6 +275,17 @@ int iris_vb2_map_dmabuf(void *buf_priv) return -EINVAL; } + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + buf->sg_table = ro_buf->sg_table; + buf->device_addr = ro_buf->device_addr; + ro_buf->sg_table = NULL; + return 0; + } + } + buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL); if (IS_ERR(buf->sg_table)) return -EINVAL; @@ -246,6 +303,7 @@ int iris_vb2_map_dmabuf(void *buf_priv) void iris_vb2_unmap_dmabuf(void *buf_priv) { + struct iris_buffer *ro_buf, *dummy; struct iris_buffer *buf = buf_priv; struct iris_core *core; struct iris_inst *inst; @@ -266,6 +324,17 @@ void iris_vb2_unmap_dmabuf(void *buf_priv) return; } + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + ro_buf->sg_table = buf->sg_table; + buf->sg_table = NULL; + buf->device_addr = 0x0; + return; + } + } + if (buf->attach && buf->sg_table) { dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); buf->sg_table = NULL; @@ -275,6 +344,7 @@ void iris_vb2_unmap_dmabuf(void *buf_priv) void iris_vb2_detach_dmabuf(void *buf_priv) { + struct iris_buffer *ro_buf, *dummy; struct iris_buffer *buf = buf_priv; struct iris_core *core; struct iris_inst *inst; @@ -291,11 +361,22 @@ void iris_vb2_detach_dmabuf(void *buf_priv) buf->sg_table = NULL; } + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + ro_buf->attach = buf->attach; + buf->attach = NULL; + goto exit; + } + } + if (buf->attach && buf->dmabuf) { dma_buf_detach(buf->dmabuf, buf->attach); buf->attach = NULL; } +exit: buf->dmabuf = NULL; buf->inst = NULL; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h index fc0e804..0757f01 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h @@ -13,6 +13,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int sizes[], struct device *alloc_devs[]); int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count); void iris_vb2_stop_streaming(struct vb2_queue *q); +void iris_vb2_buf_queue(struct vb2_buffer *vb2); /* vb2_mem_ops */ void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 0d8ca4b..7eb74c3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -1193,3 +1193,52 @@ int vdec_streamon_output(struct iris_inst *inst) return ret; } + +static int vb2_buffer_to_driver(struct vb2_buffer *vb2, + struct iris_buffer *buf) +{ + struct vb2_v4l2_buffer *vbuf; + + if (!vb2 || !buf) + return -EINVAL; + + vbuf = to_vb2_v4l2_buffer(vb2); + + buf->fd = vb2->planes[0].m.fd; + buf->data_offset = vb2->planes[0].data_offset; + buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset; + buf->buffer_size = vb2->planes[0].length; + buf->timestamp = vb2->timestamp; + buf->flags = vbuf->flags; + buf->attr = 0; + + return 0; +} + +int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) +{ + struct iris_buffer *buf = NULL; + int ret = 0; + + buf = get_driver_buf(inst, vb2->type, vb2->index); + if (!buf) + return -EINVAL; + + ret = vb2_buffer_to_driver(vb2, buf); + if (ret) + return ret; + + if (!allow_qbuf(inst, vb2->type)) { + buf->attr |= BUF_ATTR_DEFERRED; + return 0; + } + + ret = queue_buffer(inst, buf); + if (ret) + return ret; + + if (vb2->type == OUTPUT_MPLANE) + ret = iris_release_nonref_buffers(inst); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index 0722da1..d666c54 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -18,5 +18,6 @@ int vdec_init_src_change_param(struct iris_inst *inst); int vdec_src_change(struct iris_inst *inst); int vdec_streamon_input(struct iris_inst *inst); int vdec_streamon_output(struct iris_inst *inst); +int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 14d0077..0165340 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -252,6 +252,7 @@ int vidc_close(struct file *filp) iris_inst_change_state(inst, IRIS_INST_CLOSE); vidc_vb2_queue_deinit(inst); vidc_v4l2_fh_deinit(inst); + iris_destroy_buffers(inst); vidc_remove_session(inst); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock); @@ -912,6 +913,7 @@ static const struct vb2_ops iris_vb2_ops = { .queue_setup = iris_vb2_queue_setup, .start_streaming = iris_vb2_start_streaming, .stop_streaming = iris_vb2_stop_streaming, + .buf_queue = iris_vb2_buf_queue, }; static struct vb2_mem_ops iris_vb2_mem_ops = { From patchwork Mon Dec 18 11:32:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755545 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C76339AC4; Mon, 18 Dec 2023 11:40:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="SnLij52D" Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsh8Q018842; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Cw+LvLfKYSxI3dwsopUF85tuELonc2iteRwa9ian3SQ=; b=Sn Lij52D31QkAoNWeejGz5FJ+95Xfjk+VZTtH46Uo2DNB5fw8I6MJGvHciEnP2v6Cq 3gl9ZU0B6zbTI+adzBRaNbOO62T3rnbEIkG/BOn+qxjUcSpNY4YkmHNDD0xUwKMU qSC0Kl3VA1W8HOP+gk2/4Xl6E5cYIiJxG9Eee78ETbROu00xIvtpA5r5VLHzYQmp ynyJc9RVxMNwg+GmNl4dotYsrgI4GGiEf7wDKddg6hWK7k3pqhz7Ylb5h70jhT8O GNm2c25z7+BLWnFXGm/GzeJ2KwMWgMWcG1MSi6DbR1HOO5b6g2fAoCh8zhlcOZZO scojmTIm8i08JhRoAYUg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mfe04nq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX6PM029956; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ad-2; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX8rY030072; Mon, 18 Dec 2023 11:33:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX86P030060; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 6480E237A; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 28/34] media: iris: add instance sub states and implement DRC and Drain sequence Date: Mon, 18 Dec 2023 17:02:23 +0530 Message-Id: <1702899149-21321-29-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: sBoqHN7be_eJTBDeL1-DYLxK4PL4_mzC X-Proofpoint-GUID: sBoqHN7be_eJTBDeL1-DYLxK4PL4_mzC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1015 suspectscore=0 malwarescore=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 adultscore=0 impostorscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Implement decoder commands for CMD_START and CMD_STOP. HFI_CMD_DRAIN - set to firmware as part of stop command to initiate drain. HFI_CMD_RESUME - set to firmware as part of start command to resume the buffer processing on specific plane. Also, introduce different sub states for instances. Sub state transitions are defined, based on which they would be allowed/rejected/ignored. - IRIS_INST_SUB_DRAIN - indicates drain is in progress. - IRIS_INST_SUB_DRC - indicates source change is received from firmware and source change event is sent to client. - IRIS_INST_SUB_DRAIN_LAST - indicates last buffer is received from firmware as part of drain sequence . - IRIS_INST_SUB_DRC_LAST - indicates last buffer is received from firmware as part of source change. - IRIS_INST_SUB_INPUT_PAUSE - source change is received form firmware. This indicates that firmware is paused to process any further input frames. - IRIS_INST_SUB_OUTPUT_PAUSE - last buffer is received form firmware as part of drain or drc sequence . This indicates that firmware is paused to process any further output frames. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 19 ++ .../media/platform/qcom/vcodec/iris/iris_buffer.c | 56 +++++- .../media/platform/qcom/vcodec/iris/iris_buffer.h | 3 +- .../media/platform/qcom/vcodec/iris/iris_common.h | 1 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 118 ++++++++++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 +- .../platform/qcom/vcodec/iris/iris_hfi_response.c | 106 ++++++++++- .../platform/qcom/vcodec/iris/iris_instance.h | 2 + .../media/platform/qcom/vcodec/iris/iris_state.c | 211 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_state.h | 24 +++ .../media/platform/qcom/vcodec/iris/iris_vdec.c | 145 ++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 2 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 79 +++++++- 13 files changed, 760 insertions(+), 10 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index 104ef9c..bbe8ca6 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -16,7 +16,10 @@ #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 #define HFI_CMD_STOP 0x01000006 +#define HFI_CMD_DRAIN 0x01000007 +#define HFI_CMD_RESUME 0x01000008 #define HFI_CMD_BUFFER 0x01000009 +#define HFI_CMD_PAUSE 0x01000011 #define HFI_PROP_IMAGE_VERSION 0x03000001 @@ -208,6 +211,20 @@ enum hfi_picture_type { #define HFI_SYSTEM_ERROR_END 0x05FFFFFF +#define HFI_INFORMATION_BEGIN 0x06000000 + +#define HFI_INFO_UNSUPPORTED 0x06000001 + +#define HFI_INFO_DATA_CORRUPT 0x06000002 + +#define HFI_INFO_BUFFER_OVERFLOW 0x06000004 + +#define HFI_INFO_HFI_FLAG_DRAIN_LAST 0x06000006 + +#define HFI_INFO_HFI_FLAG_PSC_LAST 0x06000007 + +#define HFI_INFORMATION_END 0x06FFFFFF + struct hfi_debug_header { u32 size; u32 debug_level; @@ -242,6 +259,8 @@ enum hfi_buffer_firmware_flags { HFI_BUF_FW_FLAG_NONE = 0x00000000, HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001, HFI_BUF_FW_FLAG_READONLY = 0x00000010, + HFI_BUF_FW_FLAG_LAST = 0x10000000, + HFI_BUF_FW_FLAG_PSC_LAST = 0x20000000, }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c index ab25026..f85b52c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -547,7 +547,7 @@ static int iris_release_internal_buffers(struct iris_inst *inst, return ret; } -int iris_release_input_internal_buffers(struct iris_inst *inst) +static int iris_release_input_internal_buffers(struct iris_inst *inst) { int ret = 0; u32 i = 0; @@ -581,3 +581,57 @@ int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst, return ret; } + +int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst) +{ + int ret; + + ret = iris_get_internal_buffers(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = iris_release_input_internal_buffers(inst); + if (ret) + return ret; + + ret = iris_create_input_internal_buffers(inst); + if (ret) + return ret; + + ret = iris_queue_input_internal_buffers(inst); + + return ret; +} + +int iris_alloc_and_queue_additional_dpb_buffers(struct iris_inst *inst) +{ + struct iris_buffers *buffers; + struct iris_buffer *buffer; + int cur_min_count = 0; + int ret; + int i; + + ret = iris_get_internal_buf_info(inst, BUF_DPB); + if (ret) + return ret; + + buffers = iris_get_buffer_list(inst, BUF_DPB); + if (!buffers) + return -EINVAL; + + list_for_each_entry(buffer, &buffers->list, list) + cur_min_count++; + + if (cur_min_count >= buffers->min_count) + return 0; + + for (i = cur_min_count; i < buffers->min_count; i++) { + ret = iris_create_internal_buffers(inst, BUF_DPB); + if (ret) + return ret; + } + + ret = iris_queue_internal_buffers(inst, BUF_DPB); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h index 59c4a10..6ab5f22 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -57,8 +57,9 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer); int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); -int iris_release_input_internal_buffers(struct iris_inst *inst); int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst, enum iris_buffer_type buffer_type); +int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst); +int iris_alloc_and_queue_additional_dpb_buffers(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 5fd96d3..24b322a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -76,6 +76,7 @@ enum iris_buffer_flags { BUF_FLAG_PFRAME = 0x00000010, BUF_FLAG_BFRAME = 0x00000020, BUF_FLAG_ERROR = 0x00000040, + BUF_FLAG_LAST = 0x00100000, }; enum iris_buffer_attributes { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index e0da3be..2b3fa47 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -445,6 +445,124 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst, return ret; } +int iris_hfi_pause(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core; + int ret = 0; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE) + goto unlock; + + ret = hfi_packet_session_command(inst, + HFI_CMD_PAUSE, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + get_hfi_port(plane), + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 payload) +{ + struct iris_core *core; + int ret = 0; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE) + goto unlock; + + ret = hfi_packet_session_command(inst, + HFI_CMD_RESUME, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + get_hfi_port(plane), + inst->session_id, + HFI_PAYLOAD_U32, + &payload, + sizeof(u32)); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_drain(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core; + int ret = 0; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + if (plane != INPUT_MPLANE) + goto unlock; + + ret = hfi_packet_session_command(inst, + HFI_CMD_DRAIN, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + get_hfi_port(plane), + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + irqreturn_t iris_hfi_isr(int irq, void *data) { disable_irq_nosync(irq); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index f054c2d..ab479a9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -25,7 +25,9 @@ int iris_hfi_session_set_default_header(struct iris_inst *inst); int iris_hfi_start(struct iris_inst *inst, u32 plane); int iris_hfi_stop(struct iris_inst *inst, u32 plane); - +int iris_hfi_drain(struct iris_inst *inst, u32 plane); +int iris_hfi_pause(struct iris_inst *inst, u32 plane); +int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 cmd); int iris_hfi_queue_buffer(struct iris_inst *inst, struct iris_buffer *buffer); int iris_hfi_release_buffer(struct iris_inst *inst, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index b1236dd..3548f9d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -103,6 +103,10 @@ static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags) if (inst->hfi_frame_info.overflow) driver_flags |= BUF_FLAG_ERROR; + if (hfi_flags & HFI_BUF_FW_FLAG_LAST || + hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST) + driver_flags |= BUF_FLAG_LAST; + return driver_flags; } @@ -187,6 +191,46 @@ static int validate_hdr_packet(struct iris_core *core, struct hfi_header *hdr) return ret; } +static int handle_session_info(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + struct iris_core *core; + int ret = 0; + char *info; + + core = inst->core; + + switch (pkt->type) { + case HFI_INFO_UNSUPPORTED: + info = "unsupported"; + break; + case HFI_INFO_DATA_CORRUPT: + info = "data corrupt"; + inst->hfi_frame_info.data_corrupt = 1; + break; + case HFI_INFO_BUFFER_OVERFLOW: + info = "buffer overflow"; + inst->hfi_frame_info.overflow = 1; + break; + case HFI_INFO_HFI_FLAG_DRAIN_LAST: + info = "drain last flag"; + ret = iris_inst_sub_state_change_drain_last(inst); + break; + case HFI_INFO_HFI_FLAG_PSC_LAST: + info = "drc last flag"; + ret = iris_inst_sub_state_change_drc_last(inst); + break; + default: + info = "unknown"; + break; + } + + dev_dbg(core->dev, "session info received %#x: %s\n", + pkt->type, info); + + return ret; +} + static int handle_session_error(struct iris_inst *inst, struct hfi_packet *pkt) { @@ -375,9 +419,21 @@ static int handle_output_buffer(struct iris_inst *inst, bool found; int ret = 0; + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) { + ret = iris_inst_sub_state_change_drain_last(inst); + if (ret) + return ret; + } + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE) return handle_release_output_buffer(inst, hfi_buffer); + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) { + ret = iris_inst_sub_state_change_drc_last(inst); + if (ret) + return ret; + } + if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)) ret = handle_non_read_only_buffer(inst, hfi_buffer); @@ -488,6 +544,26 @@ static int handle_release_internal_buffer(struct iris_inst *inst, return ret; } +static int handle_session_stop(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + int ret = 0; + enum signal_session_response signal_type = -1; + + if (pkt->port == HFI_PORT_RAW) { + signal_type = SIGNAL_CMD_STOP_OUTPUT; + ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE); + } else if (pkt->port == HFI_PORT_BITSTREAM) { + signal_type = SIGNAL_CMD_STOP_INPUT; + ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE); + } + + if (signal_type != -1) + signal_session_msg_receipt(inst, signal_type); + + return ret; +} + static int handle_session_buffer(struct iris_inst *inst, struct hfi_packet *pkt) { @@ -549,18 +625,30 @@ static int handle_session_buffer(struct iris_inst *inst, return ret; } +static int handle_session_drain(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + int ret = 0; + + if (inst->sub_state & IRIS_INST_SUB_DRAIN) + ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_INPUT_PAUSE); + + return ret; +} + static int handle_src_change(struct iris_inst *inst, struct hfi_packet *pkt) { - int ret = 0; + int ret; - if (pkt->port == HFI_PORT_BITSTREAM) - ret = vdec_src_change(inst); - else if (pkt->port == HFI_PORT_RAW) - ret = 0; + if (pkt->port != HFI_PORT_BITSTREAM) + return 0; + ret = iris_inst_sub_state_change_drc(inst); if (ret) - iris_inst_change_state(inst, IRIS_INST_ERROR); + return ret; + + ret = vdec_src_change(inst); return ret; } @@ -572,9 +660,14 @@ static int handle_session_command(struct iris_inst *inst, static const struct iris_hfi_packet_handle hfi_pkt_handle[] = { {HFI_CMD_OPEN, NULL }, {HFI_CMD_CLOSE, handle_session_close }, + {HFI_CMD_START, NULL }, + {HFI_CMD_STOP, handle_session_stop }, + {HFI_CMD_DRAIN, handle_session_drain }, {HFI_CMD_BUFFER, handle_session_buffer }, {HFI_CMD_SETTINGS_CHANGE, handle_src_change }, {HFI_CMD_SUBSCRIBE_MODE, NULL }, + {HFI_CMD_PAUSE, NULL }, + {HFI_CMD_RESUME, NULL }, }; for (i = 0; i < ARRAY_SIZE(hfi_pkt_handle); i++) { @@ -774,6 +867,7 @@ static int handle_session_response(struct iris_core *core, int i, j; static const struct iris_inst_hfi_range be[] = { {HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, handle_session_error }, + {HFI_INFORMATION_BEGIN, HFI_INFORMATION_END, handle_session_info }, {HFI_PROP_BEGIN, HFI_PROP_END, handle_session_property }, {HFI_CMD_BEGIN, HFI_CMD_END, handle_session_command }, }; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index cddf143..50154bf 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -39,6 +39,7 @@ * @buffers: structure of buffer info * @fw_min_count: minimnum count of buffers needed by fw * @state: instance state + * @sub_state: instance sub state * @ipsc_properties_set: boolean to set ipsc properties to fw * @opsc_properties_set: boolean to set opsc properties to fw * @hfi_frame_info: structure of frame info @@ -71,6 +72,7 @@ struct iris_inst { struct iris_buffers_info buffers; u32 fw_min_count; enum iris_inst_state state; + enum iris_inst_sub_state sub_state; bool ipsc_properties_set; bool opsc_properties_set; struct iris_hfi_frame_info hfi_frame_info; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 04e7dc8..952ba2a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -239,3 +239,214 @@ int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane) return iris_inst_change_state(inst, new_state); } + +struct iris_inst_sub_state_change_allow { + enum iris_inst_state state; + enum state_change allow; + u32 sub_state_mask; +}; + +static int iris_inst_allow_sub_state(struct iris_inst *inst, enum iris_inst_sub_state sub_state) +{ + int i; + static struct iris_inst_sub_state_change_allow sub_state_allow[] = { + {IRIS_INST_OPEN, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC | + IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_DRC_LAST | + IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_INPUT_PAUSE | + IRIS_INST_SUB_OUTPUT_PAUSE}, + + {IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC_LAST | + IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_OUTPUT_PAUSE}, + + {IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC | + IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_INPUT_PAUSE }, + + {IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC | + IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_INPUT_PAUSE }, + + {IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC_LAST | + IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_OUTPUT_PAUSE}, + + {IRIS_INST_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC | + IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_DRC_LAST | + IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_INPUT_PAUSE | + IRIS_INST_SUB_OUTPUT_PAUSE}, + + {IRIS_INST_CLOSE, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC | + IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_DRC_LAST | + IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_INPUT_PAUSE | + IRIS_INST_SUB_OUTPUT_PAUSE}, + + {IRIS_INST_ERROR, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC | + IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_DRC_LAST | + IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_INPUT_PAUSE | + IRIS_INST_SUB_OUTPUT_PAUSE}, + }; + + if (!sub_state) + return 0; + + for (i = 0; i < ARRAY_SIZE(sub_state_allow); i++) { + if (sub_state_allow[i].state != inst->state) + continue; + + if (sub_state_allow[i].allow == STATE_CHANGE_DISALLOW) + if (sub_state_allow[i].sub_state_mask & sub_state) { + dev_dbg(inst->core->dev, "state %d with disallowed sub state %x\n", + inst->state, sub_state); + return -EINVAL; + } + + if (sub_state_allow[i].allow == STATE_CHANGE_ALLOW) { + if ((sub_state_allow[i].sub_state_mask & sub_state) != sub_state) + return -EINVAL; + } + } + + return 0; +} + +int iris_inst_change_sub_state(struct iris_inst *inst, + enum iris_inst_sub_state clear_sub_state, + enum iris_inst_sub_state set_sub_state) +{ + enum iris_inst_sub_state prev_sub_state; + int ret; + + if (IS_SESSION_ERROR(inst)) + return 0; + + if (!clear_sub_state && !set_sub_state) + return 0; + + if ((clear_sub_state & set_sub_state) || + set_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE || + clear_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE) + return -EINVAL; + + prev_sub_state = inst->sub_state; + + ret = iris_inst_allow_sub_state(inst, set_sub_state); + if (ret) + return ret; + + inst->sub_state |= set_sub_state; + inst->sub_state &= ~clear_sub_state; + + if (inst->sub_state != prev_sub_state) + dev_dbg(inst->core->dev, "state %d and sub state changed to %x\n", + inst->sub_state, inst->sub_state); + + return 0; +} + +int iris_inst_sub_state_change_drc(struct iris_inst *inst) +{ + enum iris_inst_sub_state set_sub_state = 0; + + if (inst->sub_state & IRIS_INST_SUB_DRC) + return STATE_CHANGE_DISALLOW; + + if (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_OPEN) + set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; + else + set_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_INPUT_PAUSE; + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} + +int iris_inst_sub_state_change_drain_last(struct iris_inst *inst) +{ + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; + + if (inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) + return STATE_CHANGE_DISALLOW; + + if (!(inst->sub_state & IRIS_INST_SUB_DRAIN) || + !(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) + return STATE_CHANGE_DISALLOW; + + set_sub_state = IRIS_INST_SUB_DRAIN_LAST | IRIS_INST_SUB_OUTPUT_PAUSE; + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} + +int iris_inst_sub_state_change_drc_last(struct iris_inst *inst) +{ + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; + + if (inst->sub_state & IRIS_INST_SUB_DRC_LAST) + return STATE_CHANGE_DISALLOW; + + if (!(inst->sub_state & IRIS_INST_SUB_DRC) || + !(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) + return STATE_CHANGE_DISALLOW; + + set_sub_state = IRIS_INST_SUB_DRC_LAST | IRIS_INST_SUB_OUTPUT_PAUSE; + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} + +int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane) +{ + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; + + if (plane == INPUT_MPLANE) { + if (inst->sub_state & IRIS_INST_SUB_DRC && + !(inst->sub_state & IRIS_INST_SUB_DRC_LAST)) + return STATE_CHANGE_DISALLOW; + + if (inst->sub_state & IRIS_INST_SUB_DRAIN && + !(inst->sub_state & IRIS_INST_SUB_DRAIN_LAST)) + return STATE_CHANGE_DISALLOW; + + set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; + } else { + set_sub_state = IRIS_INST_SUB_OUTPUT_PAUSE; + } + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} + +bool is_drc_pending(struct iris_inst *inst) +{ + return inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST; +} + +bool is_drain_pending(struct iris_inst *inst) +{ + return inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST; +} + +bool allow_cmd(struct iris_inst *inst, u32 cmd) +{ + if (cmd == V4L2_DEC_CMD_START) { + if (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_OUTPUT_STREAMING || + inst->state == IRIS_INST_STREAMING) + if (is_drc_pending(inst) || is_drain_pending(inst)) + return true; + } else if (cmd == V4L2_DEC_CMD_STOP) { + if (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_STREAMING) + if (inst->sub_state != IRIS_INST_SUB_DRAIN) + return true; + } + + return false; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h index 47558ed..76db8f5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -25,6 +25,19 @@ enum iris_inst_state { IRIS_INST_ERROR, }; +#define IRIS_INST_SUB_NONE 0 +#define IRIS_INST_SUB_STATES 6 +#define IRIS_INST_MAX_SUB_STATE_VALUE ((1 << IRIS_INST_SUB_STATES) - 1) + +enum iris_inst_sub_state { + IRIS_INST_SUB_DRAIN = BIT(0), + IRIS_INST_SUB_DRC = BIT(1), + IRIS_INST_SUB_DRAIN_LAST = BIT(2), + IRIS_INST_SUB_DRC_LAST = BIT(3), + IRIS_INST_SUB_INPUT_PAUSE = BIT(4), + IRIS_INST_SUB_OUTPUT_PAUSE = BIT(5), +}; + enum state_change { STATE_CHANGE_ALLOW, STATE_CHANGE_DISALLOW, @@ -40,6 +53,9 @@ int iris_change_core_state(struct iris_core *core, int iris_inst_change_state(struct iris_inst *inst, enum iris_inst_state request_state); +int iris_inst_change_sub_state(struct iris_inst *inst, + enum iris_inst_sub_state clear_sub_state, + enum iris_inst_sub_state set_sub_state); bool allow_s_fmt(struct iris_inst *inst, u32 type); bool allow_reqbufs(struct iris_inst *inst, u32 type); @@ -51,4 +67,12 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id); int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane); int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane); +int iris_inst_sub_state_change_drc(struct iris_inst *inst); +int iris_inst_sub_state_change_drain_last(struct iris_inst *inst); +int iris_inst_sub_state_change_drc_last(struct iris_inst *inst); +int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane); +bool is_drc_pending(struct iris_inst *inst); +bool is_drain_pending(struct iris_inst *inst); +bool allow_cmd(struct iris_inst *inst, u32 cmd); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 7eb74c3..7884ba6 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -1066,16 +1066,35 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst) static int process_streamon_input(struct iris_inst *inst) { + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; int ret; ret = iris_hfi_start(inst, INPUT_MPLANE); if (ret) return ret; + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0); + if (ret) + return ret; + } + + if (inst->sub_state & IRIS_INST_SUB_DRC || + inst->sub_state & IRIS_INST_SUB_DRAIN) { + if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) { + ret = iris_hfi_pause(inst, INPUT_MPLANE); + if (ret) + return ret; + set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; + } + } + ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE); if (ret) return ret; + ret = iris_inst_change_sub_state(inst, 0, set_sub_state); + return ret; } @@ -1127,16 +1146,51 @@ int vdec_streamon_input(struct iris_inst *inst) static int process_streamon_output(struct iris_inst *inst) { + enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; + bool drain_pending = false; int ret; + if (inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST) + clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_alloc_and_queue_input_int_bufs(inst); + if (ret) + return ret; + ret = set_stage(inst, STAGE); + if (ret) + return ret; + ret = set_pipe(inst, PIPE); + if (ret) + return ret; + } + + drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST; + + if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) { + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + } + ret = iris_hfi_start(inst, OUTPUT_MPLANE); if (ret) return ret; + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE); if (ret) return ret; + ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); + return ret; } @@ -1242,3 +1296,94 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) return ret; } + +static int process_resume(struct iris_inst *inst) +{ + enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; + int ret; + + if (inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST) { + clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { + ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + } + } else if (inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) { + clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST; + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { + ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + } + } + + ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); + + return ret; +} + +int vdec_start_cmd(struct iris_inst *inst) +{ + int ret; + + vb2_clear_last_buffer_dequeued(inst->vb2q_dst); + + if (inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST && + inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_alloc_and_queue_input_int_bufs(inst); + if (ret) + return ret; + + ret = set_stage(inst, STAGE); + if (ret) + return ret; + + ret = set_pipe(inst, PIPE); + if (ret) + return ret; + } + + ret = iris_alloc_and_queue_additional_dpb_buffers(inst); + if (ret) + return ret; + + ret = queue_deferred_buffers(inst, BUF_OUTPUT); + if (ret) + return ret; + + ret = process_resume(inst); + + return ret; +} + +int vdec_stop_cmd(struct iris_inst *inst) +{ + int ret; + + ret = iris_hfi_drain(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_DRAIN); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index d666c54..974b95e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -19,5 +19,7 @@ int vdec_src_change(struct iris_inst *inst); int vdec_streamon_input(struct iris_inst *inst); int vdec_streamon_output(struct iris_inst *inst); int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2); +int vdec_start_cmd(struct iris_inst *inst); +int vdec_stop_cmd(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 0165340..d37ef04 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -901,7 +901,82 @@ static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection * return ret; } -static const struct v4l2_file_operations v4l2_file_ops = { +static int vidc_try_dec_cmd(struct file *filp, void *fh, + struct v4l2_decoder_cmd *dec) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !dec) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (dec->cmd != V4L2_DEC_CMD_STOP && dec->cmd != V4L2_DEC_CMD_START) { + ret = -EINVAL; + goto unlock; + } + dec->flags = 0; + if (dec->cmd == V4L2_DEC_CMD_STOP) { + dec->stop.pts = 0; + } else if (dec->cmd == V4L2_DEC_CMD_START) { + dec->start.speed = 0; + dec->start.format = V4L2_DEC_START_FMT_NONE; + } + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_dec_cmd(struct file *filp, void *fh, + struct v4l2_decoder_cmd *dec) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !dec) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (dec->cmd != V4L2_DEC_CMD_START && + dec->cmd != V4L2_DEC_CMD_STOP) { + ret = -EINVAL; + goto unlock; + } + + if (inst->state == IRIS_INST_OPEN) + goto unlock; + + if (!allow_cmd(inst, dec->cmd)) { + ret = -EBUSY; + goto unlock; + } + + if (dec->cmd == V4L2_DEC_CMD_START) + ret = vdec_start_cmd(inst); + else if (dec->cmd == V4L2_DEC_CMD_STOP) + ret = vdec_stop_cmd(inst); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static struct v4l2_file_operations v4l2_file_ops = { .owner = THIS_MODULE, .open = vidc_open, .release = vidc_close, @@ -950,6 +1025,8 @@ static const struct v4l2_ioctl_ops v4l2_ioctl_ops = { .vidioc_subscribe_event = vidc_subscribe_event, .vidioc_unsubscribe_event = vidc_unsubscribe_event, .vidioc_g_selection = vidc_g_selection, + .vidioc_try_decoder_cmd = vidc_try_dec_cmd, + .vidioc_decoder_cmd = vidc_dec_cmd, }; int init_ops(struct iris_core *core) From patchwork Mon Dec 18 11:32:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757234 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 345B41CAB2; Mon, 18 Dec 2023 11:40:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="j6MQRq59" Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBHw62032490; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=VvqHhZOTTYvc/rwKV2zir+K0UrIHVwkgCzFSeUnMfF4=; b=j6 MQRq590+BQX8ZYbrtTHRCXKoEqT1ZmCSXa5c2lg9PYt2gBVuDYEoj3OsbUTinr6H cxWOaDyLiXUfuueqPaJih7mFUlo75OYk4hxZ4igmmoMcXRv3HkSTInrD8UzQOzrf Pk9E7AjZ9AOmp95O4kpTov8Ob0kUmZu+wRAj56jLBR1OqfRpgsykmkXYgplo9wqQ SojX3JyLNgIrJQGBchOg9qxUuvquX0ZGbtDF0w7zOk8IzTaPAnOrqUZEXmNOwsEA TfRwGtewUuOsHaPe8cXvSsjHqha6EKcNFcqaihqMY7aOJleI6WVLNWGQBOOb4Taq WvUfo/tBZy6NE/UQE+eA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mfe04nr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0Hm004853; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ae-2; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBa9rU000378; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8m5030064; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 6725A2383; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 29/34] media: iris: implement power management Date: Mon, 18 Dec 2023 17:02:24 +0530 Message-Id: <1702899149-21321-30-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: bTHmOPnAKerMh8myzEXcM0HtyEWAWpNV X-Proofpoint-GUID: bTHmOPnAKerMh8myzEXcM0HtyEWAWpNV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1015 suspectscore=0 malwarescore=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 adultscore=0 impostorscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Hardware specific power sequence include programming specific sequence of registers. Implements this sequence for iris3. Introduce video device suspend and resume using runtime PM. Also, implement iris3 specific calculation for clock and bus bandwidth which depends on hardware configuration, codec format, resolution and frame rate. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 2 + .../media/platform/qcom/vcodec/iris/hfi_defines.h | 7 + .../media/platform/qcom/vcodec/iris/iris_common.h | 7 + .../media/platform/qcom/vcodec/iris/iris_core.h | 4 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 66 +++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 175 +++++++- drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 5 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 48 +++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 4 + .../platform/qcom/vcodec/iris/iris_hfi_response.c | 2 + .../platform/qcom/vcodec/iris/iris_instance.h | 10 + .../media/platform/qcom/vcodec/iris/iris_power.c | 173 ++++++++ .../media/platform/qcom/vcodec/iris/iris_power.h | 15 + .../media/platform/qcom/vcodec/iris/iris_probe.c | 118 +++++- drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 42 +- .../media/platform/qcom/vcodec/iris/iris_vdec.c | 7 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 36 +- .../platform/qcom/vcodec/iris/platform_common.h | 18 + .../media/platform/qcom/vcodec/iris/resources.c | 280 +++++++++++++ .../media/platform/qcom/vcodec/iris/resources.h | 8 + .../media/platform/qcom/vcodec/iris/vpu_common.c | 87 ++++ .../media/platform/qcom/vcodec/iris/vpu_common.h | 10 + .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 454 ++++++++++++++++++--- .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 97 +++++ .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 + 26 files changed, 1614 insertions(+), 78 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 7e3d9f1..4c8f8f6 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -13,10 +13,12 @@ iris-objs += iris_probe.o \ iris_hfi_response.o \ iris_hfi_packet.o \ iris_buffer.o \ + iris_power.o \ resources.o \ vpu_common.o \ vpu_iris3.o \ vpu_iris3_buffer.o \ + vpu_iris3_power.o \ platform_common.o \ platform_sm8550.o diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index bbe8ca6..872674e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -12,6 +12,7 @@ #define HFI_CMD_BEGIN 0x01000000 #define HFI_CMD_INIT 0x01000001 +#define HFI_CMD_POWER_COLLAPSE 0x01000002 #define HFI_CMD_OPEN 0x01000003 #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 @@ -46,10 +47,16 @@ enum hfi_property_mode_type { #define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 +#define HFI_TRUE 0x00000001 + +#define HFI_FALSE 0x00000000 + #define HFI_PROP_BEGIN 0x03000000 #define HFI_PROP_IMAGE_VERSION 0x03000001 +#define HFI_PROP_INTRA_FRAME_POWER_COLLAPSE 0x03000002 + enum hfi_codec_type { HFI_CODEC_DECODE_AVC = 1, HFI_CODEC_ENCODE_AVC = 2, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 24b322a..3ab4767 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -28,6 +28,8 @@ #define MAX_DPB_LIST_ARRAY_SIZE (16 * 4) #define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4) +#define INPUT_TIMER_LIST_SIZE 30 + enum codec_type { H264 = BIT(0), HEVC = BIT(1), @@ -128,4 +130,9 @@ struct iris_hfi_frame_info { u32 overflow; }; +struct iris_input_timer { + struct list_head list; + u64 time_us; +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 6452ea7..50553d2 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -101,6 +101,10 @@ struct iris_core { struct list_head instances; u32 intr_status; char fw_version[IRIS_VERSION_LENGTH]; + struct mutex pm_lock; /* lock for pm operations */ + u32 skip_pc_count; + bool power_enabled; + struct iris_core_power power; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index a5b8aef..395a189 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -3,12 +3,15 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "hfi_defines.h" #include "iris_core.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vidc.h" int check_core_lock(struct iris_core *core) @@ -479,6 +482,8 @@ int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_typ if (!buffers) return -EINVAL; + iris_scale_power(inst); + list_for_each_entry(buf, &buffers->list, list) { if (!(buf->attr & BUF_ATTR_DEFERRED)) continue; @@ -775,6 +780,9 @@ int session_streamoff(struct iris_inst *inst, u32 plane) if (ret) goto error; + if (plane == INPUT_MPLANE) + iris_flush_input_timer(inst); + /* no more queued buffers after streamoff */ count = get_num_queued_buffers(inst, buffer_type); if (count) { @@ -797,3 +805,61 @@ int session_streamoff(struct iris_inst *inst, u32 plane) return ret; } + +int iris_pm_get(struct iris_core *core) +{ + int ret; + + mutex_lock(&core->pm_lock); + ret = pm_runtime_resume_and_get(core->dev); + mutex_unlock(&core->pm_lock); + + return ret; +} + +int iris_pm_put(struct iris_core *core, bool autosuspend) +{ + int ret; + + mutex_lock(&core->pm_lock); + + if (autosuspend) + ret = pm_runtime_put_autosuspend(core->dev); + else + ret = pm_runtime_put_sync(core->dev); + if (ret > 0) + ret = 0; + + mutex_unlock(&core->pm_lock); + + return ret; +} + +int iris_pm_get_put(struct iris_core *core) +{ + int ret = 0; + + mutex_lock(&core->pm_lock); + + if (pm_runtime_suspended(core->dev)) { + ret = pm_runtime_resume_and_get(core->dev); + if (ret < 0) + goto error; + + ret = pm_runtime_put_autosuspend(core->dev); + } + if (ret > 0) + ret = 0; + +error: + mutex_unlock(&core->pm_lock); + + return ret; +} + +void iris_pm_touch(struct iris_core *core) +{ + mutex_lock(&core->pm_lock); + pm_runtime_mark_last_busy(core->dev); + mutex_unlock(&core->pm_lock); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 3a9889e..c628b2e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -51,5 +51,9 @@ int iris_vb2_buffer_done(struct iris_inst *inst, int iris_release_nonref_buffers(struct iris_inst *inst); void iris_destroy_buffers(struct iris_inst *inst); int session_streamoff(struct iris_inst *inst, u32 plane); +int iris_pm_get(struct iris_core *core); +int iris_pm_put(struct iris_core *core, bool autosuspend); +int iris_pm_get_put(struct iris_core *core); +void iris_pm_touch(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 2b3fa47..2850fd5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -67,6 +67,41 @@ static bool validate_session(struct iris_core *core, return valid; } +static int iris_power_off(struct iris_core *core) +{ + int ret; + + if (!core->power_enabled) + return 0; + + ret = call_vpu_op(core, power_off, core); + if (ret) { + dev_err(core->dev, "Failed to power off, err: %d\n", ret); + return ret; + } + core->power_enabled = false; + + return ret; +} + +static int iris_power_on(struct iris_core *core) +{ + int ret; + + if (core->power_enabled) + return 0; + + ret = call_vpu_op(core, power_on, core); + if (ret) { + dev_err(core->dev, "Failed to power on, err: %d\n", ret); + return ret; + } + + core->power_enabled = true; + + return ret; +} + static int sys_init(struct iris_core *core) { int ret; @@ -113,20 +148,24 @@ int iris_hfi_core_init(struct iris_core *core) if (ret) goto error; + ret = iris_power_on(core); + if (ret) + goto error; + core->use_tz = use_tz(core->dev); if (!core->use_tz) - goto error; + goto error_power_off; pas_id = core->platform_data->pas_id; fw_name = core->platform_data->fwname; ret = load_fw(core->dev, fw_name, &mem_phys, &mem_size, pas_id, core->use_tz); if (ret) - goto error; + goto error_power_off; ret = auth_reset_fw(pas_id); if (ret) - goto error; + goto error_power_off; cp_start = core->cap[CP_START].value; cp_size = core->cap[CP_SIZE].value; @@ -135,22 +174,24 @@ int iris_hfi_core_init(struct iris_core *core) ret = protect_secure_region(cp_start, cp_size, cp_nonpixel_start, cp_nonpixel_size, pas_id); if (ret) - goto error; + goto error_power_off; ret = call_vpu_op(core, boot_firmware, core); if (ret) - goto error; + goto error_power_off; ret = sys_init(core); if (ret) - goto error; + goto error_power_off; ret = sys_image_version(core); if (ret) - goto error; + goto error_power_off; return ret; +error_power_off: + iris_power_off(core); error: dev_err(core->dev, "%s(): failed with ret %d\n", __func__, ret); @@ -170,6 +211,8 @@ int iris_hfi_core_deinit(struct iris_core *core) unload_fw(core->platform_data->pas_id); + iris_power_off(core); + return ret; } @@ -573,16 +616,24 @@ irqreturn_t iris_hfi_isr(int irq, void *data) irqreturn_t iris_hfi_isr_handler(int irq, void *data) { struct iris_core *core = data; + int ret; if (!core) return IRQ_NONE; + ret = iris_pm_get(core); + if (ret) + goto exit; + mutex_lock(&core->lock); call_vpu_op(core, clear_interrupt, core); mutex_unlock(&core->lock); __response_handler(core); + iris_pm_put(core, true); + +exit: if (!call_vpu_op(core, watchdog, core, core->intr_status)) enable_irq(irq); @@ -701,3 +752,113 @@ int iris_hfi_release_buffer(struct iris_inst *inst, return ret; } + +int prepare_pc(struct iris_core *core) +{ + int ret; + + ret = hfi_packet_sys_pc_prep(core, core->packet, core->packet_size); + if (ret) + goto err_pc_prep; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + if (ret) + goto err_pc_prep; + + return ret; + +err_pc_prep: + dev_err(core->dev, "Failed to prepare iris for power off\n"); + + return ret; +} + +int iris_hfi_pm_suspend(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core_in_valid_state(core)) + return -EINVAL; + + if (!core->power_enabled) + return 0; + + if (core->skip_pc_count >= MAX_PC_SKIP_COUNT) { + dev_err(core->dev, "Failed to PC for %d times\n", core->skip_pc_count); + core->skip_pc_count = 0; + iris_change_core_state(core, IRIS_CORE_ERROR); + iris_core_deinit_locked(core); + return -EINVAL; + } + + iris_flush_debug_queue(core, core->packet, core->packet_size); + + ret = call_vpu_op(core, prepare_pc, core); + if (ret) { + core->skip_pc_count++; + iris_pm_touch(core); + return -EAGAIN; + } + + ret = set_hw_state(false); + if (ret) + return ret; + + ret = iris_power_off(core); + if (ret) + return ret; + + core->skip_pc_count = 0; + + return ret; +} + +int iris_hfi_pm_resume(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core_in_valid_state(core)) + return -EINVAL; + + if (core->power_enabled) + return 0; + + ret = iris_power_on(core); + if (ret) + goto error; + + ret = set_hw_state(true); + if (ret) + goto err_power_off; + + ret = call_vpu_op(core, boot_firmware, core); + if (ret) + goto err_suspend_hw; + + ret = hfi_packet_sys_interframe_powercollapse(core, core->packet, core->packet_size); + if (ret) + goto err_suspend_hw; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + if (ret) + goto err_suspend_hw; + + return ret; + +err_suspend_hw: + set_hw_state(false); +err_power_off: + iris_power_off(core); +error: + dev_err(core->dev, "Failed to Resume\n"); + + return -EBUSY; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index ab479a9..465bfc5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -9,6 +9,8 @@ #include "iris_instance.h" #include "iris_core.h" +#define MAX_PC_SKIP_COUNT 10 + int iris_hfi_core_init(struct iris_core *core); int iris_hfi_core_deinit(struct iris_core *core); int iris_hfi_session_open(struct iris_inst *inst); @@ -32,6 +34,9 @@ int iris_hfi_queue_buffer(struct iris_inst *inst, struct iris_buffer *buffer); int iris_hfi_release_buffer(struct iris_inst *inst, struct iris_buffer *buffer); +int iris_hfi_pm_suspend(struct iris_core *core); +int iris_hfi_pm_resume(struct iris_core *core); +int prepare_pc(struct iris_core *core); irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index 8dafd04..fab206cf 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -622,3 +622,51 @@ int hfi_packet_session_property(struct iris_inst *inst, return ret; } + +int hfi_packet_sys_interframe_powercollapse(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + u32 payload = 0; + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0 /*session_id*/, + core->header_id++); + if (ret) + return ret; + + payload = HFI_TRUE; + + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_INTRA_FRAME_POWER_COLLAPSE, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + return ret; +} + +int hfi_packet_sys_pc_prep(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0 /*session_id*/, + core->header_id++); + if (ret) + return ret; + + ret = hfi_create_packet(pkt, pkt_size, + HFI_CMD_POWER_COLLAPSE, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_NONE, + HFI_PORT_NONE, + core->packet_id++, + NULL, 0); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index 8999c28..849b585 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -104,5 +104,9 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, int hfi_packet_session_property(struct iris_inst *inst, u32 pkt_type, u32 flags, u32 port, u32 payload_type, void *payload, u32 payload_size); +int hfi_packet_sys_interframe_powercollapse(struct iris_core *core, + u8 *pkt, u32 pkt_size); +int hfi_packet_sys_pc_prep(struct iris_core *core, + u8 *pkt, u32 pkt_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index 3548f9d..a4d5d9c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -928,6 +928,8 @@ static int handle_response(struct iris_core *core, void *response) struct hfi_header *hdr; int ret; + iris_pm_touch(core); + hdr = (struct hfi_header *)response; ret = validate_hdr_packet(core, hdr); if (ret) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 50154bf..3c85e78 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -47,6 +47,11 @@ * @dst_subcr_params: subscription params to fw on output port * @dpb_list_payload: array of dpb buffers * @once_per_session_set: boolean to set once per session property + * @max_rate: max input rate + * @max_input_data_size: max size of input data + * @power: structure of power info + * @bus_data: structure of bus data + * @input_timer_list: list head of input timer */ struct iris_inst { @@ -80,6 +85,11 @@ struct iris_inst { struct subscription_params dst_subcr_params; u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE]; bool once_per_session_set; + u32 max_rate; + u32 max_input_data_size; + struct iris_inst_power power; + struct bus_vote_data bus_data; + struct list_head input_timer_list; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.c b/drivers/media/platform/qcom/vcodec/iris/iris_power.c new file mode 100644 index 0000000..77439e3 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_power.h" +#include "iris_helpers.h" +#include "resources.h" + +static int iris_set_buses(struct iris_inst *inst) +{ + struct iris_inst *instance; + struct iris_core *core; + u64 total_bw_ddr = 0; + int ret; + + core = inst->core; + + mutex_lock(&core->lock); + list_for_each_entry(instance, &core->instances, list) { + if (!instance->max_input_data_size) + continue; + + total_bw_ddr += instance->power.bus_bw; + } + + ret = vote_buses(core, total_bw_ddr); + + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_vote_buses(struct iris_inst *inst) +{ + struct v4l2_format *out_f, *inp_f; + struct bus_vote_data *vote_data; + struct iris_core *core; + + core = inst->core; + + vote_data = &inst->bus_data; + + out_f = inst->fmt_dst; + inp_f = inst->fmt_src; + + vote_data->width = inp_f->fmt.pix_mp.width; + vote_data->height = inp_f->fmt.pix_mp.height; + vote_data->fps = inst->max_rate; + + if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) { + vote_data->color_formats[0] = V4L2_PIX_FMT_NV12; + vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat; + } else { + vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat; + } + + call_session_op(core, calc_bw, inst, vote_data); + + inst->power.bus_bw = vote_data->bus_bw; + + return iris_set_buses(inst); +} + +static int iris_set_clocks(struct iris_inst *inst) +{ + struct iris_inst *instance; + struct iris_core *core; + int ret = 0; + u64 freq; + + core = inst->core; + + mutex_lock(&core->lock); + + freq = 0; + list_for_each_entry(instance, &core->instances, list) { + if (!instance->max_input_data_size) + continue; + + freq += instance->power.min_freq; + } + + core->power.clk_freq = freq; + + ret = opp_set_rate(core, freq); + + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_scale_clocks(struct iris_inst *inst) +{ + struct iris_buffer *vbuf; + struct iris_core *core; + u32 data_size = 0; + + core = inst->core; + + list_for_each_entry(vbuf, &inst->buffers.input.list, list) + data_size = max(data_size, vbuf->data_size); + + inst->max_input_data_size = data_size; + + inst->max_rate = inst->cap[QUEUED_RATE].value >> 16; + + if (!inst->max_input_data_size) + return 0; + + inst->power.min_freq = call_session_op(core, calc_freq, inst, + inst->max_input_data_size); + iris_set_clocks(inst); + + return 0; +} + +int iris_scale_power(struct iris_inst *inst) +{ + iris_scale_clocks(inst); + iris_vote_buses(inst); + + return 0; +} + +int iris_update_input_rate(struct iris_inst *inst, u64 time_us) +{ + struct iris_input_timer *prev_timer = NULL; + struct iris_input_timer *input_timer; + u64 input_timer_sum_us = 0; + u64 counter = 0; + + input_timer = kzalloc(sizeof(*input_timer), GFP_KERNEL); + if (!input_timer) + return -ENOMEM; + + input_timer->time_us = time_us; + INIT_LIST_HEAD(&input_timer->list); + list_add_tail(&input_timer->list, &inst->input_timer_list); + list_for_each_entry(input_timer, &inst->input_timer_list, list) { + if (prev_timer) { + input_timer_sum_us += input_timer->time_us - prev_timer->time_us; + counter++; + } + prev_timer = input_timer; + } + + if (input_timer_sum_us && counter >= INPUT_TIMER_LIST_SIZE) + inst->cap[QUEUED_RATE].value = + (s32)(DIV64_U64_ROUND_CLOSEST(counter * 1000000, + input_timer_sum_us) << 16); + + if (counter >= INPUT_TIMER_LIST_SIZE) { + input_timer = list_first_entry(&inst->input_timer_list, + struct iris_input_timer, list); + list_del_init(&input_timer->list); + kfree(input_timer); + } + + return 0; +} + +int iris_flush_input_timer(struct iris_inst *inst) +{ + struct iris_input_timer *input_timer, *dummy_timer; + + list_for_each_entry_safe(input_timer, dummy_timer, &inst->input_timer_list, list) { + list_del_init(&input_timer->list); + kfree(input_timer); + } + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.h b/drivers/media/platform/qcom/vcodec/iris/iris_power.h new file mode 100644 index 0000000..28dbd5f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_POWER_H_ +#define _IRIS_POWER_H_ + +#include "iris_instance.h" + +int iris_scale_power(struct iris_inst *inst); +int iris_update_input_rate(struct iris_inst *inst, u64 time_us); +int iris_flush_input_timer(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index bf484a3..2571e27 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -6,14 +6,15 @@ #include #include #include +#include #include "../hfi_queue.h" #include "iris_core.h" +#include "iris_ctrls.h" #include "iris_helpers.h" #include "iris_hfi.h" -#include "resources.h" #include "iris_vidc.h" -#include "iris_ctrls.h" +#include "resources.h" static int init_iris_isr(struct iris_core *core) { @@ -70,6 +71,8 @@ static void iris_remove(struct platform_device *pdev) if (!core) return; + iris_pm_get(core); + iris_core_deinit(core); hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, @@ -79,7 +82,10 @@ static void iris_remove(struct platform_device *pdev) v4l2_device_unregister(&core->v4l2_dev); + iris_pm_put(core, false); + mutex_destroy(&core->lock); + mutex_destroy(&core->pm_lock); core->state = IRIS_CORE_DEINIT; } @@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev) core->state = IRIS_CORE_DEINIT; mutex_init(&core->lock); + mutex_init(&core->pm_lock); core->packet_size = IFACEQ_CORE_PKT_SIZE; core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL); @@ -117,58 +124,66 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + pm_runtime_set_autosuspend_delay(core->dev, AUTOSUSPEND_DELAY_VALUE); + pm_runtime_use_autosuspend(core->dev); + ret = devm_pm_runtime_enable(core->dev); + if (ret) { + dev_err(core->dev, "failed to enable runtime pm\n"); + goto err_runtime_disable; + } + ret = init_iris_isr(core); if (ret) { dev_err_probe(core->dev, ret, "%s: Failed to init isr with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_platform(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init platform failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_vpu(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init vpu failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_ops(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init ops failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_resources(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init resource failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = iris_init_core_caps(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init core caps failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = iris_init_instance_caps(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init inst caps failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) - return ret; + goto err_runtime_disable; ret = iris_register_video_device(core); if (ret) @@ -194,14 +209,29 @@ static int iris_probe(struct platform_device *pdev) goto err_vdev_unreg; } + ret = iris_pm_get(core); + if (ret) { + dev_err_probe(core->dev, ret, "%s: failed to get runtime pm\n", __func__); + goto err_queue_deinit; + } + ret = iris_core_init(core); if (ret) { dev_err_probe(core->dev, ret, "%s: core init failed\n", __func__); goto err_queue_deinit; } + ret = iris_pm_put(core, false); + if (ret) { + pm_runtime_get_noresume(core->dev); + dev_err_probe(core->dev, ret, "%s: failed to put runtime pm\n", __func__); + goto err_core_deinit; + } + return ret; +err_core_deinit: + iris_core_deinit(core); err_queue_deinit: hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, @@ -210,10 +240,77 @@ static int iris_probe(struct platform_device *pdev) video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); +err_runtime_disable: + pm_runtime_put_noidle(core->dev); + pm_runtime_set_suspended(core->dev); return ret; } +static int iris_pm_suspend(struct device *dev) +{ + struct iris_core *core; + int ret; + + if (!dev || !dev->driver) + return 0; + + core = dev_get_drvdata(dev); + + mutex_lock(&core->lock); + if (!core_in_valid_state(core)) { + ret = 0; + goto unlock; + } + + if (!core->power_enabled) { + ret = 0; + goto unlock; + } + + ret = iris_hfi_pm_suspend(core); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_pm_resume(struct device *dev) +{ + struct iris_core *core; + int ret; + + if (!dev || !dev->driver) + return 0; + + core = dev_get_drvdata(dev); + + mutex_lock(&core->lock); + if (!core_in_valid_state(core)) { + ret = 0; + goto unlock; + } + + if (core->power_enabled) { + ret = 0; + goto unlock; + } + + ret = iris_hfi_pm_resume(core); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +static const struct dev_pm_ops iris_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, + pm_runtime_force_resume) + SET_RUNTIME_PM_OPS(iris_pm_suspend, iris_pm_resume, NULL) +}; + static const struct of_device_id iris_dt_match[] = { { .compatible = "qcom,sm8550-iris", @@ -229,6 +326,7 @@ static struct platform_driver qcom_iris_driver = { .driver = { .name = "qcom-iris", .of_match_table = iris_dt_match, + .pm = &iris_pm_ops, }, }; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index f34434fc..c0878f1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -10,6 +10,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vb2.h" #include "iris_vdec.h" @@ -111,39 +112,52 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) goto error; } + ret = iris_pm_get(inst->core); + if (ret) + goto error; + if (!inst->once_per_session_set) { inst->once_per_session_set = true; ret = iris_hfi_session_set_codec(inst); if (ret) - goto error; + goto err_pm_get; ret = iris_hfi_session_set_default_header(inst); if (ret) - goto error; + goto err_pm_get; ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST); if (ret) - goto error; + goto err_pm_get; } + iris_scale_power(inst); + if (q->type == INPUT_MPLANE) ret = vdec_streamon_input(inst); else if (q->type == OUTPUT_MPLANE) ret = vdec_streamon_output(inst); if (ret) - goto error; + goto err_pm_get; buf_type = v4l2_type_to_driver(q->type); if (!buf_type) { ret = -EINVAL; - goto error; + goto err_pm_get; } ret = queue_deferred_buffers(inst, buf_type); if (ret) + goto err_pm_get; + + ret = iris_pm_put(inst->core, true); + if (ret) goto error; return ret; + +err_pm_get: + iris_pm_put(inst->core, false); error: iris_inst_change_state(inst, IRIS_INST_ERROR); @@ -165,6 +179,10 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE) goto error; + ret = iris_pm_get_put(inst->core); + if (ret) + goto error; + if (q->type == INPUT_MPLANE) ret = session_streamoff(inst, INPUT_MPLANE); else if (q->type == OUTPUT_MPLANE) @@ -181,6 +199,8 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) void iris_vb2_buf_queue(struct vb2_buffer *vb2) { + u64 ktime_ns = ktime_get_ns(); + struct iris_core *core; struct iris_inst *inst; int ret; @@ -188,11 +208,23 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2) if (!inst || !inst->core) return; + core = inst->core; + if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) { ret = -EINVAL; goto exit; } + if (vb2->type == INPUT_MPLANE) { + ret = iris_update_input_rate(inst, div_u64(ktime_ns, 1000)); + if (ret) + goto exit; + } + + ret = iris_pm_get_put(core); + if (ret) + goto exit; + ret = vdec_qbuf(inst, vb2); exit: diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 7884ba6..371615e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -12,6 +12,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" +#include "iris_power.h" #include "iris_vdec.h" #define UNSPECIFIED_COLOR_FORMAT 5 @@ -1069,6 +1070,8 @@ static int process_streamon_input(struct iris_inst *inst) enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; int ret; + iris_scale_power(inst); + ret = iris_hfi_start(inst, INPUT_MPLANE); if (ret) return ret; @@ -1150,6 +1153,8 @@ static int process_streamon_output(struct iris_inst *inst) bool drain_pending = false; int ret; + iris_scale_power(inst); + if (inst->sub_state & IRIS_INST_SUB_DRC && inst->sub_state & IRIS_INST_SUB_DRC_LAST) clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; @@ -1287,6 +1292,8 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) return 0; } + iris_scale_power(inst); + ret = queue_buffer(inst, buf); if (ret) return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index d37ef04..437d6b4 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -9,12 +9,13 @@ #include "iris_buffer.h" #include "iris_common.h" +#include "iris_ctrls.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vdec.h" #include "iris_vidc.h" -#include "iris_ctrls.h" #include "iris_vb2.h" #define VIDC_DRV_NAME "iris_driver" @@ -162,17 +163,23 @@ int vidc_open(struct file *filp) int i = 0; int ret; - ret = iris_core_init(core); + ret = iris_pm_get(core); if (ret) return ret; + ret = iris_core_init(core); + if (ret) + goto fail_pm_put; + ret = iris_core_init_wait(core); if (ret) - return ret; + goto fail_pm_put; inst = kzalloc(sizeof(*inst), GFP_KERNEL); - if (!inst) - return -ENOMEM; + if (!inst) { + ret = -ENOMEM; + goto fail_pm_put; + } inst->core = core; inst->session_id = hash32_ptr(inst); @@ -198,6 +205,7 @@ int vidc_open(struct file *filp) INIT_LIST_HEAD(&inst->buffers.persist.list); INIT_LIST_HEAD(&inst->buffers.vpss.list); INIT_LIST_HEAD(&inst->caps_list); + INIT_LIST_HEAD(&inst->input_timer_list); for (i = 0; i < MAX_SIGNAL; i++) init_completion(&inst->completions[i]); @@ -211,11 +219,16 @@ int vidc_open(struct file *filp) if (ret) goto fail_inst_deinit; + iris_scale_power(inst); + ret = iris_hfi_session_open(inst); if (ret) { dev_err(core->dev, "%s: session open failed\n", __func__); goto fail_core_deinit; } + + iris_pm_put(core, true); + filp->private_data = &inst->fh; return 0; @@ -233,6 +246,8 @@ int vidc_open(struct file *filp) mutex_destroy(&inst->ctx_q_lock); mutex_destroy(&inst->lock); kfree(inst); +fail_pm_put: + iris_pm_put(core, false); return ret; } @@ -240,20 +255,25 @@ int vidc_open(struct file *filp) int vidc_close(struct file *filp) { struct iris_inst *inst; + struct iris_core *core; inst = get_vidc_inst(filp, NULL); if (!inst) return -EINVAL; + core = inst->core; + v4l2_ctrl_handler_free(&inst->ctrl_handler); vdec_inst_deinit(inst); mutex_lock(&inst->lock); + iris_pm_get(core); close_session(inst); iris_inst_change_state(inst, IRIS_INST_CLOSE); vidc_vb2_queue_deinit(inst); vidc_v4l2_fh_deinit(inst); iris_destroy_buffers(inst); vidc_remove_session(inst); + iris_pm_put(core, false); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock); mutex_destroy(&inst->lock); @@ -965,11 +985,17 @@ static int vidc_dec_cmd(struct file *filp, void *fh, goto unlock; } + ret = iris_pm_get(inst->core); + if (ret) + goto unlock; + if (dec->cmd == V4L2_DEC_CMD_START) ret = vdec_start_cmd(inst); else if (dec->cmd == V4L2_DEC_CMD_STOP) ret = vdec_stop_cmd(inst); + iris_pm_put(inst->core, true); + unlock: mutex_unlock(&inst->lock); diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 22a8f5b..81de610 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -15,6 +15,7 @@ struct iris_core; struct iris_inst; #define HW_RESPONSE_TIMEOUT_VALUE (1000) +#define AUTOSUSPEND_DELAY_VALUE (HW_RESPONSE_TIMEOUT_VALUE + 500) #define BIT_DEPTH_8 (8 << 16 | 8) #define BIT_DEPTH_10 (10 << 16 | 10) @@ -64,6 +65,11 @@ struct reg_preset_info { u32 mask; }; +struct iris_core_power { + u64 clk_freq; + u64 bus_bw; +}; + struct ubwc_config_data { u32 max_channels; u32 mal_length; @@ -74,6 +80,18 @@ struct ubwc_config_data { u32 bank_spreading; }; +struct bus_vote_data { + u32 color_formats[2]; + int height, width; + u32 fps; + u64 bus_bw; +}; + +struct iris_inst_power { + u64 min_freq; + u32 bus_bw; +}; + enum plat_core_cap_type { CORE_CAP_NONE = 0, DEC_CODECS, diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c index 8cfdcd6..c9e3754 100644 --- a/drivers/media/platform/qcom/vcodec/iris/resources.c +++ b/drivers/media/platform/qcom/vcodec/iris/resources.c @@ -4,6 +4,8 @@ */ #include +#include +#include #include #include #include @@ -15,6 +17,8 @@ #include "platform_common.h" #include "resources.h" +#define BW_THRESHOLD 50000 + static void iris_pd_release(void *res) { struct device *pd = (struct device *)res; @@ -59,6 +63,34 @@ static int iris_opp_dl_get(struct device *dev, struct device *supplier) return ret; } +int opp_set_rate(struct iris_core *core, u64 freq) +{ + unsigned long opp_freq = 0; + struct dev_pm_opp *opp; + int ret; + + opp_freq = freq; + + opp = dev_pm_opp_find_freq_ceil(core->dev, &opp_freq); + if (IS_ERR(opp)) { + opp = dev_pm_opp_find_freq_floor(core->dev, &opp_freq); + if (IS_ERR(opp)) { + dev_err(core->dev, + "unable to find freq %lld in opp table\n", freq); + return -EINVAL; + } + } + dev_pm_opp_put(opp); + + ret = dev_pm_opp_set_rate(core->dev, opp_freq); + if (ret) { + dev_err(core->dev, "failed to set rate\n"); + return ret; + } + + return ret; +} + static int init_bus(struct iris_core *core) { const struct bus_info *bus_tbl; @@ -135,6 +167,10 @@ static int init_power_domains(struct iris_core *core) } } + ret = devm_pm_opp_set_clkname(core->dev, "vcodec_core"); + if (ret) + return ret; + ret = devm_pm_opp_of_add_table(core->dev); if (ret) { dev_err(core->dev, "%s: failed to add opp table\n", __func__); @@ -144,6 +180,56 @@ static int init_power_domains(struct iris_core *core) return ret; } +int enable_power_domains(struct iris_core *core, const char *name) +{ + struct power_domain_info *pdinfo = NULL; + int ret; + u32 i; + + ret = opp_set_rate(core, ULONG_MAX); + if (ret) + return ret; + + core->pd_count = core->platform_data->pd_tbl_size; + for (i = 0; i < (core->pd_count - 1); i++) { + pdinfo = &core->power_domain_tbl[i]; + if (strcmp(pdinfo->name, name)) + continue; + ret = pm_runtime_get_sync(pdinfo->genpd_dev); + if (ret < 0) + return ret; + } + + ret = opp_set_rate(core, ULONG_MAX); + if (ret) + return ret; + + return ret; +} + +int disable_power_domains(struct iris_core *core, const char *name) +{ + struct power_domain_info *pdinfo = NULL; + int ret; + u32 i; + + ret = opp_set_rate(core, 0); + if (ret) + return ret; + + core->pd_count = core->platform_data->pd_tbl_size; + for (i = 0; i < (core->pd_count - 1); i++) { + pdinfo = &core->power_domain_tbl[i]; + if (strcmp(pdinfo->name, name)) + continue; + ret = pm_runtime_put_sync(pdinfo->genpd_dev); + if (ret) + return ret; + } + + return ret; +} + static int init_clocks(struct iris_core *core) { const struct clock_info *clk_tbl; @@ -204,6 +290,200 @@ static int init_reset_clocks(struct iris_core *core) return 0; } +int unvote_buses(struct iris_core *core) +{ + struct bus_info *bus = NULL; + int ret = 0; + u32 i; + + core->power.bus_bw = 0; + core->bus_count = core->platform_data->bus_tbl_size; + + for (i = 0; i < core->bus_count; i++) { + bus = &core->bus_tbl[i]; + if (!bus->icc) + return -EINVAL; + + ret = icc_set_bw(bus->icc, 0, 0); + if (ret) + return ret; + } + + return ret; +} + +int vote_buses(struct iris_core *core, unsigned long bus_bw) +{ + unsigned long bw_kbps = 0, bw_prev = 0; + struct bus_info *bus = NULL; + int ret = 0; + u32 i; + + core->bus_count = core->platform_data->bus_tbl_size; + + for (i = 0; i < core->bus_count; i++) { + bus = &core->bus_tbl[i]; + if (bus && bus->icc) { + if (!strcmp(bus->name, "iris-ddr")) { + bw_kbps = bus_bw; + bw_prev = core->power.bus_bw; + } else { + bw_kbps = bus->bw_max_kbps; + bw_prev = core->power.bus_bw ? + bw_kbps : 0; + } + + bw_kbps = clamp_t(typeof(bw_kbps), bw_kbps, + bus->bw_min_kbps, bus->bw_max_kbps); + + if (abs(bw_kbps - bw_prev) < BW_THRESHOLD && bw_prev) + continue; + + ret = icc_set_bw(bus->icc, bw_kbps, 0); + if (ret) + return ret; + + if (!strcmp(bus->name, "iris-ddr")) + core->power.bus_bw = bw_kbps; + } + } + + return ret; +} + +static int deassert_reset_control(struct iris_core *core) +{ + struct reset_info *rcinfo = NULL; + int ret = 0; + u32 i; + + core->reset_count = core->platform_data->clk_rst_tbl_size; + + for (i = 0; i < (core->reset_count - 1); i++) { + rcinfo = &core->reset_tbl[i]; + ret = reset_control_deassert(rcinfo->rst); + if (ret) { + dev_err(core->dev, "deassert reset control failed. ret = %d\n", ret); + continue; + } + } + + return ret; +} + +static int assert_reset_control(struct iris_core *core) +{ + struct reset_info *rcinfo = NULL; + int ret = 0, cnt = 0; + u32 i; + + core->reset_count = core->platform_data->clk_rst_tbl_size; + + for (i = 0; i < (core->reset_count - 1); i++) { + rcinfo = &core->reset_tbl[i]; + if (!rcinfo->rst) + return -EINVAL; + + ret = reset_control_assert(rcinfo->rst); + if (ret) { + dev_err(core->dev, "failed to assert reset control %s, ret = %d\n", + rcinfo->name, ret); + goto deassert_reset_control; + } + cnt++; + + usleep_range(1000, 1100); + } + + return ret; +deassert_reset_control: + for (i = 0; i < cnt; i++) { + rcinfo = &core->reset_tbl[i]; + reset_control_deassert(rcinfo->rst); + } + + return ret; +} + +int reset_ahb2axi_bridge(struct iris_core *core) +{ + int ret; + + ret = assert_reset_control(core); + if (ret) + return ret; + + ret = deassert_reset_control(core); + + return ret; +} + +int disable_unprepare_clock(struct iris_core *core, const char *clk_name) +{ + struct clock_info *cl; + bool found = false; + u32 i; + + core->clk_count = core->platform_data->clk_tbl_size; + + for (i = 0; i < core->clk_count; i++) { + cl = &core->clock_tbl[i]; + if (!cl->clk) + return -EINVAL; + + if (strcmp(cl->name, clk_name)) + continue; + + found = true; + clk_disable_unprepare(cl->clk); + cl->prev = 0; + break; + } + + if (!found) + return -EINVAL; + + return 0; +} + +int prepare_enable_clock(struct iris_core *core, const char *clk_name) +{ + struct clock_info *cl; + bool found = false; + int ret = 0; + u32 i; + + core->clk_count = core->platform_data->clk_tbl_size; + + for (i = 0; i < core->clk_count; i++) { + cl = &core->clock_tbl[i]; + if (!cl->clk) + return -EINVAL; + + if (strcmp(cl->name, clk_name)) + continue; + + found = true; + + ret = clk_prepare_enable(cl->clk); + if (ret) { + dev_err(core->dev, "failed to enable clock %s\n", cl->name); + return ret; + } + + if (!__clk_is_enabled(cl->clk)) { + clk_disable_unprepare(cl->clk); + return -EINVAL; + } + break; + } + + if (!found) + return -EINVAL; + + return ret; +} + int init_resources(struct iris_core *core) { int ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h index d21bcc7e..09857ed 100644 --- a/drivers/media/platform/qcom/vcodec/iris/resources.h +++ b/drivers/media/platform/qcom/vcodec/iris/resources.h @@ -31,6 +31,14 @@ struct reset_info { const char *name; }; +int enable_power_domains(struct iris_core *core, const char *name); +int disable_power_domains(struct iris_core *core, const char *name); +int unvote_buses(struct iris_core *core); +int vote_buses(struct iris_core *core, unsigned long bus_bw); +int reset_ahb2axi_bridge(struct iris_core *core); +int opp_set_rate(struct iris_core *core, u64 freq); +int disable_unprepare_clock(struct iris_core *core, const char *clk_name); +int prepare_enable_clock(struct iris_core *core, const char *clk_name); int init_resources(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c index 3282510..8ef1029 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "vpu_iris3.h" #include "iris_core.h" #include "iris_helpers.h" @@ -17,6 +19,9 @@ int write_register(struct iris_core *core, u32 reg, u32 value) if (ret) return ret; + if (!core->power_enabled) + return -EINVAL; + base_addr = core->reg_base; base_addr += reg; writel_relaxed(value, base_addr); @@ -27,10 +32,46 @@ int write_register(struct iris_core *core, u32 reg, u32 value) return ret; } +int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask) +{ + void __iomem *base_addr; + u32 prev_val, new_val; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core->power_enabled) + return -EINVAL; + + base_addr = core->reg_base; + base_addr += reg; + + prev_val = readl_relaxed(base_addr); + /* + * Memory barrier to ensure register read is correct + */ + rmb(); + + new_val = (prev_val & ~mask) | (value & mask); + + writel_relaxed(new_val, base_addr); + /* + * Memory barrier to make sure value is written into the register. + */ + wmb(); + + return ret; +} + int read_register(struct iris_core *core, u32 reg, u32 *value) { void __iomem *base_addr; + if (!core->power_enabled) + return -EINVAL; + base_addr = core->reg_base; *value = readl_relaxed(base_addr + reg); @@ -41,6 +82,52 @@ int read_register(struct iris_core *core, u32 reg, u32 *value) return 0; } +int read_register_with_poll_timeout(struct iris_core *core, u32 reg, + u32 mask, u32 exp_val, u32 sleep_us, + u32 timeout_us) +{ + void __iomem *base_addr; + u32 val = 0; + int ret; + + if (!core->power_enabled) + return -EINVAL; + + base_addr = core->reg_base; + + ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val), + sleep_us, timeout_us); + /* + * Memory barrier to make sure value is read correctly from the + * register. + */ + rmb(); + + return ret; +} + +int set_preset_registers(struct iris_core *core) +{ + const struct reg_preset_info *reg_prst; + unsigned int prst_count; + int cnt, ret = 0; + + reg_prst = core->platform_data->reg_prst_tbl; + prst_count = core->platform_data->reg_prst_tbl_size; + + if (!reg_prst || !prst_count) + return 0; + + for (cnt = 0; cnt < prst_count; cnt++) { + ret = write_register_masked(core, reg_prst[cnt].reg, + reg_prst[cnt].value, reg_prst[cnt].mask); + if (ret) + return ret; + } + + return ret; +} + static const struct compat_handle compat_handle[] = { { .compat = "qcom,sm8550-iris", diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h index 7fba8c2..aef5606 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h @@ -24,6 +24,9 @@ struct vpu_ops { int (*raise_interrupt)(struct iris_core *core); int (*clear_interrupt)(struct iris_core *core); int (*watchdog)(struct iris_core *core, u32 intr_status); + int (*power_on)(struct iris_core *core); + int (*power_off)(struct iris_core *core); + int (*prepare_pc)(struct iris_core *core); }; #define call_session_op(c, op, ...) \ @@ -32,11 +35,18 @@ struct vpu_ops { struct vpu_session_ops { int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type); + u64 (*calc_freq)(struct iris_inst *inst, u32 data_size); + int (*calc_bw)(struct iris_inst *inst, struct bus_vote_data *data); }; int init_vpu(struct iris_core *core); int write_register(struct iris_core *core, u32 reg, u32 value); +int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask); int read_register(struct iris_core *core, u32 reg, u32 *value); +int read_register_with_poll_timeout(struct iris_core *core, u32 reg, + u32 mask, u32 exp_val, u32 sleep_us, + u32 timeout_us); +int set_preset_registers(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c index efea5aa..85ed38d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c @@ -5,73 +5,105 @@ #include +#include "iris_hfi.h" #include "vpu_iris3.h" #include "vpu_iris3_buffer.h" +#include "vpu_iris3_power.h" #define VIDEO_ARCH_LX 1 -#define CPU_BASE_OFFS_IRIS3 0x000A0000 - -#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) -#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) - -#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24) -#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28) - -#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C) +#define VCODEC_BASE_OFFS_IRIS3 0x00000000 +#define AON_MVP_NOC_RESET 0x0001F000 +#define CPU_BASE_OFFS_IRIS3 0x000A0000 +#define AON_BASE_OFFS 0x000E0000 +#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) +#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) +#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C) +#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24) +#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28) /* HFI_CTRL_INIT */ -#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48) - +#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48) /* HFI_CTRL_STATUS */ -#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C) -#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000 - -#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148) - -#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168) - -/* UC_REGION_ADDR */ -#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64) - -/* UC_REGION_ADDR */ -#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68) - +#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C) /* HFI_QTBL_INFO */ -#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50) - +#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50) /* HFI_QTBL_ADDR */ -#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54) - +#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54) /* SFR_ADDR */ -#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C) - -#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3 -#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3 - -#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3 -#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3 - -#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3 +#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C) +#define CPU_CS_SCIBCMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x60) +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64) +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68) +#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148) +#define CPU_CS_AHB_BRIDGE_SYNC_RESET (CPU_CS_BASE_OFFS_IRIS3 + 0x160) +#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168) -#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3 +#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150) +#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0 -#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3 -#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe -#define CTRL_ERROR_STATUS__M_IRIS3 \ - CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 -#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 \ - CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 +#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000 +#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe +#define CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3 0x100 + +#define AON_WRAPPER_MVP_NOC_RESET_REQ (AON_MVP_NOC_RESET + 0x000) +#define AON_WRAPPER_MVP_NOC_RESET_ACK (AON_MVP_NOC_RESET + 0x004) + +#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000 +#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C) +#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 +#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4 + +#define WRAPPER_INTR_MASK_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x10) +#define WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 0x8 +#define WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3 0x4 + +#define WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x54) +#define WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x58) +#define WRAPPER_IRIS_CPU_NOC_LPI_CONTROL (WRAPPER_BASE_OFFS_IRIS3 + 0x5C) +#define WRAPPER_IRIS_CPU_NOC_LPI_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x60) +#define WRAPPER_CORE_POWER_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x80) +#define WRAPPER_CORE_CLOCK_CONFIG_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x88) + +#define WRAPPER_TZ_BASE_OFFS 0x000C0000 +#define WRAPPER_TZ_CPU_STATUS (WRAPPER_TZ_BASE_OFFS + 0x10) +#define WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG (WRAPPER_TZ_BASE_OFFS + 0x14) +#define WRAPPER_TZ_QNS4PDXFIFO_RESET (WRAPPER_TZ_BASE_OFFS + 0x18) + +#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3 +#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3 +#define CTRL_ERROR_STATUS__M_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 +#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 +#define CTRL_STATUS_PC_READY_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3 + +#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3 +#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3 +#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3 +#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3 +#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3 + +#define AON_WRAPPER_MVP_NOC_LPI_CONTROL (AON_BASE_OFFS) +#define AON_WRAPPER_MVP_NOC_LPI_STATUS (AON_BASE_OFFS + 0x4) + +#define VCODEC_SS_IDLE_STATUSN (VCODEC_BASE_OFFS_IRIS3 + 0x70) + +static int interrupt_init_iris3(struct iris_core *core) +{ + u32 mask_val; + int ret; -#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000 -#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C) -#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 -#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4 + ret = read_register(core, WRAPPER_INTR_MASK_IRIS3, &mask_val); + if (ret) + return ret; -#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150) -#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0 + mask_val &= ~(WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 | + WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3); + ret = write_register(core, WRAPPER_INTR_MASK_IRIS3, mask_val); -#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 + return ret; +} static int setup_ucregion_memory_map_iris3(struct iris_core *core) { @@ -196,15 +228,335 @@ static int watchdog_iris3(struct iris_core *core, u32 intr_status) return 0; } +static bool is_iris3_hw_power_collapsed(struct iris_core *core) +{ + u32 value = 0, pwr_status = 0; + int ret; + + ret = read_register(core, WRAPPER_CORE_POWER_STATUS, &value); + if (ret) + return false; + + pwr_status = value & BIT(1); + + return pwr_status ? false : true; +} + +static int power_off_iris3_hardware(struct iris_core *core) +{ + u32 value = 0; + int ret, i; + + if (is_iris3_hw_power_collapsed(core)) + goto disable_power; + + dev_err(core->dev, "Video hw is power ON\n"); + + ret = read_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, &value); + if (ret) + goto disable_power; + + if (value) { + ret = write_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, 0); + if (ret) + goto disable_power; + } + + for (i = 0; i < core->cap[NUM_VPP_PIPE].value; i++) { + ret = read_register_with_poll_timeout(core, VCODEC_SS_IDLE_STATUSN + 4 * i, + 0x400000, 0x400000, 2000, 20000); + } + + ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x3); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK, + 0x3, 0x3, 200, 2000); + ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x0); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK, + 0x3, 0x0, 200, 2000); + + ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x3); + if (ret) + goto disable_power; + ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x2); + if (ret) + goto disable_power; + ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x0); + if (ret) + goto disable_power; + +disable_power: + ret = disable_power_domains(core, "vcodec"); + if (ret) { + dev_err(core->dev, "disable power domain vcodec failed\n"); + ret = 0; + } + + disable_unprepare_clock(core, "vcodec_core"); + if (ret) { + dev_err(core->dev, "disable unprepare vcodec_core failed\n"); + ret = 0; + } + + return ret; +} + +static int power_off_iris3_controller(struct iris_core *core) +{ + int ret; + + ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x3); + if (ret) + goto disable_power; + + ret = write_register_masked(core, AON_WRAPPER_MVP_NOC_LPI_CONTROL, + 0x1, BIT(0)); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_LPI_STATUS, + 0x1, 0x1, 200, 2000); + + ret = write_register_masked(core, WRAPPER_IRIS_CPU_NOC_LPI_CONTROL, + 0x1, BIT(0)); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, WRAPPER_IRIS_CPU_NOC_LPI_STATUS, + 0x1, 0x1, 200, 2000); + + ret = write_register(core, WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3, 0x0); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3, + 0xffffffff, 0x0, 200, 2000); + + ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x3); + if (ret) + goto disable_power; + + ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x1); + if (ret) + goto disable_power; + + ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x0); + if (ret) + goto disable_power; + + ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x0); + if (ret) + goto disable_power; + +disable_power: + disable_unprepare_clock(core, "core_clk"); + if (ret) { + dev_err(core->dev, "disable unprepare core_clk failed\n"); + ret = 0; + } + + /* power down process */ + ret = disable_power_domains(core, "iris-ctl"); + if (ret) { + dev_err(core->dev, "disable power domain iris-ctl failed\n"); + ret = 0; + } + + return ret; +} + +static int power_off_iris3(struct iris_core *core) +{ + if (!core->power_enabled) + return 0; + + opp_set_rate(core, 0); + power_off_iris3_hardware(core); + power_off_iris3_controller(core); + unvote_buses(core); + + if (!call_vpu_op(core, watchdog, core, core->intr_status)) + disable_irq_nosync(core->irq); + + core->power_enabled = false; + + return 0; +} + +static int power_on_iris3_controller(struct iris_core *core) +{ + int ret; + + ret = enable_power_domains(core, "iris-ctl"); + if (ret) + return ret; + + ret = reset_ahb2axi_bridge(core); + if (ret) + goto err_disable_power; + + ret = prepare_enable_clock(core, "gcc_video_axi0"); + if (ret) + goto err_disable_power; + + ret = prepare_enable_clock(core, "core_clk"); + if (ret) + goto err_disable_clock; + + return ret; + +err_disable_clock: + disable_unprepare_clock(core, "gcc_video_axi0"); +err_disable_power: + disable_power_domains(core, "iris-ctl"); + + return ret; +} + +static int power_on_iris3_hardware(struct iris_core *core) +{ + int ret; + + ret = enable_power_domains(core, "vcodec"); + if (ret) + return ret; + + ret = prepare_enable_clock(core, "vcodec_core"); + if (ret) + goto err_disable_power; + + return ret; + +err_disable_power: + disable_power_domains(core, "vcodec"); + + return ret; +} + +static int power_on_iris3(struct iris_core *core) +{ + u32 freq = 0; + int ret; + + if (core->power_enabled) + return 0; + + if (!core_in_valid_state(core)) + return -EINVAL; + + ret = vote_buses(core, INT_MAX); + if (ret) + goto err; + + ret = power_on_iris3_controller(core); + if (ret) + goto err_unvote_bus; + + ret = power_on_iris3_hardware(core); + if (ret) + goto err_power_off_ctrl; + + core->power_enabled = true; + + freq = core->power.clk_freq ? core->power.clk_freq : + (u32)ULONG_MAX; + + opp_set_rate(core, freq); + + set_preset_registers(core); + + interrupt_init_iris3(core); + core->intr_status = 0; + enable_irq(core->irq); + + return ret; + +err_power_off_ctrl: + power_off_iris3_controller(core); +err_unvote_bus: + unvote_buses(core); +err: + core->power_enabled = false; + + return ret; +} + +static int prepare_pc_iris3(struct iris_core *core) +{ + u32 wfi_status = 0, idle_status = 0, pc_ready = 0; + u32 ctrl_status = 0; + int ret; + + ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status); + if (ret) + return ret; + + pc_ready = ctrl_status & CTRL_STATUS_PC_READY_IRIS3; + idle_status = ctrl_status & BIT(30); + + if (pc_ready) + return 0; + + ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status); + if (ret) + return ret; + + wfi_status &= BIT(0); + if (!wfi_status || !idle_status) + goto skip_power_off; + + ret = prepare_pc(core); + if (ret) + goto skip_power_off; + + ret = read_register_with_poll_timeout(core, CTRL_STATUS_IRIS3, + CTRL_STATUS_PC_READY_IRIS3, + CTRL_STATUS_PC_READY_IRIS3, 250, 2500); + if (ret) + goto skip_power_off; + + ret = read_register_with_poll_timeout(core, WRAPPER_TZ_CPU_STATUS, + BIT(0), 0x1, 250, 2500); + if (ret) + goto skip_power_off; + + return ret; + +skip_power_off: + ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status); + if (ret) + return ret; + + ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status); + if (ret) + return ret; + + wfi_status &= BIT(0); + dev_err(core->dev, "Skip PC, wfi=%#x, idle=%#x, pcr=%#x, ctrl=%#x)\n", + wfi_status, idle_status, pc_ready, ctrl_status); + + return -EAGAIN; +} + static const struct vpu_ops iris3_ops = { .boot_firmware = boot_firmware_iris3, .raise_interrupt = raise_interrupt_iris3, .clear_interrupt = clear_interrupt_iris3, .watchdog = watchdog_iris3, + .power_on = power_on_iris3, + .power_off = power_off_iris3, + .prepare_pc = prepare_pc_iris3, }; static const struct vpu_session_ops iris3_session_ops = { .int_buf_size = iris_int_buf_size_iris3, + .calc_freq = iris_calc_freq_iris3, + .calc_bw = iris_calc_bw_iris3, }; int init_iris3(struct iris_core *core) diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c new file mode 100644 index 0000000..5a02f24 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_instance.h" +#include "iris_helpers.h" +#include "platform_common.h" +#include "vpu_iris3_power.h" + +u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size) +{ + u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0; + u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0; + u32 base_cycles = 0, fps, mbpf; + u32 height = 0, width = 0; + struct v4l2_format *inp_f; + u32 mbs_per_second; + + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + + mbpf = NUM_MBS_PER_FRAME(height, width); + fps = inst->max_rate; + mbs_per_second = mbpf * fps; + + fw_cycles = fps * inst->cap[MB_CYCLES_FW].value; + fw_vpp_cycles = fps * inst->cap[MB_CYCLES_FW_VPP].value; + + vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value / + inst->cap[PIPE].value; + vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles); + + if (inst->cap[PIPE].value > 1) + vpp_cycles += div_u64(vpp_cycles * 59, 1000); + + base_cycles = inst->cap[MB_CYCLES_VSP].value; + bitrate = fps * data_size * 8; + vsp_cycles = bitrate; + + if (inst->codec == VP9) { + vsp_cycles = div_u64(vsp_cycles * 170, 100); + } else { + base_cycles = 0; + vsp_cycles = div_u64(vsp_cycles, 2); + } + vsp_cycles = div_u64(vsp_cycles * 21, 20); + + if (inst->cap[STAGE].value == STAGE_1) + vsp_cycles = vsp_cycles * 3; + + vsp_cycles += mbs_per_second * base_cycles; + + freq = max3(vpp_cycles, vsp_cycles, fw_cycles); + + return freq; +} + +int iris_calc_bw_iris3(struct iris_inst *inst, struct bus_vote_data *data) +{ + const struct bw_info *bw_tbl = NULL; + unsigned int num_rows = 0; + unsigned int i, mbs, mbps; + struct iris_core *core; + + if (!data) + return 0; + + core = inst->core; + + mbs = (ALIGN(data->height, 16) / 16) * (ALIGN(data->width, 16) / 16); + mbps = mbs * data->fps; + if (mbps == 0) + return 0; + + bw_tbl = core->platform_data->bw_tbl_dec; + num_rows = core->platform_data->bw_tbl_dec_size; + + if (!bw_tbl || num_rows == 0) + return 0; + + for (i = 0; i < num_rows; i++) { + if (i != 0 && mbps > bw_tbl[i].mbs_per_sec) + break; + + if (is_10bit_colorformat(data->color_formats[0])) + data->bus_bw = bw_tbl[i].bw_ddr_10bit; + else + data->bus_bw = bw_tbl[i].bw_ddr; + } + + dev_info(core->dev, "bus_bw %llu\n", data->bus_bw); + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h new file mode 100644 index 0000000..2a37a9e --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef __H_VPU_IRIS3_POWER_H__ +#define __H_VPU_IRIS3_POWER_H__ + +u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size); +int iris_calc_bw_iris3(struct iris_inst *inst, + struct bus_vote_data *vote_data); + +#endif From patchwork Mon Dec 18 11:32:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757239 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4373F374D7; Mon, 18 Dec 2023 11:40:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="XzZN8sHB" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAtAkP007257; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=iNKyHQ8h/NpFc83aGl2oTQT1CzgFzngR8xdKmuX4b8k=; b=Xz ZN8sHB0XNgIBi+6EaRknJe5/ifxN/X18P7MLOR9wqWuvk+VHC+rdpEIO54oDc6K5 s+vyKYjiJuzIHYPRurOG0b2QnA6tZzkMLCqj8D1UUnpTh8UOEgCmPjzK0QzT5JLl x/NqJ3GtzTjmDyfmk+7iGcA4vUFa4wFchllFswl6lguhKiRVwCQZCPYqUsnFaW7t 07BUa5QuNuMIcYvrw82zL0dOpmvV6eydmB599ed54WXSDqhHq9jsJQiq4oVPOssJ JxQTRGrViXMYBBL9KfzsDzTgC7jhGfGkzcITRImwNmGthXUXgCD9y6D3hxGWSQDX nFEi6Xx/unmTCH3eSKQg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4y3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0FQ004875; Mon, 18 Dec 2023 11:40:01 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ac-3; Mon, 18 Dec 2023 11:40:01 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBa9rY000378; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8rd030068; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 698372389; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 30/34] media: iris: register video encoder device to platform driver Date: Mon, 18 Dec 2023 17:02:25 +0530 Message-Id: <1702899149-21321-31-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: BM17vDS2em8D_fMU0u92xI97O_1Ahvlb X-Proofpoint-GUID: BM17vDS2em8D_fMU0u92xI97O_1Ahvlb X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Register/unregister video Encoder device. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/iris_common.h | 5 +++ .../media/platform/qcom/vcodec/iris/iris_core.h | 2 ++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 42 ++++++++++++++++------ 3 files changed, 38 insertions(+), 11 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 3ab4767..b1273d0 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -30,6 +30,11 @@ #define INPUT_TIMER_LIST_SIZE 30 +enum domain_type { + ENCODER = BIT(0), + DECODER = BIT(1), +}; + enum codec_type { H264 = BIT(0), HEVC = BIT(1), diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 50553d2..c56eb24 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -23,6 +23,7 @@ * @irq: iris irq * @v4l2_dev: a holder for v4l2 device structure * @vdev_dec: iris video device structure for decoder + * @vdev_enc: iris video device structure for encoder * @v4l2_file_ops: iris v4l2 file ops * @v4l2_ioctl_ops: iris v4l2 ioctl ops * @bus_tbl: table of iris buses @@ -66,6 +67,7 @@ struct iris_core { int irq; struct v4l2_device v4l2_dev; struct video_device *vdev_dec; + struct video_device *vdev_enc; const struct v4l2_file_operations *v4l2_file_ops; const struct v4l2_ioctl_ops *v4l2_ioctl_ops; struct bus_info *bus_tbl; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 2571e27..b487e83 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -31,16 +31,15 @@ static int init_iris_isr(struct iris_core *core) return ret; } -static int iris_register_video_device(struct iris_core *core) +static int iris_register_video_device(struct iris_core *core, enum domain_type type) { struct video_device *vdev; - int ret; + int ret = 0; vdev = video_device_alloc(); if (!vdev) return -ENOMEM; - strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); vdev->release = video_device_release; vdev->fops = core->v4l2_file_ops; vdev->ioctl_ops = core->v4l2_ioctl_ops; @@ -48,11 +47,24 @@ static int iris_register_video_device(struct iris_core *core) vdev->v4l2_dev = &core->v4l2_dev; vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; - ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); - if (ret) - goto err_vdev_release; + if (type == DECODER) { + strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_dec = vdev; + } else if (type == ENCODER) { + strscpy(vdev->name, "qcom-iris-encoder", sizeof(vdev->name)); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_enc = vdev; + } - core->vdev_dec = vdev; video_set_drvdata(vdev, core); return ret; @@ -80,6 +92,8 @@ static void iris_remove(struct platform_device *pdev) video_unregister_device(core->vdev_dec); + video_unregister_device(core->vdev_enc); + v4l2_device_unregister(&core->v4l2_dev); iris_pm_put(core, false); @@ -185,17 +199,21 @@ static int iris_probe(struct platform_device *pdev) if (ret) goto err_runtime_disable; - ret = iris_register_video_device(core); + ret = iris_register_video_device(core, DECODER); if (ret) goto err_v4l2_unreg; + ret = iris_register_video_device(core, ENCODER); + if (ret) + goto err_vdev_unreg_dec; + platform_set_drvdata(pdev, core); dma_mask = core->cap[DMA_MASK].value; ret = dma_set_mask_and_coherent(dev, dma_mask); if (ret) - goto err_vdev_unreg; + goto err_vdev_unreg_enc; dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32)); dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64)); @@ -206,7 +224,7 @@ static int iris_probe(struct platform_device *pdev) if (ret) { dev_err_probe(core->dev, ret, "%s: interface queues init failed\n", __func__); - goto err_vdev_unreg; + goto err_vdev_unreg_enc; } ret = iris_pm_get(core); @@ -236,7 +254,9 @@ static int iris_probe(struct platform_device *pdev) hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, &core->debug_queue); -err_vdev_unreg: +err_vdev_unreg_enc: + video_unregister_device(core->vdev_enc); +err_vdev_unreg_dec: video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); From patchwork Mon Dec 18 11:32:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 757235 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF2E9374D7; Mon, 18 Dec 2023 11:40:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="N+0Eut/L" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArvvt003529; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=vLyhCEyyM++iSdmMnhPhEeX/SePVJRxrjXz9jo/+Aj8=; b=N+ 0Eut/LJZRxg02e4C1Pq0XZJWmA2o4pKqGbF+DGTm1hKe9dDm1I+C875LyLUgNZB4 i8u44/b2WaAbAmNEdlDHI7GbPhRL6YNcQq+tJ8TJmhTYG4FHg5jkFKuNdvpbEBAe wElL0Yq6p3QhGVNRoE9liJ0OMbO9WzDOUGt8RcJ2cwQeAAZZSE02SRwvtrw8hGGt zYEg+sCMeyTO+zlCCW1IwGXfjexNkArDAseOameOEc8qvVGpoNbLHyv2io+0kWFU 0ZXC572Z7yYF+I5vSMhjT/4rJGEo546GUg+ctAwD/1quxwM2T7pjPI4zLwUBI7gL //9mXB1vkyhrXtG1+xXQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4xw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX6PL029956; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ad-1; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX8rc030072; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX853030069; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 6C0E7238B; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 31/34] media: iris: add platform specific instance capabilities for encoder Date: Mon, 18 Dec 2023 17:02:26 +0530 Message-Id: <1702899149-21321-32-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: jEWxbDd7IENax4tQu8kZHHd5C7UvPkHF X-Proofpoint-GUID: jEWxbDd7IENax4tQu8kZHHd5C7UvPkHF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Capabilities are set of video specifications and features supported by a specific platform(SOC). Each capability is defined with min, max, range, default value and corresponding HFI. Also, platform data defines different resource details for a specific platform(SOC). This change defines resource tables for sm8550 platform data and use for initializing these resources. Add Children, Set, Adjust functions for to each capability for Encoder. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 128 ++- .../media/platform/qcom/vcodec/iris/iris_common.h | 4 + .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 981 ++++++++++++++++++++- .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 35 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 16 +- drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 53 ++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 7 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 10 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 5 + .../platform/qcom/vcodec/iris/iris_hfi_response.c | 6 + .../platform/qcom/vcodec/iris/iris_instance.h | 6 + .../platform/qcom/vcodec/iris/platform_common.h | 74 +- .../platform/qcom/vcodec/iris/platform_sm8550.c | 768 +++++++++++++++- .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 3 + 14 files changed, 2024 insertions(+), 72 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index 872674e..bc32c99 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -45,6 +45,8 @@ enum hfi_property_mode_type { #define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000 #define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff +#define HFI_LEVEL_NONE 0xFFFFFFFF + #define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 #define HFI_TRUE 0x00000001 @@ -68,14 +70,14 @@ enum hfi_codec_type { #define HFI_PROP_CODEC 0x03000100 enum hfi_color_format { - HFI_COLOR_FMT_OPAQUE = 0, - HFI_COLOR_FMT_NV12 = 1, - HFI_COLOR_FMT_NV12_UBWC = 2, - HFI_COLOR_FMT_P010 = 3, - HFI_COLOR_FMT_TP10_UBWC = 4, - HFI_COLOR_FMT_RGBA8888 = 5, - HFI_COLOR_FMT_RGBA8888_UBWC = 6, - HFI_COLOR_FMT_NV21 = 7, + HFI_COLOR_FMT_OPAQUE = 0, + HFI_COLOR_FMT_NV12 = 1, + HFI_COLOR_FMT_NV12_UBWC = 2, + HFI_COLOR_FMT_P010 = 3, + HFI_COLOR_FMT_TP10_UBWC = 4, + HFI_COLOR_FMT_RGBA8888 = 5, + HFI_COLOR_FMT_RGBA8888_UBWC = 6, + HFI_COLOR_FMT_NV21 = 7, }; #define HFI_PROP_COLOR_FORMAT 0x03000101 @@ -104,14 +106,108 @@ enum hfi_color_format { #define HFI_PROP_CABAC_SESSION 0x03000121 +#define HFI_PROP_8X8_TRANSFORM 0x03000122 + #define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123 #define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124 #define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128 +enum hfi_syncframe_request_mode { + HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR = 0x00000001, + HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR = 0x00000002, +}; + +enum hfi_rate_control { + HFI_RC_VBR_CFR = 0x00000000, + HFI_RC_CBR_CFR = 0x00000001, + HFI_RC_CQ = 0x00000002, + HFI_RC_OFF = 0x00000003, + HFI_RC_CBR_VFR = 0x00000004, + HFI_RC_LOSSLESS = 0x00000005, +}; + +#define HFI_PROP_RATE_CONTROL 0x0300012a + +#define HFI_PROP_QP_PACKED 0x0300012e + +#define HFI_PROP_MIN_QP_PACKED 0x0300012f + +#define HFI_PROP_MAX_QP_PACKED 0x03000130 + +enum hfi_layer_enc_type { + HFI_HIER_P_SLIDING_WINDOW = 0x1, + HFI_HIER_P_HYBRID_LTR = 0x2, + HFI_HIER_B = 0x3, +}; + +#define HFI_PROP_IR_RANDOM_PERIOD 0x03000131 + +#define HFI_PROP_MULTI_SLICE_MB_COUNT 0x03000132 + +#define HFI_PROP_MULTI_SLICE_BYTES_COUNT 0x03000133 + +#define HFI_PROP_LTR_COUNT 0x03000134 + +#define HFI_PROP_LTR_MARK 0x03000135 + +#define HFI_PROP_LTR_USE 0x03000136 + +#define HFI_PROP_LAYER_ENCODING_TYPE 0x03000138 + +#define HFI_PROP_LAYER_COUNT 0x03000139 + +#define HFI_PROP_TOTAL_BITRATE 0x0300013b + +#define HFI_PROP_BITRATE_LAYER1 0x0300013c + +#define HFI_PROP_BITRATE_LAYER2 0x0300013d + +#define HFI_PROP_BITRATE_LAYER3 0x0300013e + +#define HFI_PROP_BITRATE_LAYER4 0x0300013f + +#define HFI_PROP_BITRATE_LAYER5 0x03000140 + +#define HFI_PROP_BITRATE_LAYER6 0x03000141 + +#define HFI_PROP_BASELAYER_PRIORITYID 0x03000142 + +#define HFI_PROP_REQUEST_SYNC_FRAME 0x03000145 + +#define HFI_PROP_MAX_GOP_FRAMES 0x03000146 + +#define HFI_PROP_MAX_B_FRAMES 0x03000147 + #define HFI_PROP_QUALITY_MODE 0x03000148 +enum hfi_seq_header_mode { + HFI_SEQ_HEADER_SEPERATE_FRAME = 0x00000001, + HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME = 0x00000002, + HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME = 0x00000004, + HFI_SEQ_HEADER_METADATA = 0x00000008, +}; + +#define HFI_PROP_SEQ_HEADER_MODE 0x03000149 + +enum hfi_rotation { + HFI_ROTATION_NONE = 0x00000000, + HFI_ROTATION_90 = 0x00000001, + HFI_ROTATION_180 = 0x00000002, + HFI_ROTATION_270 = 0x00000003, +}; + +#define HFI_PROP_ROTATION 0x0300014b + +enum hfi_flip { + HFI_DISABLE_FLIP = 0x00000000, + HFI_HORIZONTAL_FLIP = 0x00000001, + HFI_VERTICAL_FLIP = 0x00000002, +}; + +#define HFI_PROP_FLIP 0x0300014c + enum hfi_color_primaries { HFI_PRIMARIES_RESERVED = 0, HFI_PRIMARIES_BT709 = 1, @@ -182,7 +278,7 @@ enum hfi_picture_type { #define HFI_PROP_PICTURE_TYPE 0x03000162 -#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 +#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 @@ -190,6 +286,12 @@ enum hfi_picture_type { #define HFI_PROP_DPB_LIST 0x0300017A +#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C + +#define HFI_PROP_MAINTAIN_MIN_QUALITY 0x0300017D + +#define HFI_PROP_IR_CYCLIC_PERIOD 0x0300017E + #define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190 #define HFI_PROP_COMV_BUFFER_COUNT 0x03000193 @@ -214,7 +316,7 @@ enum hfi_picture_type { #define HFI_SYSTEM_ERROR_BEGIN 0x05000000 -#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 +#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 #define HFI_SYSTEM_ERROR_END 0x05FFFFFF @@ -233,9 +335,9 @@ enum hfi_picture_type { #define HFI_INFORMATION_END 0x06FFFFFF struct hfi_debug_header { - u32 size; - u32 debug_level; - u32 reserved[2]; + u32 size; + u32 debug_level; + u32 reserved[2]; }; enum hfi_buffer_type { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index b1273d0..ca5406a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -30,6 +30,10 @@ #define INPUT_TIMER_LIST_SIZE 30 +#define CABAC_MAX_BITRATE 160000000 + +#define CAVLC_MAX_BITRATE 220000000 + enum domain_type { ENCODER = BIT(0), DECODER = BIT(1), diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c index 94fff74..a648cc1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -611,6 +611,45 @@ int prepare_dependency_list(struct iris_inst *inst) return ret; } +static inline bool is_layer_bitrate_set(struct iris_inst *inst) +{ + u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR}; + u32 cap_id = 0, i, enh_layer_count; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + for (i = 0; i <= enh_layer_count; i++) { + if (i >= ARRAY_SIZE(layer_br_caps)) + break; + + cap_id = layer_br_caps[i]; + if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET)) + return false; + } + + return true; +} + +static inline u32 get_cumulative_bitrate(struct iris_inst *inst) +{ + u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR}; + u32 cumulative_br = 0; + s32 enh_layer_count; + u32 cap_id = 0; + int i; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + for (i = 0; i <= enh_layer_count; i++) { + if (i >= ARRAY_SIZE(layer_br_caps)) + break; + cap_id = layer_br_caps[i]; + cumulative_br += inst->cap[cap_id].value; + } + + return cumulative_br; +} + int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id) { @@ -635,6 +674,18 @@ int set_u32(struct iris_inst *inst, &hfi_value, sizeof(u32)); } +int set_q16(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = inst->cap[cap_id].value; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_Q16, + &hfi_value, sizeof(u32)); +} + int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id) { @@ -662,7 +713,7 @@ int set_pipe(struct iris_inst *inst, { u32 work_route, hfi_id; - work_route = inst->cap[PIPE].value; + work_route = inst->cap[cap_id].value; hfi_id = inst->cap[cap_id].hfi_id; return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, @@ -671,6 +722,419 @@ int set_pipe(struct iris_inst *inst, &work_route, sizeof(u32)); } +int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = inst->cap[cap_id].value; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET)) + hfi_value = HFI_LEVEL_NONE; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_value, sizeof(u32)); +} + +int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + s32 prepend_sps_pps; + + prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value; + hfi_id = inst->cap[cap_id].hfi_id; + + if (prepend_sps_pps) + hfi_val = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR; + else + hfi_val = HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_val, sizeof(u32)); +} + +int set_flip(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + u32 hflip, vflip, ret = 0; + + u32 hfi_value = HFI_DISABLE_FLIP; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + hflip = inst->cap[HFLIP].value; + vflip = inst->cap[VFLIP].value; + + if (hflip) + hfi_value |= HFI_HORIZONTAL_FLIP; + + if (vflip) + hfi_value |= HFI_VERTICAL_FLIP; + + if (inst->vb2q_dst->streaming) { + if (hfi_value != HFI_DISABLE_FLIP) { + ret = set_req_sync_frame(inst, REQUEST_I_FRAME); + if (ret) + return ret; + } + } + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_value, sizeof(u32)); +} + +int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + + hfi_id = inst->cap[cap_id].hfi_id; + + switch (inst->cap[cap_id].value) { + case 0: + hfi_val = HFI_ROTATION_NONE; + break; + case 90: + hfi_val = HFI_ROTATION_90; + break; + case 180: + hfi_val = HFI_ROTATION_180; + break; + case 270: + hfi_val = HFI_ROTATION_270; + break; + default: + hfi_val = HFI_ROTATION_NONE; + break; + } + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 header_mode, hfi_id, hfi_val; + s32 prepend_sps_pps; + + prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value; + header_mode = inst->cap[cap_id].value; + hfi_id = inst->cap[cap_id].hfi_id; + + if (prepend_sps_pps) + hfi_val = HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME; + else if (header_mode == V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME) + hfi_val = HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME; + else + hfi_val = HFI_SEQ_HEADER_SEPERATE_FRAME; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_val, sizeof(u32)); +} + +int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value, hfi_id; + + if (inst->vb2q_dst->streaming) { + if (inst->hfi_layer_type == HFI_HIER_B) + return 0; + } + + hfi_value = inst->cap[GOP_SIZE].value; + hfi_id = inst->cap[cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + +int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) + goto set_total_bitrate; + + if (inst->vb2q_dst->streaming) + return 0; + +set_total_bitrate: + hfi_id = inst->cap[cap_id].hfi_id; + hfi_val = inst->cap[cap_id].value; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = 0; + u32 hfi_id; + + if (!inst->vb2q_dst->streaming) + return 0; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) + return 0; + + if (!inst->cap[ENH_LAYER_COUNT].max || + !is_layer_bitrate_set(inst)) + return 0; + + hfi_value = inst->cap[BIT_RATE].value; + hfi_id = inst->cap[BIT_RATE].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + +int set_peak_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + s32 rc_mode; + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_val = inst->cap[cap_id].value; + + rc_mode = inst->cap[BITRATE_MODE].value; + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) + return 0; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_use_and_mark_ltr(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_val = inst->cap[cap_id].value; + + if (!inst->cap[LTR_COUNT].value || + inst->cap[cap_id].value == INVALID_DEFAULT_MARK_OR_USE_LTR) + return 0; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_ir_period(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id = 0, hfi_val; + + hfi_val = inst->cap[cap_id].value; + + if (inst->cap[IR_TYPE].value == + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) { + hfi_id = HFI_PROP_IR_RANDOM_PERIOD; + } else if (inst->cap[IR_TYPE].value == + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC) { + hfi_id = HFI_PROP_IR_CYCLIC_PERIOD; + } + + return iris_hfi_set_ir_period(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, min_qp_enable = 0; + u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0; + u32 client_qp_enable = 0, hfi_value = 0; + u32 hfi_id; + + if (inst->cap[MIN_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + min_qp_enable = 1; + + if (min_qp_enable || + (inst->cap[I_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET)) + i_qp_enable = 1; + if (min_qp_enable || + (inst->cap[P_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET)) + p_qp_enable = 1; + if (min_qp_enable || + (inst->cap[B_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET)) + b_qp_enable = 1; + + client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2; + if (!client_qp_enable) + return 0; + + i_frame_qp = max(inst->cap[I_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value); + p_frame_qp = max(inst->cap[P_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value); + b_frame_qp = max(inst->cap[B_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value); + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 | + client_qp_enable << 24; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_32_PACKED, + &hfi_value, sizeof(u32)); +} + +int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, max_qp_enable = 0; + u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0; + u32 client_qp_enable = 0, hfi_value = 0; + u32 hfi_id; + + if (inst->cap[MAX_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + max_qp_enable = 1; + + if (max_qp_enable || + (inst->cap[I_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET)) + i_qp_enable = 1; + if (max_qp_enable || + (inst->cap[P_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET)) + p_qp_enable = 1; + if (max_qp_enable || + (inst->cap[B_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET)) + b_qp_enable = 1; + + client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2; + if (!client_qp_enable) + return 0; + + i_frame_qp = min(inst->cap[I_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value); + p_frame_qp = min(inst->cap[P_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value); + b_frame_qp = min(inst->cap[B_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value); + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 | + client_qp_enable << 24; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_32_PACKED, + &hfi_value, sizeof(u32)); +} + +int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0; + u32 client_qp_enable = 0, hfi_value = 0; + s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0; + s32 rc_type = -1; + u32 hfi_id; + + rc_type = inst->hfi_rc_type; + if (inst->vb2q_dst->streaming) { + if (rc_type != HFI_RC_OFF) + return 0; + } + + if (rc_type == HFI_RC_OFF) { + i_qp_enable = 1; + p_qp_enable = 1; + b_qp_enable = 1; + } else { + if (inst->cap[I_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + i_qp_enable = 1; + if (inst->cap[P_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + p_qp_enable = 1; + if (inst->cap[B_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + b_qp_enable = 1; + } + + client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2; + if (!client_qp_enable) + return 0; + + i_frame_qp = inst->cap[I_FRAME_QP].value; + p_frame_qp = inst->cap[P_FRAME_QP].value; + b_frame_qp = inst->cap[B_FRAME_QP].value; + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 | + client_qp_enable << 24; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_32_PACKED, + &hfi_value, sizeof(u32)); +} + +int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_layer_count, hfi_layer_type = 0; + int ret, hfi_id; + + if (!inst->vb2q_dst->streaming) { + hfi_layer_type = inst->hfi_layer_type; + hfi_id = inst->cap[LAYER_TYPE].hfi_id; + + ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, LAYER_TYPE), + HFI_PAYLOAD_U32_ENUM, + &hfi_layer_type, sizeof(u32)); + if (ret) + return ret; + } else { + if (inst->hfi_layer_type == HFI_HIER_B) + return 0; + } + + hfi_id = inst->cap[ENH_LAYER_COUNT].hfi_id; + hfi_layer_count = inst->cap[ENH_LAYER_COUNT].value + 1; + + ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, ENH_LAYER_COUNT), + HFI_PAYLOAD_U32, + &hfi_layer_count, sizeof(u32)); + + return ret; +} + +int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = 0, set_cap_id = 0, hfi_id; + s32 slice_mode = -1; + + slice_mode = inst->cap[SLICE_MODE].value; + + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) + return 0; + + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) { + hfi_value = (inst->codec == HEVC) ? + (inst->cap[SLICE_MAX_MB].value + 3) / 4 : + inst->cap[SLICE_MAX_MB].value; + set_cap_id = SLICE_MAX_MB; + } else if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES) { + hfi_value = inst->cap[SLICE_MAX_BYTES].value; + set_cap_id = SLICE_MAX_BYTES; + } + + hfi_id = inst->cap[set_cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, set_cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + int set_v4l2_properties(struct iris_inst *inst) { struct cap_entry *entry = NULL, *temp = NULL; @@ -723,8 +1187,7 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl) u32 adjusted_value; s32 pix_fmt = -1; - adjusted_value = ctrl ? ctrl->val : - inst->cap[PROFILE].value; + adjusted_value = ctrl ? ctrl->val : inst->cap[PROFILE].value; pix_fmt = inst->cap[PIX_FMTS].value; @@ -737,3 +1200,515 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl) return 0; } + +int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR}; + u32 adjusted_value, cumulative_bitrate, cap_id, cap_val, i; + s32 layer_count, max_bitrate = 0, entropy_mode; + + adjusted_value = ctrl ? ctrl->val : inst->cap[BIT_RATE].value; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) { + inst->cap[BIT_RATE].value = adjusted_value; + return 0; + } + + entropy_mode = inst->cap[ENTROPY_MODE].value; + + if (inst->codec == HEVC) + max_bitrate = CABAC_MAX_BITRATE; + + if (inst->codec == H264) { + if (entropy_mode == V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) + max_bitrate = CABAC_MAX_BITRATE; + else + max_bitrate = CAVLC_MAX_BITRATE; + } + + if (inst->cap[BIT_RATE].value > max_bitrate) + inst->cap[BIT_RATE].value = max_bitrate; + + layer_count = inst->cap[ENH_LAYER_COUNT].value; + if (!layer_count) + return 0; + + if (!is_layer_bitrate_set(inst)) + return 0; + + cumulative_bitrate = get_cumulative_bitrate(inst); + + if (cumulative_bitrate > max_bitrate) { + u32 decrement_in_value = 0; + u32 decrement_in_percent = ((cumulative_bitrate - max_bitrate) * 100) / + max_bitrate; + + cumulative_bitrate = 0; + for (i = 0; i <= layer_count; i++) { + if (i >= ARRAY_SIZE(layer_br_caps)) + break; + cap_id = layer_br_caps[i]; + cap_val = inst->cap[cap_id].value; + decrement_in_value = (cap_val * decrement_in_percent) / 100; + cumulative_bitrate += (cap_val - decrement_in_value); + inst->cap[cap_id].value = cap_val - decrement_in_value; + } + inst->cap[BIT_RATE].value = cumulative_bitrate; + } + + return 0; +} + +int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + u32 old_br = 0, new_br = 0, exceeded_br = 0; + u32 client_set_cap_id = INST_CAP_NONE; + u32 cumulative_bitrate = 0; + s32 max_bitrate; + + if (!ctrl) + return 0; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET || + !inst->vb2q_dst->streaming) + return 0; + + if (!inst->cap[ENH_LAYER_COUNT].max) + return -EINVAL; + + if (!is_layer_bitrate_set(inst)) + return 0; + + client_set_cap_id = get_cap_id(inst, ctrl->id); + if (!is_valid_cap_id(client_set_cap_id)) + return -EINVAL; + + cumulative_bitrate = get_cumulative_bitrate(inst); + max_bitrate = inst->cap[BIT_RATE].max; + old_br = inst->cap[client_set_cap_id].value; + new_br = ctrl->val; + + if ((cumulative_bitrate - old_br + new_br) > max_bitrate) { + exceeded_br = (cumulative_bitrate - old_br + new_br) - max_bitrate; + new_br = ctrl->val - exceeded_br; + } + inst->cap[client_set_cap_id].value = new_br; + + inst->cap[BIT_RATE].value = get_cumulative_bitrate(inst); + + return 0; +} + +int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + u32 adjusted_value; + s32 rc_mode, bitrate; + + adjusted_value = ctrl ? ctrl->val : inst->cap[PEAK_BITRATE].value; + + rc_mode = inst->cap[BITRATE_MODE].value; + + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) + return 0; + + bitrate = inst->cap[BIT_RATE].value; + + if (inst->cap[PEAK_BITRATE].flags & CAP_FLAG_CLIENT_SET) { + if (adjusted_value < bitrate) + adjusted_value = bitrate; + } else { + adjusted_value = inst->cap[BIT_RATE].value; + } + + inst->cap[PEAK_BITRATE].value = adjusted_value; + + return 0; +} + +int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 frame_rc, bitrate_mode, frame_skip; + + bitrate_mode = inst->cap[BITRATE_MODE].value; + frame_rc = inst->cap[FRAME_RC_ENABLE].value; + frame_skip = inst->cap[FRAME_SKIP_MODE].value; + + if (!frame_rc) { + inst->hfi_rc_type = HFI_RC_OFF; + return 0; + } + + if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) { + inst->hfi_rc_type = HFI_RC_VBR_CFR; + } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) { + if (frame_skip) + inst->hfi_rc_type = HFI_RC_CBR_VFR; + else + inst->hfi_rc_type = HFI_RC_CBR_CFR; + } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) { + inst->hfi_rc_type = HFI_RC_CQ; + } + + return 0; +} + +int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, enh_layer_count; + u32 min_gop_size, num_subgops; + + adjusted_value = ctrl ? ctrl->val : inst->cap[GOP_SIZE].value; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + if (!enh_layer_count) + goto exit; + + /* + * Layer encoding needs GOP size to be multiple of subgop size + * And subgop size is 2 ^ number of enhancement layers. + */ + min_gop_size = 1 << enh_layer_count; + num_subgops = (adjusted_value + (min_gop_size >> 1)) / min_gop_size; + if (num_subgops) + adjusted_value = num_subgops * min_gop_size; + else + adjusted_value = min_gop_size; + +exit: + inst->cap[GOP_SIZE].value = adjusted_value; + + return 0; +} + +int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, enh_layer_count = -1; + const u32 max_bframe_size = 7; + + adjusted_value = ctrl ? ctrl->val : inst->cap[B_FRAME].value; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + if (!enh_layer_count || inst->hfi_layer_type != HFI_HIER_B) { + adjusted_value = 0; + goto exit; + } + + adjusted_value = (1 << enh_layer_count) - 1; + + if (adjusted_value > max_bframe_size) + adjusted_value = max_bframe_size; + +exit: + inst->cap[B_FRAME].value = adjusted_value; + + return 0; +} + +int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, rc_mode, pix_fmt; + + adjusted_value = ctrl ? ctrl->val : inst->cap[LTR_COUNT].value; + + rc_mode = inst->cap[BITRATE_MODE].value; + + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR || + inst->hfi_rc_type != HFI_RC_OFF) + adjusted_value = 0; + + pix_fmt = inst->cap[PIX_FMTS].value; + if (is_10bit_colorformat(pix_fmt)) + adjusted_value = 0; + + inst->cap[LTR_COUNT].value = adjusted_value; + + return 0; +} + +int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, ltr_count; + + adjusted_value = ctrl ? ctrl->val : inst->cap[USE_LTR].value; + + ltr_count = inst->cap[LTR_COUNT].value; + if (!ltr_count) + return 0; + + /* + * USE_LTR is bitmask value, hence should be + * > 0 and <= (2 ^ LTR_COUNT) - 1 + */ + if (adjusted_value <= 0 || + adjusted_value > (1 << ltr_count) - 1) + return 0; + + inst->cap[USE_LTR].value = adjusted_value; + + return 0; +} + +int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, ltr_count; + + adjusted_value = ctrl ? ctrl->val : inst->cap[MARK_LTR].value; + + ltr_count = inst->cap[LTR_COUNT].value; + if (!ltr_count) + return 0; + + if (adjusted_value < 0 || adjusted_value > ltr_count - 1) + return 0; + + inst->cap[MARK_LTR].value = adjusted_value; + + return 0; +} + +int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, pix_fmt, rc_mode; + + adjusted_value = ctrl ? ctrl->val : inst->cap[IR_PERIOD].value; + + pix_fmt = inst->cap[PIX_FMTS].value; + if (is_10bit_colorformat(pix_fmt)) + adjusted_value = 0; + + rc_mode = inst->cap[BITRATE_MODE].value; + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) + adjusted_value = 0; + + inst->cap[IR_PERIOD].value = adjusted_value; + + return 0; +} + +int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, pix_fmt, rc_mode, layer_count; + u32 width, height, frame_rate; + struct v4l2_format *f; + + if (inst->vb2q_dst->streaming) + return 0; + + adjusted_value = MAX_SUPPORTED_MIN_QUALITY; + + rc_mode = inst->cap[BITRATE_MODE].value; + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) + adjusted_value = 0; + + layer_count = inst->cap[ENH_LAYER_COUNT].value; + if (layer_count && inst->hfi_layer_type != HFI_HIER_B) + adjusted_value = 0; + + pix_fmt = inst->cap[PIX_FMTS].value; + if (is_10bit_colorformat(pix_fmt)) + adjusted_value = 0; + + frame_rate = inst->cap[FRAME_RATE].value >> 16; + f = inst->fmt_dst; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (!res_is_less_than(width, height, 1920, 1080)) + adjusted_value = 0; + + if (frame_rate > 60) + adjusted_value = 0; + + inst->cap[MIN_QUALITY].value = adjusted_value; + + return 0; +} + +static int adjust_static_layer_count_and_type(struct iris_inst *inst, s32 layer_count) +{ + bool hb_requested = false; + s32 max_enh_count = 0; + + if (!layer_count) + goto exit; + + if (inst->hfi_rc_type == HFI_RC_CQ) { + layer_count = 0; + goto exit; + } + + if (inst->codec == H264) { + if (!inst->cap[LAYER_ENABLE].value) { + layer_count = 0; + goto exit; + } + hb_requested = inst->cap[LAYER_TYPE].value == + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B; + } else if (inst->codec == HEVC) { + hb_requested = inst->cap[LAYER_TYPE].value == + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B; + } + + if (hb_requested && inst->hfi_rc_type != HFI_RC_VBR_CFR) { + layer_count = 0; + goto exit; + } + + inst->hfi_layer_type = hb_requested ? HFI_HIER_B : + (inst->codec == H264 && inst->hfi_rc_type == HFI_RC_VBR_CFR) ? + HFI_HIER_P_HYBRID_LTR : HFI_HIER_P_SLIDING_WINDOW; + + max_enh_count = inst->hfi_layer_type == HFI_HIER_B ? MAX_ENH_LAYER_HB : + inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR ? MAX_AVC_ENH_LAYER_HYBRID_HP : + inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW ? + (inst->codec == H264 ? MAX_AVC_ENH_LAYER_SLIDING_WINDOW : + (inst->codec == HEVC && inst->hfi_rc_type == HFI_RC_VBR_CFR) ? + MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW : + MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW) : + layer_count; + + layer_count = min(layer_count, max_enh_count); + +exit: + inst->cap[ENH_LAYER_COUNT].value = layer_count; + inst->cap[ENH_LAYER_COUNT].max = layer_count; + + return 0; +} + +int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 client_layer_count; + int ret = 0; + + client_layer_count = ctrl ? ctrl->val : inst->cap[ENH_LAYER_COUNT].value; + + if (!inst->vb2q_dst->streaming) { + ret = adjust_static_layer_count_and_type(inst, client_layer_count); + if (ret) + return ret; + } else { + if (inst->hfi_rc_type == HFI_RC_CBR_CFR || + inst->hfi_rc_type == HFI_RC_CBR_VFR) + return ret; + + if (inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR || + inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW) + inst->cap[ENH_LAYER_COUNT].value = + min(client_layer_count, inst->cap[ENH_LAYER_COUNT].max); + } + + return ret; +} + +int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value; + s32 profile = -1; + + adjusted_value = ctrl ? ctrl->val : inst->cap[ENTROPY_MODE].value; + + profile = inst->cap[PROFILE].value; + if (profile == V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE || + profile == V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) + adjusted_value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC; + + inst->cap[ENTROPY_MODE].value = adjusted_value; + + return 0; +} + +int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, rc_type = -1, slice_mode, enh_layer_count = 0; + u32 slice_val, mbpf = 0, mbps = 0, max_mbpf = 0, max_mbps = 0, bitrate = 0; + u32 update_cap, max_avg_slicesize, output_width, output_height; + u32 min_width, min_height, max_width, max_height, fps; + + slice_mode = ctrl ? ctrl->val : inst->cap[SLICE_MODE].value; + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) + return 0; + + bitrate = inst->cap[BIT_RATE].value; + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + if (enh_layer_count && is_layer_bitrate_set(inst)) + bitrate = get_cumulative_bitrate(inst); + + rc_type = inst->hfi_rc_type; + fps = inst->cap[FRAME_RATE].value >> 16; + if (fps > MAX_SLICES_FRAME_RATE || + (rc_type != HFI_RC_OFF && rc_type != HFI_RC_CBR_CFR && + rc_type != HFI_RC_CBR_VFR && rc_type != HFI_RC_VBR_CFR)) { + adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE; + update_cap = SLICE_MODE; + goto exit; + } + + output_width = inst->fmt_dst->fmt.pix_mp.width; + output_height = inst->fmt_dst->fmt.pix_mp.height; + + max_width = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ? + MAX_MB_SLICE_WIDTH : MAX_BYTES_SLICE_WIDTH; + max_height = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ? + MAX_MB_SLICE_HEIGHT : MAX_BYTES_SLICE_HEIGHT; + min_width = (inst->codec == HEVC) ? + MIN_HEVC_SLICE_WIDTH : MIN_AVC_SLICE_WIDTH; + min_height = MIN_SLICE_HEIGHT; + + if (output_width < min_width || output_height < min_height || + output_width > max_width || output_height > max_width) { + adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE; + update_cap = SLICE_MODE; + goto exit; + } + + mbpf = NUM_MBS_PER_FRAME(output_height, output_width); + mbps = mbpf * fps; + max_mbpf = NUM_MBS_PER_FRAME(max_height, max_width); + max_mbps = max_mbpf * MAX_SLICES_FRAME_RATE; + + if (mbpf > max_mbpf || mbps > max_mbps) { + adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE; + update_cap = SLICE_MODE; + goto exit; + } + + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) { + update_cap = SLICE_MAX_MB; + slice_val = inst->cap[SLICE_MAX_MB].value; + slice_val = max(slice_val, mbpf / MAX_SLICES_PER_FRAME); + } else { + slice_val = inst->cap[SLICE_MAX_BYTES].value; + update_cap = SLICE_MAX_BYTES; + if (rc_type != HFI_RC_OFF) { + max_avg_slicesize = + ((bitrate / fps) / 8) / MAX_SLICES_PER_FRAME; + slice_val = max(slice_val, max_avg_slicesize); + } + } + adjusted_value = slice_val; + +exit: + inst->cap[update_cap].value = adjusted_value; + + return 0; +} + +int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value; + s32 profile = -1; + + adjusted_value = ctrl ? ctrl->val : inst->cap[TRANSFORM_8X8].value; + + profile = inst->cap[PROFILE].value; + if (profile != V4L2_MPEG_VIDEO_H264_PROFILE_HIGH && + profile != V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) + adjusted_value = 0; + + inst->cap[TRANSFORM_8X8].value = adjusted_value; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h index 28ce767..5421d9f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h @@ -31,5 +31,40 @@ int get_inst_capability(struct iris_inst *inst); int set_v4l2_properties(struct iris_inst *inst); int adjust_v4l2_properties(struct iris_inst *inst); int ctrls_init(struct iris_inst *inst, bool init); +int set_q16(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_flip(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_peak_bitrate(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); +int set_use_and_mark_ltr(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); +int set_ir_period(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); +int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 395a189..d9d22e2 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -40,12 +40,20 @@ bool res_is_less_than(u32 width, u32 height, u32 get_port_info(struct iris_inst *inst, enum plat_inst_cap_type cap_id) { + if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT && + inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) { + if (inst->vb2q_dst->streaming) + return get_hfi_port(INPUT_MPLANE); + else + return get_hfi_port(OUTPUT_MPLANE); + } + if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT) - return HFI_PORT_BITSTREAM; + return get_hfi_port(INPUT_MPLANE); else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) - return HFI_PORT_RAW; - - return HFI_PORT_NONE; + return get_hfi_port(OUTPUT_MPLANE); + else + return HFI_PORT_NONE; } enum iris_buffer_type v4l2_type_to_driver(u32 type) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 2850fd5..00e598d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -668,6 +668,59 @@ int iris_hfi_set_property(struct iris_inst *inst, return ret; } +int iris_hfi_set_ir_period(struct iris_inst *inst, + u32 packet_type, u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + u32 sync_frame_req = 0; + struct iris_core *core; + int ret; + + core = inst->core; + + mutex_lock(&core->lock); + + ret = hfi_create_header(inst->packet, inst->packet_size, + inst->session_id, core->header_id++); + if (ret) + goto exit; + + if (!inst->ir_enabled) { + inst->ir_enabled = ((*(u32 *)payload > 0) ? true : false); + if (inst->ir_enabled && inst->vb2q_dst->streaming) { + sync_frame_req = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR; + ret = hfi_create_packet(inst->packet, inst->packet_size, + HFI_PROP_REQUEST_SYNC_FRAME, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32_ENUM, + HFI_PORT_BITSTREAM, + core->packet_id++, + &sync_frame_req, + sizeof(u32)); + if (ret) + goto exit; + } + } + + ret = hfi_create_packet(inst->packet, inst->packet_size, + packet_type, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + plane, + core->packet_id++, + payload, + sizeof(u32)); + if (ret) + goto exit; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +exit: + mutex_unlock(&core->lock); + + return ret; +} + int iris_hfi_queue_buffer(struct iris_inst *inst, struct iris_buffer *buffer) { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index 465bfc5..95e0523 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -40,9 +40,8 @@ int prepare_pc(struct iris_core *core); irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); -int iris_hfi_queue_buffer(struct iris_inst *inst, - struct iris_buffer *buffer); -int iris_hfi_release_buffer(struct iris_inst *inst, - struct iris_buffer *buffer); +int iris_hfi_set_ir_period(struct iris_inst *inst, + u32 packet_type, u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index fab206cf..dd27fa4 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -364,8 +364,8 @@ int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) return 0; } -static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, - u32 header_id) +int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, + u32 header_id) { struct hfi_header *hdr = (struct hfi_header *)packet; @@ -382,9 +382,9 @@ static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, return 0; } -static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, - u32 pkt_flags, u32 payload_type, u32 port, - u32 packet_id, void *payload, u32 payload_size) +int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, + u32 pkt_flags, u32 payload_type, u32 port, + u32 packet_id, void *payload, u32 payload_size) { struct hfi_header *hdr; struct hfi_packet *pkt; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index 849b585..82148b7 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -108,5 +108,10 @@ int hfi_packet_sys_interframe_powercollapse(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_sys_pc_prep(struct iris_core *core, u8 *pkt, u32 pkt_size); +int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, + u32 header_id); +int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, + u32 pkt_flags, u32 payload_type, u32 port, + u32 packet_id, void *payload, u32 payload_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index a4d5d9c..1b667a5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -758,6 +758,12 @@ static int handle_session_property(struct iris_inst *inst, case HFI_PROP_PICTURE_TYPE: inst->hfi_frame_info.picture_type = payload_ptr[0]; break; + case HFI_PROP_CABAC_SESSION: + if (payload_ptr[0] == 1) + inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC; + else + inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC; + break; case HFI_PROP_DPB_LIST: ret = handle_dpb_list_property(inst, pkt); if (ret) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 3c85e78..70f4c7d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -52,6 +52,9 @@ * @power: structure of power info * @bus_data: structure of bus data * @input_timer_list: list head of input timer + * @ir_enabled: boolean for intra refresh + * @hfi_rc_type: rate control type + * @hfi_layer_type: type of HFI layer encoding */ struct iris_inst { @@ -90,6 +93,9 @@ struct iris_inst { struct iris_inst_power power; struct bus_vote_data bus_data; struct list_head input_timer_list; + bool ir_enabled; + u32 hfi_rc_type; + u32 hfi_layer_type; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 81de610..443894c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -20,9 +20,29 @@ struct iris_inst; #define BIT_DEPTH_8 (8 << 16 | 8) #define BIT_DEPTH_10 (10 << 16 | 10) -#define CODED_FRAMES_PROGRESSIVE 0x0 -#define CODED_FRAMES_INTERLACE 0x1 -#define MAX_NUM_CHILD 10 +#define CODED_FRAMES_PROGRESSIVE 0x0 +#define CODED_FRAMES_INTERLACE 0x1 +#define MAX_NUM_CHILD 10 +#define MAX_ENH_LAYER_HB 3 +#define MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW 5 +#define MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW 3 +#define MAX_AVC_ENH_LAYER_SLIDING_WINDOW 3 +#define MAX_AVC_ENH_LAYER_HYBRID_HP 5 +#define MAX_SLICES_FRAME_RATE 60 +#define MAX_MB_SLICE_WIDTH 4096 +#define MAX_MB_SLICE_HEIGHT 2160 +#define MAX_BYTES_SLICE_WIDTH 1920 +#define MAX_BYTES_SLICE_HEIGHT 1088 +#define MIN_HEVC_SLICE_WIDTH 384 +#define MIN_AVC_SLICE_WIDTH 192 +#define MIN_SLICE_HEIGHT 128 +#define MAX_SLICES_PER_FRAME 10 +#define MIN_QP_8BIT 1 +#define MIN_SLICE_BYTE_SIZE 512 +#define MAX_SLICE_BYTE_SIZE ((MAX_BITRATE) >> 3) +#define MAX_SLICE_MB_SIZE (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) +#define INVALID_DEFAULT_MARK_OR_USE_LTR -1 +#define MAX_SUPPORTED_MIN_QUALITY 70 #define DEFAULT_MAX_HOST_BUF_COUNT 64 #define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256 @@ -128,12 +148,17 @@ enum plat_inst_cap_type { PIX_FMTS, MBPF, QUEUED_RATE, + FRAME_RATE, + OPERATING_RATE, MB_CYCLES_VSP, MB_CYCLES_VPP, MB_CYCLES_LP, MB_CYCLES_FW, MB_CYCLES_FW_VPP, NUM_COMV, + ENTROPY_MODE, + BASELAYER_PRIORITY, + IR_TYPE, PROFILE, LEVEL, HEVC_TIER, @@ -148,6 +173,48 @@ enum plat_inst_cap_type { BIT_DEPTH, DEFAULT_HEADER, RAP_FRAME, + MIN_FRAME_QP, + MAX_FRAME_QP, + B_FRAME, + I_FRAME_QP, + P_FRAME_QP, + B_FRAME_QP, + BIT_RATE, + PEAK_BITRATE, + BITRATE_MODE, + FRAME_SKIP_MODE, + FRAME_RC_ENABLE, + GOP_SIZE, + MIN_QUALITY, + IR_PERIOD, + LTR_COUNT, + USE_LTR, + MARK_LTR, + I_FRAME_MIN_QP, + P_FRAME_MIN_QP, + B_FRAME_MIN_QP, + I_FRAME_MAX_QP, + P_FRAME_MAX_QP, + B_FRAME_MAX_QP, + LAYER_TYPE, + LAYER_ENABLE, + L0_BR, + L1_BR, + L2_BR, + L3_BR, + L4_BR, + L5_BR, + ENH_LAYER_COUNT, + TRANSFORM_8X8, + SLICE_MODE, + SLICE_MAX_MB, + SLICE_MAX_BYTES, + HFLIP, + VFLIP, + ROTATION, + HEADER_MODE, + PREPEND_SPSPPS_TO_IDR, + REQUEST_I_FRAME, INST_CAP_MAX, }; @@ -164,6 +231,7 @@ enum plat_inst_cap_flags { struct plat_inst_cap { enum plat_inst_cap_type cap_id; + enum domain_type domain; enum codec_type codec; s32 min; s32 max; diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 7ae9715..6d5192a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -13,11 +13,19 @@ #include "platform_common.h" #include "resources.h" -#define CODECS_ALL (H264 | HEVC | VP9) - -#define DEFAULT_FPS 30 -#define MINIMUM_FPS 1 -#define MAXIMUM_FPS 480 +#define CODECS_ALL (H264 | HEVC | VP9) +#define ENC ENCODER +#define DEC DECODER + +#define DEFAULT_FPS 30 +#define MINIMUM_FPS 1 +#define MAXIMUM_FPS 480 +#define MAX_BITRATE 245000000 +#define DEFAULT_BITRATE 20000000 +#define MAX_LTR_FRAME_COUNT 2 +#define MAX_BASE_LAYER_PRIORITY_ID 63 +#define MAX_QP 51 +#define DEFAULT_QP 20 static struct codec_info codec_data_sm8550[] = { { @@ -68,21 +76,37 @@ static struct plat_core_cap core_data_sm8550[] = { }; static struct plat_inst_cap instance_cap_data_sm8550[] = { - {FRAME_WIDTH, CODECS_ALL, 96, 8192, 1, 1920}, + {FRAME_WIDTH, DEC, CODECS_ALL, 96, 8192, 1, 1920}, + + {FRAME_WIDTH, DEC, VP9, 96, 4096, 1, 1920}, + + {FRAME_WIDTH, ENC, CODECS_ALL, 128, 8192, 1, 1920}, - {FRAME_WIDTH, VP9, 96, 4096, 1, 1920}, + {FRAME_WIDTH, ENC, HEVC, 96, 8192, 1, 1920}, - {FRAME_HEIGHT, CODECS_ALL, 96, 8192, 1, 1080}, + {FRAME_HEIGHT, DEC, CODECS_ALL, 96, 8192, 1, 1080}, - {FRAME_HEIGHT, VP9, 96, 4096, 1, 1080}, + {FRAME_HEIGHT, DEC, VP9, 96, 4096, 1, 1080}, - {PIX_FMTS, H264, + {FRAME_HEIGHT, ENC, CODECS_ALL, 128, 8192, 1, 1080}, + + {FRAME_HEIGHT, ENC, HEVC, 96, 8192, 1, 1080}, + + {PIX_FMTS, DEC, H264, FMT_NV12, FMT_NV12C, FMT_NV12 | FMT_NV21 | FMT_NV12C, FMT_NV12C}, - {PIX_FMTS, HEVC, + {PIX_FMTS, ENC, H264, + FMT_NV12, + FMT_NV12C, + FMT_NV12 | FMT_NV21 | FMT_NV12C, + FMT_NV12C, + 0, 0, + CAP_FLAG_NONE}, + + {PIX_FMTS, DEC, HEVC, FMT_NV12, FMT_TP10C, FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, @@ -91,37 +115,73 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { CAP_FLAG_NONE, {PROFILE}}, - {PIX_FMTS, VP9, + {PIX_FMTS, ENC, HEVC, + FMT_NV12, + FMT_TP10C, + FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, + FMT_NV12C, + 0, 0, + CAP_FLAG_NONE, + {PROFILE, MIN_QUALITY, IR_PERIOD, LTR_COUNT}}, + + {PIX_FMTS, DEC, VP9, FMT_NV12, FMT_TP10C, FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, FMT_NV12C}, - {MBPF, CODECS_ALL, 36, 138240, 1, 138240}, + {MBPF, DEC, CODECS_ALL, 36, 138240, 1, 138240}, /* (4096 * 2304) / 256 */ - {MBPF, VP9, 36, 36864, 1, 36864}, + {MBPF, DEC, VP9, 36, 36864, 1, 36864}, + + /* (8192 * 4320) / 256 */ + {MBPF, ENC, CODECS_ALL, 64, 138240, 1, 138240}, + + {MBPF, ENC, HEVC, 36, 138240, 1, 138240}, - {QUEUED_RATE, CODECS_ALL, + {QUEUED_RATE, DEC | ENC, CODECS_ALL, (MINIMUM_FPS << 16), INT_MAX, 1, (DEFAULT_FPS << 16)}, - {MB_CYCLES_VSP, CODECS_ALL, 25, 25, 1, 25}, + {FRAME_RATE, ENC, CODECS_ALL, + (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16), + 1, (DEFAULT_FPS << 16), + 0, + HFI_PROP_FRAME_RATE, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_q16}, + + {OPERATING_RATE, ENC, CODECS_ALL, + (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16), + 1, (DEFAULT_FPS << 16)}, + + {MB_CYCLES_VSP, DEC, CODECS_ALL, 25, 25, 1, 25}, + + {MB_CYCLES_VSP, DEC, VP9, 60, 60, 1, 60}, + + {MB_CYCLES_VSP, ENC, CODECS_ALL, 25, 25, 1, 25}, - {MB_CYCLES_VSP, VP9, 60, 60, 1, 60}, + {MB_CYCLES_VPP, DEC, CODECS_ALL, 200, 200, 1, 200}, - {MB_CYCLES_VPP, CODECS_ALL, 200, 200, 1, 200}, + {MB_CYCLES_VPP, ENC, CODECS_ALL, 675, 675, 1, 675}, - {MB_CYCLES_LP, CODECS_ALL, 200, 200, 1, 200}, + {MB_CYCLES_LP, DEC, CODECS_ALL, 200, 200, 1, 200}, - {MB_CYCLES_FW, CODECS_ALL, 489583, 489583, 1, 489583}, + {MB_CYCLES_LP, ENC, CODECS_ALL, 320, 320, 1, 320}, - {MB_CYCLES_FW_VPP, CODECS_ALL, 66234, 66234, 1, 66234}, + {MB_CYCLES_FW, DEC | ENC, CODECS_ALL, 489583, 489583, 1, 489583}, - {NUM_COMV, CODECS_ALL, + {MB_CYCLES_FW_VPP, DEC, CODECS_ALL, 66234, 66234, 1, 66234}, + + {MB_CYCLES_FW_VPP, ENC, CODECS_ALL, 48405, 48405, 1, 48405}, + + {NUM_COMV, DEC, CODECS_ALL, 0, INT_MAX, 1, 0}, - {PROFILE, H264, + {PROFILE, DEC, H264, V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH, BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | @@ -137,7 +197,23 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {PROFILE, HEVC, + {PROFILE, ENC, H264, + V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH, + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH), + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + V4L2_CID_MPEG_VIDEO_H264_PROFILE, + HFI_PROP_PROFILE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {ENTROPY_MODE, TRANSFORM_8X8}, + NULL, + set_u32_enum}, + + {PROFILE, ENC | DEC, HEVC, V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10, BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) | @@ -151,7 +227,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { adjust_profile, set_u32_enum}, - {PROFILE, VP9, + {PROFILE, DEC, VP9, V4L2_MPEG_VIDEO_VP9_PROFILE_0, V4L2_MPEG_VIDEO_VP9_PROFILE_2, BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) | @@ -164,7 +240,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {LEVEL, H264, + {LEVEL, DEC, H264, V4L2_MPEG_VIDEO_H264_LEVEL_1_0, V4L2_MPEG_VIDEO_H264_LEVEL_6_2, BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) | @@ -195,7 +271,36 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {LEVEL, HEVC, + {LEVEL, ENC, H264, + V4L2_MPEG_VIDEO_H264_LEVEL_1_0, + V4L2_MPEG_VIDEO_H264_LEVEL_6_0, + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0), + V4L2_MPEG_VIDEO_H264_LEVEL_5_0, + V4L2_CID_MPEG_VIDEO_H264_LEVEL, + HFI_PROP_LEVEL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_level}, + + {LEVEL, DEC, HEVC, V4L2_MPEG_VIDEO_HEVC_LEVEL_1, V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2, BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) | @@ -219,7 +324,31 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {LEVEL, VP9, + {LEVEL, ENC, HEVC, + V4L2_MPEG_VIDEO_HEVC_LEVEL_1, + V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2, + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2), + V4L2_MPEG_VIDEO_HEVC_LEVEL_5, + V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, + HFI_PROP_LEVEL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_level}, + + {LEVEL, DEC, VP9, V4L2_MPEG_VIDEO_VP9_LEVEL_1_0, V4L2_MPEG_VIDEO_VP9_LEVEL_6_0, BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) | @@ -242,7 +371,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {HEVC_TIER, HEVC, + {HEVC_TIER, DEC | ENC, HEVC, V4L2_MPEG_VIDEO_HEVC_TIER_MAIN, V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) | @@ -255,7 +384,566 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {DISPLAY_DELAY_ENABLE, CODECS_ALL, + {HFLIP, ENC, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_HFLIP, + HFI_PROP_FLIP, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_flip}, + + {VFLIP, ENC, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_VFLIP, + HFI_PROP_FLIP, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_flip}, + + {ROTATION, ENC, CODECS_ALL, + 0, 270, 90, 0, + V4L2_CID_ROTATE, + HFI_PROP_ROTATION, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_rotation}, + + {HEADER_MODE, ENC, CODECS_ALL, + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE, + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME, + BIT(V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) | + BIT(V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME), + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE, + V4L2_CID_MPEG_VIDEO_HEADER_MODE, + HFI_PROP_SEQ_HEADER_MODE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_header_mode}, + + {PREPEND_SPSPPS_TO_IDR, ENC, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR}, + + {REQUEST_I_FRAME, ENC, H264 | HEVC, + 0, 0, 0, 0, + V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME, + HFI_PROP_REQUEST_SYNC_FRAME, + CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_req_sync_frame}, + + {BIT_RATE, ENC, H264 | HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_BITRATE, + HFI_PROP_TOTAL_BITRATE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {PEAK_BITRATE, L0_BR}, + adjust_bitrate, + set_bitrate}, + + {PEAK_BITRATE, ENC, H264 | HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_BITRATE_PEAK, + HFI_PROP_TOTAL_PEAK_BITRATE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_peak_bitrate, + set_peak_bitrate}, + + {BITRATE_MODE, ENC, H264 | HEVC, + V4L2_MPEG_VIDEO_BITRATE_MODE_VBR, + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, + BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) | + BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_CBR), + V4L2_MPEG_VIDEO_BITRATE_MODE_VBR, + V4L2_CID_MPEG_VIDEO_BITRATE_MODE, + HFI_PROP_RATE_CONTROL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {LTR_COUNT, IR_PERIOD, I_FRAME_QP, P_FRAME_QP, + B_FRAME_QP, ENH_LAYER_COUNT, BIT_RATE, + MIN_QUALITY, PEAK_BITRATE, SLICE_MODE}, + adjust_bitrate_mode, + set_u32_enum}, + + {FRAME_SKIP_MODE, ENC, H264 | HEVC, + V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED, + V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT, + BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED) | + BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_LEVEL_LIMIT) | + BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT), + V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED, + V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE, + 0, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {FRAME_RC_ENABLE, ENC, H264 | HEVC, + 0, 1, 1, 1, + V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE}, + + {GOP_SIZE, ENC, CODECS_ALL, + 0, INT_MAX, 1, 2 * DEFAULT_FPS - 1, + V4L2_CID_MPEG_VIDEO_GOP_SIZE, + HFI_PROP_MAX_GOP_FRAMES, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_gop_size, + set_gop_size}, + + {B_FRAME, ENC, H264 | HEVC, + 0, 7, 1, 0, + V4L2_CID_MPEG_VIDEO_B_FRAMES, + HFI_PROP_MAX_B_FRAMES, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_b_frame, + set_u32}, + + {LTR_COUNT, ENC, H264 | HEVC, + 0, MAX_LTR_FRAME_COUNT, 1, 0, + V4L2_CID_MPEG_VIDEO_LTR_COUNT, + HFI_PROP_LTR_COUNT, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_ltr_count, + set_u32}, + + {USE_LTR, ENC, H264 | HEVC, + 0, + ((1 << MAX_LTR_FRAME_COUNT) - 1), + 0, 0, + V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES, + HFI_PROP_LTR_USE, + CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_use_ltr, + set_use_and_mark_ltr}, + + {MARK_LTR, ENC, H264 | HEVC, + INVALID_DEFAULT_MARK_OR_USE_LTR, + (MAX_LTR_FRAME_COUNT - 1), + 1, INVALID_DEFAULT_MARK_OR_USE_LTR, + V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX, + HFI_PROP_LTR_MARK, + CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_mark_ltr, + set_use_and_mark_ltr}, + + {BASELAYER_PRIORITY, ENC, H264, + 0, MAX_BASE_LAYER_PRIORITY_ID, 1, 0, + V4L2_CID_MPEG_VIDEO_BASELAYER_PRIORITY_ID, + HFI_PROP_BASELAYER_PRIORITYID, + CAP_FLAG_OUTPUT_PORT}, + + {IR_TYPE, ENC, H264 | HEVC, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC, + BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) | + BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC), + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE, + 0, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {IR_PERIOD, ENC, H264 | HEVC, + 0, INT_MAX, 1, 0, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD, + 0, + CAP_FLAG_INPUT_PORT | CAP_FLAG_OUTPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_ir_period, + set_ir_period}, + + {MIN_QUALITY, ENC, H264 | HEVC, + 0, MAX_SUPPORTED_MIN_QUALITY, 70, MAX_SUPPORTED_MIN_QUALITY, + 0, + HFI_PROP_MAINTAIN_MIN_QUALITY, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_min_quality, + set_u32}, + + {MIN_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_MIN_QP, + HFI_PROP_MIN_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_min_qp}, + + {MIN_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP, + HFI_PROP_MIN_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_min_qp}, + + {MAX_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_MAX_QP, + HFI_PROP_MAX_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_max_qp}, + + {MAX_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP, + HFI_PROP_MAX_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_max_qp}, + + {I_FRAME_MIN_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MIN_QP}, + + {I_FRAME_MIN_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MIN_QP}, + + {P_FRAME_MIN_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MIN_QP}, + + {P_FRAME_MIN_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MIN_QP}, + + {B_FRAME_MIN_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MIN_QP}, + + {B_FRAME_MIN_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MIN_QP}, + + {I_FRAME_MAX_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MAX_QP}, + + {I_FRAME_MAX_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MAX_QP}, + + {P_FRAME_MAX_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MAX_QP}, + + {P_FRAME_MAX_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MAX_QP}, + + {B_FRAME_MAX_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MAX_QP}, + + {B_FRAME_MAX_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MAX_QP}, + + {I_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {I_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {P_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {P_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {B_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {B_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {LAYER_TYPE, ENC, HEVC, + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B, + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B) | + BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P), + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE, + HFI_PROP_LAYER_ENCODING_TYPE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {LTR_COUNT}}, + + {LAYER_TYPE, ENC, H264, + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B, + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) | + BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P), + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE, + HFI_PROP_LAYER_ENCODING_TYPE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {LTR_COUNT}}, + + {LAYER_ENABLE, ENC, H264, + 0, 1, 1, 0, + V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING, + HFI_PROP_LAYER_ENCODING_TYPE, + CAP_FLAG_OUTPUT_PORT}, + + {LAYER_ENABLE, ENC, HEVC, + 0, 1, 1, 0, + 0, + 0, + CAP_FLAG_OUTPUT_PORT}, + + {ENH_LAYER_COUNT, ENC, HEVC, + 0, 5, 1, 0, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER, + HFI_PROP_LAYER_COUNT, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE, + LTR_COUNT}, + adjust_layer_count, + set_layer_count_and_type}, + + {ENH_LAYER_COUNT, ENC, H264, + 0, 5, 1, 0, + V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER, + HFI_PROP_LAYER_COUNT, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE, + LTR_COUNT}, + adjust_layer_count, + set_layer_count_and_type}, + + {L0_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR, + HFI_PROP_BITRATE_LAYER1, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L1_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L0_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR, + HFI_PROP_BITRATE_LAYER1, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L1_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L1_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR, + HFI_PROP_BITRATE_LAYER2, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L2_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L1_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR, + HFI_PROP_BITRATE_LAYER2, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L2_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L2_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR, + HFI_PROP_BITRATE_LAYER3, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L3_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L2_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR, + HFI_PROP_BITRATE_LAYER3, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L3_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L3_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR, + HFI_PROP_BITRATE_LAYER4, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L4_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L3_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR, + HFI_PROP_BITRATE_LAYER4, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L4_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L4_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR, + HFI_PROP_BITRATE_LAYER5, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L5_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L4_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR, + HFI_PROP_BITRATE_LAYER5, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L5_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L5_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR, + HFI_PROP_BITRATE_LAYER6, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L5_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR, + HFI_PROP_BITRATE_LAYER6, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {ENTROPY_MODE, ENC, H264, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC, + BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC) | + BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC), + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC, + V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE, + HFI_PROP_CABAC_SESSION, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {BIT_RATE}, + adjust_entropy_mode, + set_u32}, + + {SLICE_MODE, ENC, H264 | HEVC, + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE, + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES, + BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) | + BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) | + BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES), + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE, + V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE, + 0, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {STAGE}, + adjust_slice_count, + set_slice_count}, + + {SLICE_MAX_BYTES, ENC, H264 | HEVC, + MIN_SLICE_BYTE_SIZE, MAX_SLICE_BYTE_SIZE, + 1, MIN_SLICE_BYTE_SIZE, + V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES, + HFI_PROP_MULTI_SLICE_BYTES_COUNT, + CAP_FLAG_OUTPUT_PORT}, + + {SLICE_MAX_MB, ENC, H264 | HEVC, + 1, MAX_SLICE_MB_SIZE, 1, 1, + V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB, + HFI_PROP_MULTI_SLICE_MB_COUNT, + CAP_FLAG_OUTPUT_PORT}, + + {TRANSFORM_8X8, ENC, H264, + 0, 1, 1, 1, + V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM, + HFI_PROP_8X8_TRANSFORM, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_transform_8x8, + set_u32}, + + {DISPLAY_DELAY_ENABLE, DEC, CODECS_ALL, 0, 1, 1, 0, V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE, HFI_PROP_DECODE_ORDER_OUTPUT, @@ -264,7 +952,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, NULL}, - {DISPLAY_DELAY, CODECS_ALL, + {DISPLAY_DELAY, DEC, CODECS_ALL, 0, 1, 1, 0, V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY, HFI_PROP_DECODE_ORDER_OUTPUT, @@ -273,7 +961,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, NULL}, - {OUTPUT_ORDER, CODECS_ALL, + {OUTPUT_ORDER, DEC, CODECS_ALL, 0, 1, 1, 0, 0, HFI_PROP_DECODE_ORDER_OUTPUT, @@ -282,7 +970,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { adjust_output_order, set_u32}, - {INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL, + {INPUT_BUF_HOST_MAX_COUNT, ENC | DEC, CODECS_ALL, DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT, 1, DEFAULT_MAX_HOST_BUF_COUNT, 0, @@ -292,7 +980,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32}, - {STAGE, CODECS_ALL, + {STAGE, ENC | DEC, CODECS_ALL, STAGE_1, STAGE_2, 1, STAGE_2, @@ -303,7 +991,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_stage}, - {PIPE, CODECS_ALL, + {PIPE, ENC | DEC, CODECS_ALL, PIPE_1, PIPE_4, 1, PIPE_4, @@ -314,26 +1002,26 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_pipe}, - {POC, H264, 0, 2, 1, 1, + {POC, DEC, H264, 0, 2, 1, 1, 0, HFI_PROP_PIC_ORDER_CNT_TYPE}, - {CODED_FRAMES, H264 | HEVC, + {CODED_FRAMES, DEC, H264 | HEVC, CODED_FRAMES_PROGRESSIVE, CODED_FRAMES_PROGRESSIVE, 0, CODED_FRAMES_PROGRESSIVE, 0, HFI_PROP_CODED_FRAMES}, - {BIT_DEPTH, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8, + {BIT_DEPTH, DEC, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8, 0, HFI_PROP_LUMA_CHROMA_BIT_DEPTH}, - {DEFAULT_HEADER, CODECS_ALL, + {DEFAULT_HEADER, DEC, CODECS_ALL, 0, 1, 1, 0, 0, HFI_PROP_DEC_DEFAULT_HEADER}, - {RAP_FRAME, CODECS_ALL, + {RAP_FRAME, DEC, CODECS_ALL, 0, 1, 1, 1, 0, HFI_PROP_DEC_START_FROM_RAP_FRAME, diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c index 5a02f24..58498af 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c @@ -42,6 +42,9 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size) if (inst->codec == VP9) { vsp_cycles = div_u64(vsp_cycles * 170, 100); + } else if (inst->cap[ENTROPY_MODE].value == + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) { + vsp_cycles = div_u64(vsp_cycles * 135, 100); } else { base_cycles = 0; vsp_cycles = div_u64(vsp_cycles, 2); From patchwork Mon Dec 18 11:32:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755542 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD096381B2; Mon, 18 Dec 2023 11:40:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="m9IQQP6A" Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAslf2028697; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=N8s5yYXwEHYgj9TDxp9LJSATexqdj9j0R5d1BKVwOlA=; b=m9 IQQP6A6smD7brqB/UO4AswFn7kCbqLzWTEuMvD95J0FVOlt4zkAqwshSbs5YfkHR Lq6exVdweHFAVEVsNEIvurkiuxOzeGfgfwA+5K35jmUYQnDz8Q36oDH9rfHotujt /mMq7h2UbSo4HF4bBxt6uTx0uzUuZvbSiScwAFwDYP+Ky8yMu8e+Sx/jValqFVFX tPa9gJAqMm5WDPnQFM68OdDM+2shC+CJjVyPJ26bd5R2A69yPjN1NMxw8zSt5h+A 2Pq7KSmeyZ+tDLrAQWX5D/pLQd6P3HCiy3DFddH9SZWC2Pe7oSCGY5t0Tbk5Yb0D uMV+WgnuOnmn8wunhPog== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2ghdgnc2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0x0004854; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ag-4; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX8ra030072; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8qi030067; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 6E9B2238D; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 32/34] media: iris: implement iris v4l2 ioctl ops supported by encoder Date: Mon, 18 Dec 2023 17:02:27 +0530 Message-Id: <1702899149-21321-33-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: xg1w6oxGrxLf3k8OAReWWYDLMPGW0hk5 X-Proofpoint-ORIG-GUID: xg1w6oxGrxLf3k8OAReWWYDLMPGW0hk5 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 impostorscore=0 phishscore=0 clxscore=1015 adultscore=0 suspectscore=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement all iris v4l2 ioctls ops supported by encoder. Add state checks to ensure ioctl are allowed in valid instance state only. Codec format can be changed by client during s_fmt. Update the v4l2 control values according to the updated codec format. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 1 + .../media/platform/qcom/vcodec/iris/iris_core.h | 8 +- .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 59 ++- .../media/platform/qcom/vcodec/iris/iris_helpers.c | 89 +++- .../media/platform/qcom/vcodec/iris/iris_helpers.h | 3 + .../platform/qcom/vcodec/iris/iris_instance.h | 5 + .../media/platform/qcom/vcodec/iris/iris_probe.c | 3 +- .../media/platform/qcom/vcodec/iris/iris_state.c | 5 +- .../media/platform/qcom/vcodec/iris/iris_vdec.c | 80 +--- .../media/platform/qcom/vcodec/iris/iris_venc.c | 525 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_venc.h | 24 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 415 ++++++++++++++-- .../platform/qcom/vcodec/iris/platform_common.h | 2 + .../platform/qcom/vcodec/iris/platform_sm8550.c | 1 + 14 files changed, 1088 insertions(+), 132 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 4c8f8f6..b95d627 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -6,6 +6,7 @@ iris-objs += iris_probe.o \ iris_vidc.o \ iris_vb2.o \ iris_vdec.o \ + iris_venc.o \ iris_state.o \ iris_ctrls.o \ iris_helpers.o \ diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index c56eb24..baced21 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -25,7 +25,8 @@ * @vdev_dec: iris video device structure for decoder * @vdev_enc: iris video device structure for encoder * @v4l2_file_ops: iris v4l2 file ops - * @v4l2_ioctl_ops: iris v4l2 ioctl ops + * @v4l2_ioctl_ops_dec: iris v4l2 ioctl ops for decoder + * @v4l2_ioctl_ops_enc: iris v4l2 ioctl ops for encoder * @bus_tbl: table of iris buses * @bus_count: count of iris buses * @power_domain_tbl: table of iris power domains @@ -52,6 +53,7 @@ * @packet_id: id of packet * @vpu_ops: a pointer to vpu ops * @session_ops: a pointer to session level ops + * @enc_codecs_count: supported codec count for encoder * @dec_codecs_count: supported codec count for decoder * @platform_data: a structure for platform data * @cap: an array for supported core capabilities @@ -69,7 +71,8 @@ struct iris_core { struct video_device *vdev_dec; struct video_device *vdev_enc; const struct v4l2_file_operations *v4l2_file_ops; - const struct v4l2_ioctl_ops *v4l2_ioctl_ops; + const struct v4l2_ioctl_ops *v4l2_ioctl_ops_dec; + const struct v4l2_ioctl_ops *v4l2_ioctl_ops_enc; struct bus_info *bus_tbl; u32 bus_count; struct power_domain_info *power_domain_tbl; @@ -97,6 +100,7 @@ struct iris_core { const struct vpu_ops *vpu_ops; const struct vpu_session_ops *session_ops; u32 dec_codecs_count; + u32 enc_codecs_count; struct platform_data *platform_data; struct plat_core_cap cap[CORE_CAP_MAX + 1]; struct plat_inst_caps *inst_caps; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c index a648cc1..af99ac73 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -217,7 +217,7 @@ static int set_dynamic_property(struct iris_inst *inst, return ret; } -static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl) +static int iris_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl) { enum plat_inst_cap_type cap_id; struct iris_inst *inst = NULL; @@ -242,7 +242,7 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl) return ret; } -static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) +static int iris_op_s_ctrl(struct v4l2_ctrl *ctrl) { struct cap_entry *entry = NULL, *temp = NULL; struct list_head children_list, firmware_list; @@ -273,7 +273,8 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) cap[cap_id].flags |= CAP_FLAG_CLIENT_SET; - if (!inst->vb2q_src->streaming) { + if ((inst->domain == ENCODER && !inst->vb2q_dst->streaming) || + (inst->domain == DECODER && !inst->vb2q_src->streaming)) { inst->cap[cap_id].value = ctrl->val; } else { ret = adjust_dynamic_property(inst, cap_id, ctrl, @@ -300,16 +301,16 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) } static const struct v4l2_ctrl_ops ctrl_ops = { - .s_ctrl = vdec_op_s_ctrl, - .g_volatile_ctrl = vdec_op_g_volatile_ctrl, + .s_ctrl = iris_op_s_ctrl, + .g_volatile_ctrl = iris_op_g_volatile_ctrl, }; int ctrls_init(struct iris_inst *inst, bool init) { int num_ctrls = 0, ctrl_idx = 0; + u64 codecs_count, step_or_mask; struct plat_inst_cap *cap; struct iris_core *core; - u64 step_or_mask; int idx = 0; int ret = 0; @@ -324,8 +325,11 @@ int ctrls_init(struct iris_inst *inst, bool init) return -EINVAL; if (init) { + codecs_count = inst->domain == ENCODER ? + core->enc_codecs_count : + core->dec_codecs_count; ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, - INST_CAP_MAX * core->dec_codecs_count); + INST_CAP_MAX * codecs_count); if (ret) return ret; } @@ -451,29 +455,49 @@ static int update_inst_capability(struct plat_inst_cap *in, int iris_init_instance_caps(struct iris_core *core) { struct plat_inst_cap *inst_plat_cap_data = NULL; - u8 dec_codecs_count = 0; - int num_inst_cap; - u32 dec_valid_codecs; + u8 enc_codecs_count = 0, dec_codecs_count = 0; + u32 enc_valid_codecs, dec_valid_codecs; int i, j, check_bit = 0; + u8 codecs_count = 0; + int num_inst_cap; int ret = 0; inst_plat_cap_data = core->platform_data->inst_cap_data; if (!inst_plat_cap_data) return -EINVAL; + enc_valid_codecs = core->cap[ENC_CODECS].value; + enc_codecs_count = hweight32(enc_valid_codecs); + core->enc_codecs_count = enc_codecs_count; + dec_valid_codecs = core->cap[DEC_CODECS].value; dec_codecs_count = hweight32(dec_valid_codecs); core->dec_codecs_count = dec_codecs_count; + codecs_count = enc_codecs_count + dec_codecs_count; core->inst_caps = devm_kzalloc(core->dev, - dec_codecs_count * sizeof(struct plat_inst_caps), + codecs_count * sizeof(struct plat_inst_caps), GFP_KERNEL); if (!core->inst_caps) return -ENOMEM; - for (i = 0; i < dec_codecs_count; i++) { + for (i = 0; i < enc_codecs_count; i++) { + while (check_bit < (sizeof(enc_valid_codecs) * 8)) { + if (enc_valid_codecs & BIT(check_bit)) { + core->inst_caps[i].domain = ENCODER; + core->inst_caps[i].codec = enc_valid_codecs & + BIT(check_bit); + check_bit++; + break; + } + check_bit++; + } + } + + for (; i < codecs_count; i++) { while (check_bit < (sizeof(dec_valid_codecs) * 8)) { if (dec_valid_codecs & BIT(check_bit)) { + core->inst_caps[i].domain = DECODER; core->inst_caps[i].codec = dec_valid_codecs & BIT(check_bit); check_bit++; @@ -486,9 +510,9 @@ int iris_init_instance_caps(struct iris_core *core) num_inst_cap = core->platform_data->inst_cap_data_size; for (i = 0; i < num_inst_cap; i++) { - for (j = 0; j < dec_codecs_count; j++) { - if ((inst_plat_cap_data[i].codec & - core->inst_caps[j].codec)) { + for (j = 0; j < codecs_count; j++) { + if ((inst_plat_cap_data[i].domain & core->inst_caps[j].domain) && + (inst_plat_cap_data[i].codec & core->inst_caps[j].codec)) { ret = update_inst_capability(&inst_plat_cap_data[i], &core->inst_caps[j]); if (ret) @@ -503,11 +527,14 @@ int iris_init_instance_caps(struct iris_core *core) int get_inst_capability(struct iris_inst *inst) { struct iris_core *core; + u32 codecs_count = 0; int i; core = inst->core; - for (i = 0; i < core->dec_codecs_count; i++) { + codecs_count = core->enc_codecs_count + core->dec_codecs_count; + + for (i = 0; i < codecs_count; i++) { if (core->inst_caps[i].codec == inst->codec) { memcpy(&inst->cap[0], &core->inst_caps[i].cap[0], (INST_CAP_MAX + 1) * sizeof(struct plat_inst_cap)); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index d9d22e2..c84bb51 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -7,6 +7,7 @@ #include "hfi_defines.h" #include "iris_core.h" +#include "iris_ctrls.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" @@ -273,7 +274,7 @@ int close_session(struct iris_inst *inst) return ret; } -static int check_core_mbps_mbpf(struct iris_inst *inst) +int check_core_mbps_mbpf(struct iris_inst *inst) { u32 mbpf = 0, mbps = 0, total_mbpf = 0, total_mbps = 0; struct iris_core *core; @@ -284,7 +285,9 @@ static int check_core_mbps_mbpf(struct iris_inst *inst) mutex_lock(&core->lock); list_for_each_entry(instance, &core->instances, list) { - fps = inst->cap[QUEUED_RATE].value >> 16; + fps = max3(inst->cap[QUEUED_RATE].value >> 16, + inst->cap[FRAME_RATE].value >> 16, + inst->cap[OPERATING_RATE].value >> 16); mbpf = get_mbpf(inst); mbps = mbpf * fps; total_mbpf += mbpf; @@ -814,6 +817,88 @@ int session_streamoff(struct iris_inst *inst, u32 plane) return ret; } +int process_resume(struct iris_inst *inst) +{ + enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; + int ret; + + if (inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST) { + clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { + ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + } + } else if (inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) { + clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST; + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { + ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + } + } + + ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); + + return ret; +} + +int codec_change(struct iris_inst *inst, u32 v4l2_codec) +{ + bool session_init = false; + int ret; + + if (!inst->codec) + session_init = true; + + if (inst->codec && + ((inst->domain == DECODER && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec) || + (inst->domain == ENCODER && inst->fmt_dst->fmt.pix_mp.pixelformat == v4l2_codec))) + return 0; + + inst->codec = v4l2_codec_to_driver(inst, v4l2_codec); + if (!inst->codec) + return -EINVAL; + + if (inst->domain == DECODER) + inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec; + else if (inst->domain == ENCODER) + inst->fmt_dst->fmt.pix_mp.pixelformat = v4l2_codec; + + ret = get_inst_capability(inst); + if (ret) + return ret; + + ret = ctrls_init(inst, session_init); + if (ret) + return ret; + + ret = update_buffer_count(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = update_buffer_count(inst, OUTPUT_MPLANE); + + return ret; +} + int iris_pm_get(struct iris_core *core) { int ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index c628b2e..39cec8c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -41,6 +41,7 @@ enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec); u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat); enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat); struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type); +int check_core_mbps_mbpf(struct iris_inst *inst); int check_session_supported(struct iris_inst *inst); struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index); @@ -51,6 +52,8 @@ int iris_vb2_buffer_done(struct iris_inst *inst, int iris_release_nonref_buffers(struct iris_inst *inst); void iris_destroy_buffers(struct iris_inst *inst); int session_streamoff(struct iris_inst *inst, u32 plane); +int process_resume(struct iris_inst *inst); +int codec_change(struct iris_inst *inst, u32 v4l2_codec); int iris_pm_get(struct iris_core *core); int iris_pm_put(struct iris_core *core, bool autosuspend); int iris_pm_get_put(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 70f4c7d..a1547c5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -8,6 +8,7 @@ #include +#include "hfi_defines.h" #include "iris_buffer.h" #include "iris_common.h" #include "iris_core.h" @@ -29,6 +30,7 @@ * @fmt_dst: structure of v4l2_format for destination * @ctrl_handler: reference of v4l2 ctrl handler * @crop: structure of crop info + * @compose: structure of compose info * @packet: HFI packet * @packet_size: HFI packet size * @completions: structure of signal completions @@ -36,6 +38,7 @@ * @num_ctrls: supported number of controls * @caps_list: list head of capability * @codec: codec type + * @domain: domain type: encoder or decoder * @buffers: structure of buffer info * @fw_min_count: minimnum count of buffers needed by fw * @state: instance state @@ -70,6 +73,7 @@ struct iris_inst { struct v4l2_format *fmt_dst; struct v4l2_ctrl_handler ctrl_handler; struct rect_desc crop; + struct rect_desc compose; void *packet; u32 packet_size; struct completion completions[MAX_SIGNAL]; @@ -77,6 +81,7 @@ struct iris_inst { u32 num_ctrls; struct list_head caps_list; enum codec_type codec; + enum domain_type domain; struct iris_buffers_info buffers; u32 fw_min_count; enum iris_inst_state state; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index b487e83..49d2701 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -42,13 +42,13 @@ static int iris_register_video_device(struct iris_core *core, enum domain_type t vdev->release = video_device_release; vdev->fops = core->v4l2_file_ops; - vdev->ioctl_ops = core->v4l2_ioctl_ops; vdev->vfl_dir = VFL_DIR_M2M; vdev->v4l2_dev = &core->v4l2_dev; vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; if (type == DECODER) { strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); + vdev->ioctl_ops = core->v4l2_ioctl_ops_dec; ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); if (ret) @@ -57,6 +57,7 @@ static int iris_register_video_device(struct iris_core *core, enum domain_type t core->vdev_dec = vdev; } else if (type == ENCODER) { strscpy(vdev->name, "qcom-iris-encoder", sizeof(vdev->name)); + vdev->ioctl_ops = core->v4l2_ioctl_ops_enc; ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); if (ret) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 952ba2a..e853aec 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -198,8 +198,9 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id) { return ((inst->state == IRIS_INST_OPEN) || ((inst->cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED) && - (inst->state == IRIS_INST_INPUT_STREAMING || - inst->state == IRIS_INST_STREAMING))); + ((inst->state == IRIS_INST_INPUT_STREAMING && inst->domain == DECODER) || + (inst->state == IRIS_INST_OUTPUT_STREAMING && inst->domain == ENCODER) || + (inst->state == IRIS_INST_STREAMING)))); } int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 371615e..300d0e9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -22,39 +22,6 @@ struct vdec_prop_type_handle { int (*handle)(struct iris_inst *inst); }; -static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec) -{ - bool session_init = false; - int ret; - - if (!inst->codec) - session_init = true; - - if (inst->codec && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec) - return 0; - - inst->codec = v4l2_codec_to_driver(inst, v4l2_codec); - if (!inst->codec) - return -EINVAL; - - inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec; - ret = get_inst_capability(inst); - if (ret) - return ret; - - ret = ctrls_init(inst, session_init); - if (ret) - return ret; - - ret = update_buffer_count(inst, INPUT_MPLANE); - if (ret) - return ret; - - ret = update_buffer_count(inst, OUTPUT_MPLANE); - - return ret; -} - int vdec_inst_init(struct iris_inst *inst) { struct v4l2_format *f; @@ -93,7 +60,7 @@ int vdec_inst_init(struct iris_inst *inst) inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage; inst->fw_min_count = 0; - ret = vdec_codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat); + ret = codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat); return ret; } @@ -233,7 +200,7 @@ int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f) if (f->type == INPUT_MPLANE) { if (inst->fmt_src->fmt.pix_mp.pixelformat != f->fmt.pix_mp.pixelformat) { - ret = vdec_codec_change(inst, f->fmt.pix_mp.pixelformat); + ret = codec_change(inst, f->fmt.pix_mp.pixelformat); if (ret) return ret; } @@ -1304,49 +1271,6 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) return ret; } -static int process_resume(struct iris_inst *inst) -{ - enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; - int ret; - - if (inst->sub_state & IRIS_INST_SUB_DRC && - inst->sub_state & IRIS_INST_SUB_DRC_LAST) { - clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; - - if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { - ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); - if (ret) - return ret; - clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; - } - if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { - ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); - if (ret) - return ret; - clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; - } - } else if (inst->sub_state & IRIS_INST_SUB_DRAIN && - inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) { - clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST; - if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { - ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN); - if (ret) - return ret; - clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; - } - if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { - ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN); - if (ret) - return ret; - clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; - } - } - - ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); - - return ret; -} - int vdec_start_cmd(struct iris_inst *inst) { int ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c new file mode 100644 index 0000000..802db40 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c @@ -0,0 +1,525 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "hfi_defines.h" +#include "iris_buffer.h" +#include "iris_common.h" +#include "iris_ctrls.h" +#include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_hfi_packet.h" +#include "iris_power.h" +#include "iris_venc.h" + +int venc_inst_init(struct iris_inst *inst) +{ + struct v4l2_format *f; + int ret; + + inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL); + if (!inst->fmt_src) + return -ENOMEM; + + inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL); + if (!inst->fmt_dst) + return -ENOMEM; + + inst->vb2q_src = kzalloc(sizeof(*inst->vb2q_src), GFP_KERNEL); + if (!inst->vb2q_src) + return -ENOMEM; + + inst->vb2q_dst = kzalloc(sizeof(*inst->vb2q_dst), GFP_KERNEL); + if (!inst->vb2q_dst) + return -ENOMEM; + + f = inst->fmt_dst; + f->type = OUTPUT_MPLANE; + f->fmt.pix_mp.width = DEFAULT_WIDTH; + f->fmt.pix_mp.height = DEFAULT_HEIGHT; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264; + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = 0; + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT); + inst->buffers.output.actual_count = inst->buffers.output.min_count; + inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage; + + inst->crop.left = 0; + inst->crop.top = 0; + inst->crop.width = f->fmt.pix_mp.width; + inst->crop.height = f->fmt.pix_mp.height; + + inst->compose.left = 0; + inst->compose.top = 0; + inst->compose.width = f->fmt.pix_mp.width; + inst->compose.height = f->fmt.pix_mp.height; + + f = inst->fmt_src; + f->type = INPUT_MPLANE; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_QC08C; + f->fmt.pix_mp.width = DEFAULT_WIDTH; + f->fmt.pix_mp.height = DEFAULT_HEIGHT; + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128); + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT); + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT); + inst->buffers.input.actual_count = inst->buffers.input.min_count; + inst->buffers.input.size = f->fmt.pix_mp.plane_fmt[0].sizeimage; + + inst->hfi_rc_type = HFI_RC_VBR_CFR; + inst->hfi_layer_type = HFI_HIER_P_SLIDING_WINDOW; + + ret = codec_change(inst, inst->fmt_dst->fmt.pix_mp.pixelformat); + + return ret; +} + +void venc_inst_deinit(struct iris_inst *inst) +{ + kfree(inst->fmt_dst); + kfree(inst->fmt_src); +} + +int venc_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f) +{ + struct iris_core *core; + u32 array[32] = {0}; + u32 i = 0; + + core = inst->core; + + if (f->type == OUTPUT_MPLANE) { + u32 codecs = core->cap[ENC_CODECS].value; + u32 idx = 0; + + for (i = 0; i <= 31; i++) { + if (codecs & BIT(i)) { + if (idx >= ARRAY_SIZE(array)) + break; + array[idx] = codecs & BIT(i); + idx++; + } + } + if (!array[f->index]) + return -EINVAL; + f->pixelformat = v4l2_codec_from_driver(inst, array[f->index]); + if (!f->pixelformat) + return -EINVAL; + f->flags = V4L2_FMT_FLAG_COMPRESSED; + strscpy(f->description, "codec", sizeof(f->description)); + } else if (f->type == INPUT_MPLANE) { + u32 formats = inst->cap[PIX_FMTS].step_or_mask; + u32 idx = 0; + + for (i = 0; i <= 31; i++) { + if (formats & BIT(i)) { + if (idx >= ARRAY_SIZE(array)) + break; + array[idx] = formats & BIT(i); + idx++; + } + } + if (!array[f->index]) + return -EINVAL; + f->pixelformat = v4l2_colorformat_from_driver(inst, array[f->index]); + if (!f->pixelformat) + return -EINVAL; + strscpy(f->description, "colorformat", sizeof(f->description)); + } + + memset(f->reserved, 0, sizeof(f->reserved)); + + return 0; +} + +int venc_try_fmt(struct iris_inst *inst, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; + u32 pix_fmt; + + memset(pixmp->reserved, 0, sizeof(pixmp->reserved)); + + if (f->type == INPUT_MPLANE) { + pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat); + if (!pix_fmt) { + f->fmt.pix_mp.pixelformat = inst->fmt_src->fmt.pix_mp.pixelformat; + f->fmt.pix_mp.width = inst->fmt_src->fmt.pix_mp.width; + f->fmt.pix_mp.height = inst->fmt_src->fmt.pix_mp.height; + } + } else if (f->type == OUTPUT_MPLANE) { + pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat); + if (!pix_fmt) { + f->fmt.pix_mp.width = inst->fmt_dst->fmt.pix_mp.width; + f->fmt.pix_mp.height = inst->fmt_dst->fmt.pix_mp.height; + f->fmt.pix_mp.pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + } + } else { + return -EINVAL; + } + + if (pixmp->field == V4L2_FIELD_ANY) + pixmp->field = V4L2_FIELD_NONE; + pixmp->num_planes = 1; + + return 0; +} + +static int venc_s_fmt_output(struct iris_inst *inst, struct v4l2_format *f) +{ + struct v4l2_format *fmt; + enum codec_type codec; + u32 codec_align; + u32 width, height; + int ret = 0; + + venc_try_fmt(inst, f); + + fmt = inst->fmt_dst; + if (fmt->fmt.pix_mp.pixelformat != f->fmt.pix_mp.pixelformat) { + ret = codec_change(inst, f->fmt.pix_mp.pixelformat); + if (ret) + return ret; + } + fmt->type = OUTPUT_MPLANE; + + codec = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat); + + codec_align = (codec == HEVC) ? 32 : 16; + width = inst->compose.width; + height = inst->compose.height; + if (inst->cap[ROTATION].value == 90 || inst->cap[ROTATION].value == 270) { + width = inst->compose.height; + height = inst->compose.width; + } + fmt->fmt.pix_mp.width = ALIGN(width, codec_align); + fmt->fmt.pix_mp.height = ALIGN(height, codec_align); + fmt->fmt.pix_mp.num_planes = 1; + fmt->fmt.pix_mp.plane_fmt[0].bytesperline = 0; + fmt->fmt.pix_mp.plane_fmt[0].sizeimage = + iris_get_buffer_size(inst, BUF_OUTPUT); + if (f->fmt.pix_mp.colorspace != V4L2_COLORSPACE_DEFAULT && + f->fmt.pix_mp.colorspace != V4L2_COLORSPACE_REC709) + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; + fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace; + fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func; + fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT); + if (inst->buffers.output.actual_count < + inst->buffers.output.min_count) { + inst->buffers.output.actual_count = + inst->buffers.output.min_count; + } + inst->buffers.output.size = + fmt->fmt.pix_mp.plane_fmt[0].sizeimage; + + memcpy(f, fmt, sizeof(struct v4l2_format)); + + return ret; +} + +static int venc_s_fmt_input(struct iris_inst *inst, struct v4l2_format *f) +{ + u32 pix_fmt, width, height, size, bytesperline; + struct v4l2_format *fmt, *output_fmt; + int ret = 0; + + venc_try_fmt(inst, f); + + pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat); + inst->cap[PIX_FMTS].value = pix_fmt; + + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + bytesperline = pix_fmt == FMT_TP10C ? + ALIGN(ALIGN(f->fmt.pix_mp.width, 192) * 4 / 3, 256) : + ALIGN(f->fmt.pix_mp.width, 128); + + fmt = inst->fmt_src; + fmt->type = INPUT_MPLANE; + fmt->fmt.pix_mp.width = width; + fmt->fmt.pix_mp.height = height; + fmt->fmt.pix_mp.num_planes = 1; + fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat; + fmt->fmt.pix_mp.plane_fmt[0].bytesperline = bytesperline; + size = iris_get_buffer_size(inst, BUF_INPUT); + fmt->fmt.pix_mp.plane_fmt[0].sizeimage = size; + fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace; + fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func; + fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + + output_fmt = inst->fmt_dst; + output_fmt->fmt.pix_mp.colorspace = fmt->fmt.pix_mp.colorspace; + output_fmt->fmt.pix_mp.xfer_func = fmt->fmt.pix_mp.xfer_func; + output_fmt->fmt.pix_mp.ycbcr_enc = fmt->fmt.pix_mp.ycbcr_enc; + output_fmt->fmt.pix_mp.quantization = fmt->fmt.pix_mp.quantization; + + inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT); + if (inst->buffers.input.actual_count < + inst->buffers.input.min_count) { + inst->buffers.input.actual_count = + inst->buffers.input.min_count; + } + inst->buffers.input.size = size; + + if (f->fmt.pix_mp.width != inst->crop.width || + f->fmt.pix_mp.height != inst->crop.height) { + inst->crop.top = 0; + inst->crop.left = 0; + inst->crop.width = f->fmt.pix_mp.width; + inst->crop.height = f->fmt.pix_mp.height; + + inst->compose.top = 0; + inst->compose.left = 0; + inst->compose.width = f->fmt.pix_mp.width; + inst->compose.height = f->fmt.pix_mp.height; + + ret = venc_s_fmt_output(inst, output_fmt); + if (ret) + return ret; + } + + memcpy(f, fmt, sizeof(struct v4l2_format)); + + return ret; +} + +int venc_s_fmt(struct iris_inst *inst, struct v4l2_format *f) +{ + int ret; + + if (f->type == INPUT_MPLANE) + ret = venc_s_fmt_input(inst, f); + else if (f->type == OUTPUT_MPLANE) + ret = venc_s_fmt_output(inst, f); + else + ret = -EINVAL; + + return ret; +} + +int venc_s_selection(struct iris_inst *inst, struct v4l2_selection *s) +{ + struct v4l2_format *output_fmt; + int ret; + + if (s->type != INPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT) + return -EINVAL; + + switch (s->target) { + case V4L2_SEL_TGT_CROP: + if (s->r.left || s->r.top) { + s->r.left = 0; + s->r.top = 0; + } + if (s->r.width > inst->fmt_src->fmt.pix_mp.width) + s->r.width = inst->fmt_src->fmt.pix_mp.width; + + if (s->r.height > inst->fmt_src->fmt.pix_mp.height) + s->r.height = inst->fmt_src->fmt.pix_mp.height; + + inst->crop.left = s->r.left; + inst->crop.top = s->r.top; + inst->crop.width = s->r.width; + inst->crop.height = s->r.height; + inst->compose.left = inst->crop.left; + inst->compose.top = inst->crop.top; + inst->compose.width = inst->crop.width; + inst->compose.height = inst->crop.height; + output_fmt = inst->fmt_dst; + ret = venc_s_fmt_output(inst, output_fmt); + if (ret) + return ret; + break; + case V4L2_SEL_TGT_COMPOSE: + if (s->r.left < inst->crop.left) + s->r.left = inst->crop.left; + + if (s->r.top < inst->crop.top) + s->r.top = inst->crop.top; + + if (s->r.width > inst->crop.width) + s->r.width = inst->crop.width; + + if (s->r.height > inst->crop.height) + s->r.height = inst->crop.height; + inst->compose.left = s->r.left; + inst->compose.top = s->r.top; + inst->compose.width = s->r.width; + inst->compose.height = s->r.height; + + output_fmt = inst->fmt_dst; + ret = venc_s_fmt_output(inst, output_fmt); + if (ret) + return ret; + break; + default: + ret = -EINVAL; + break; + } + + return ret; +} + +int venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm) +{ + struct v4l2_fract *timeperframe = NULL; + u32 q16_rate, max_rate, default_rate; + u64 us_per_frame = 0, input_rate = 0; + bool is_frame_rate = false; + int ret = 0; + + if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + timeperframe = &s_parm->parm.output.timeperframe; + max_rate = inst->cap[OPERATING_RATE].max >> 16; + default_rate = inst->cap[OPERATING_RATE].value >> 16; + s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME; + } else { + timeperframe = &s_parm->parm.capture.timeperframe; + is_frame_rate = true; + max_rate = inst->cap[FRAME_RATE].max >> 16; + default_rate = inst->cap[FRAME_RATE].value >> 16; + s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME; + } + + if (!timeperframe->denominator || !timeperframe->numerator) { + if (!timeperframe->numerator) + timeperframe->numerator = 1; + if (!timeperframe->denominator) + timeperframe->denominator = default_rate; + } + + us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC; + us_per_frame = div64_u64(us_per_frame, timeperframe->denominator); + + if (!us_per_frame) { + ret = -EINVAL; + goto exit; + } + + input_rate = (u64)USEC_PER_SEC; + input_rate = div64_u64(input_rate, us_per_frame); + + q16_rate = (u32)input_rate << 16; + if (is_frame_rate) + inst->cap[FRAME_RATE].value = q16_rate; + else + inst->cap[OPERATING_RATE].value = q16_rate; + + if ((s_parm->type == INPUT_MPLANE && inst->vb2q_src->streaming) || + (s_parm->type == OUTPUT_MPLANE && inst->vb2q_dst->streaming)) { + ret = check_core_mbps_mbpf(inst); + if (ret) + goto reset_rate; + ret = input_rate > max_rate; + if (ret) { + ret = -ENOMEM; + goto reset_rate; + } + } + + if (is_frame_rate) + inst->cap[FRAME_RATE].flags |= CAP_FLAG_CLIENT_SET; + else + inst->cap[OPERATING_RATE].flags |= CAP_FLAG_CLIENT_SET; + + if (inst->vb2q_dst->streaming) { + ret = iris_hfi_set_property(inst, + HFI_PROP_FRAME_RATE, + HFI_HOST_FLAGS_NONE, + HFI_PORT_BITSTREAM, + HFI_PAYLOAD_Q16, + &q16_rate, + sizeof(u32)); + if (ret) + goto exit; + } + + return ret; + +reset_rate: + if (ret) { + if (is_frame_rate) + inst->cap[FRAME_RATE].value = default_rate << 16; + else + inst->cap[OPERATING_RATE].value = default_rate << 16; + } +exit: + return ret; +} + +int venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm) +{ + struct v4l2_fract *timeperframe = NULL; + + if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + timeperframe = &s_parm->parm.output.timeperframe; + timeperframe->numerator = 1; + timeperframe->denominator = + inst->cap[OPERATING_RATE].value >> 16; + s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME; + } else { + timeperframe = &s_parm->parm.capture.timeperframe; + timeperframe->numerator = 1; + timeperframe->denominator = + inst->cap[FRAME_RATE].value >> 16; + s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME; + } + + return 0; +} + +int venc_subscribe_event(struct iris_inst *inst, + const struct v4l2_event_subscription *sub) +{ + int ret; + + switch (sub->type) { + case V4L2_EVENT_EOS: + ret = v4l2_event_subscribe(&inst->fh, sub, MAX_EVENTS, NULL); + break; + case V4L2_EVENT_CTRL: + ret = v4l2_ctrl_subscribe_event(&inst->fh, sub); + break; + default: + return -EINVAL; + } + + return ret; +} + +int venc_start_cmd(struct iris_inst *inst) +{ + vb2_clear_last_buffer_dequeued(inst->vb2q_dst); + + return process_resume(inst); +} + +int venc_stop_cmd(struct iris_inst *inst) +{ + int ret; + + ret = iris_hfi_drain(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_DRAIN); + + iris_scale_power(inst); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h new file mode 100644 index 0000000..24da63f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VENC_H_ +#define _IRIS_VENC_H_ + +#include "iris_instance.h" + +int venc_inst_init(struct iris_inst *inst); +void venc_inst_deinit(struct iris_inst *inst); +int venc_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); +int venc_try_fmt(struct iris_inst *inst, struct v4l2_format *f); +int venc_s_fmt(struct iris_inst *inst, struct v4l2_format *f); +int venc_s_selection(struct iris_inst *inst, struct v4l2_selection *s); +int venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm); +int venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm); +int venc_subscribe_event(struct iris_inst *inst, + const struct v4l2_event_subscription *sub); +int venc_start_cmd(struct iris_inst *inst); +int venc_stop_cmd(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 437d6b4..aa19978 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -15,6 +15,7 @@ #include "iris_instance.h" #include "iris_power.h" #include "iris_vdec.h" +#include "iris_venc.h" #include "iris_vidc.h" #include "iris_vb2.h" @@ -30,7 +31,11 @@ static int vidc_v4l2_fh_init(struct iris_inst *inst) if (inst->fh.vdev) return -EINVAL; - v4l2_fh_init(&inst->fh, core->vdev_dec); + if (inst->domain == ENCODER) + v4l2_fh_init(&inst->fh, core->vdev_enc); + else if (inst->domain == DECODER) + v4l2_fh_init(&inst->fh, core->vdev_dec); + inst->fh.ctrl_handler = &inst->ctrl_handler; v4l2_fh_add(&inst->fh); @@ -160,9 +165,20 @@ int vidc_open(struct file *filp) { struct iris_core *core = video_drvdata(filp); struct iris_inst *inst = NULL; + struct video_device *vdev; + u32 session_type = 0; int i = 0; int ret; + vdev = video_devdata(filp); + if (strcmp(vdev->name, "qcom-iris-decoder") == 0) + session_type = DECODER; + else if (strcmp(vdev->name, "qcom-iris-encoder") == 0) + session_type = ENCODER; + + if (session_type != DECODER && session_type != ENCODER) + return -EINVAL; + ret = iris_pm_get(core); if (ret) return ret; @@ -182,6 +198,7 @@ int vidc_open(struct file *filp) } inst->core = core; + inst->domain = session_type; inst->session_id = hash32_ptr(inst); inst->ipsc_properties_set = false; inst->opsc_properties_set = false; @@ -213,7 +230,12 @@ int vidc_open(struct file *filp) if (ret) goto fail_remove_session; - vdec_inst_init(inst); + if (inst->domain == DECODER) + ret = vdec_inst_init(inst); + else if (inst->domain == ENCODER) + ret = venc_inst_init(inst); + if (ret) + goto fail_fh_deinit; ret = vidc_vb2_queue_init(inst); if (ret) @@ -238,7 +260,11 @@ int vidc_open(struct file *filp) iris_core_deinit(core); vidc_vb2_queue_deinit(inst); fail_inst_deinit: - vdec_inst_deinit(inst); + if (inst->domain == DECODER) + vdec_inst_deinit(inst); + else if (inst->domain == ENCODER) + venc_inst_deinit(inst); +fail_fh_deinit: vidc_v4l2_fh_deinit(inst); fail_remove_session: vidc_remove_session(inst); @@ -264,7 +290,11 @@ int vidc_close(struct file *filp) core = inst->core; v4l2_ctrl_handler_free(&inst->ctrl_handler); - vdec_inst_deinit(inst); + if (inst->domain == DECODER) + vdec_inst_deinit(inst); + else if (inst->domain == ENCODER) + venc_inst_deinit(inst); + mutex_lock(&inst->lock); iris_pm_get(core); close_session(inst); @@ -342,7 +372,7 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt) static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f) { struct iris_inst *inst; - int ret; + int ret = 0; inst = get_vidc_inst(filp, fh); if (!inst) @@ -354,7 +384,10 @@ static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f) goto unlock; } - ret = vdec_enum_fmt(inst, f); + if (inst->domain == DECODER) + ret = vdec_enum_fmt(inst, f); + else if (inst->domain == ENCODER) + ret = venc_enum_fmt(inst, f); unlock: mutex_unlock(&inst->lock); @@ -365,7 +398,7 @@ static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f) static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f) { struct iris_inst *inst; - int ret; + int ret = 0; inst = get_vidc_inst(filp, fh); if (!inst) @@ -382,7 +415,10 @@ static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f) goto unlock; } - ret = vdec_try_fmt(inst, f); + if (inst->domain == DECODER) + ret = vdec_try_fmt(inst, f); + else if (inst->domain == ENCODER) + ret = venc_try_fmt(inst, f); unlock: mutex_unlock(&inst->lock); @@ -393,7 +429,7 @@ static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f) static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f) { struct iris_inst *inst; - int ret; + int ret = 0; inst = get_vidc_inst(filp, fh); if (!inst) @@ -410,7 +446,10 @@ static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f) goto unlock; } - ret = vdec_s_fmt(inst, f); + if (inst->domain == DECODER) + ret = vdec_s_fmt(inst, f); + else if (inst->domain == ENCODER) + ret = venc_s_fmt(inst, f); unlock: mutex_unlock(&inst->lock); @@ -488,6 +527,70 @@ static int vidc_enum_framesizes(struct file *filp, void *fh, return ret; } +static int vidc_enum_frameintervals(struct file *filp, void *fh, + struct v4l2_frmivalenum *fival) + +{ + enum colorformat_type colorfmt; + struct iris_inst *inst; + struct iris_core *core; + u32 fps, mbpf; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !fival) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (inst->domain == DECODER) { + ret = -ENOTTY; + goto unlock; + } + + core = inst->core; + + if (fival->index) { + ret = -EINVAL; + goto unlock; + } + + colorfmt = v4l2_colorformat_to_driver(inst, fival->pixel_format); + if (colorfmt == FMT_NONE) { + ret = -EINVAL; + goto unlock; + } + + if (fival->width > inst->cap[FRAME_WIDTH].max || + fival->width < inst->cap[FRAME_WIDTH].min || + fival->height > inst->cap[FRAME_HEIGHT].max || + fival->height < inst->cap[FRAME_HEIGHT].min) { + ret = -EINVAL; + goto unlock; + } + + mbpf = NUM_MBS_PER_FRAME(fival->height, fival->width); + fps = core->cap[MAX_MBPS].value / mbpf; + + fival->type = V4L2_FRMIVAL_TYPE_STEPWISE; + fival->stepwise.min.numerator = 1; + fival->stepwise.min.denominator = + min_t(u32, fps, inst->cap[FRAME_RATE].max); + fival->stepwise.max.numerator = 1; + fival->stepwise.max.denominator = 1; + fival->stepwise.step.numerator = 1; + fival->stepwise.step.denominator = inst->cap[FRAME_RATE].max; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static int vidc_reqbufs(struct file *filp, void *fh, struct v4l2_requestbuffers *b) { struct vb2_queue *vb2q = NULL; @@ -751,7 +854,11 @@ static int vidc_querycap(struct file *filp, void *fh, struct v4l2_capability *ca strscpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver)); strscpy(cap->bus_info, VIDC_BUS_NAME, sizeof(cap->bus_info)); memset(cap->reserved, 0, sizeof(cap->reserved)); - strscpy(cap->card, "iris_decoder", sizeof(cap->card)); + + if (inst->domain == DECODER) + strscpy(cap->card, "iris_decoder", sizeof(cap->card)); + else if (inst->domain == ENCODER) + strscpy(cap->card, "iris_encoder", sizeof(cap->card)); unlock: mutex_unlock(&inst->lock); @@ -839,7 +946,7 @@ static int vidc_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qm static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub) { struct iris_inst *inst; - int ret; + int ret = 0; inst = container_of(fh, struct iris_inst, fh); @@ -849,7 +956,10 @@ static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subs goto unlock; } - ret = vdec_subscribe_event(inst, sub); + if (inst->domain == DECODER) + ret = vdec_subscribe_event(inst, sub); + else if (inst->domain == ENCODER) + ret = venc_subscribe_event(inst, sub); unlock: mutex_unlock(&inst->lock); @@ -893,28 +1003,152 @@ static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection * goto unlock; } - if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) { + if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE && + inst->domain == DECODER) { ret = -EINVAL; goto unlock; } - switch (s->target) { - case V4L2_SEL_TGT_CROP_BOUNDS: - case V4L2_SEL_TGT_CROP_DEFAULT: - case V4L2_SEL_TGT_CROP: - case V4L2_SEL_TGT_COMPOSE_BOUNDS: - case V4L2_SEL_TGT_COMPOSE_PADDED: - case V4L2_SEL_TGT_COMPOSE_DEFAULT: - case V4L2_SEL_TGT_COMPOSE: - s->r.left = inst->crop.left; - s->r.top = inst->crop.top; - s->r.width = inst->crop.width; - s->r.height = inst->crop.height; - break; - default: + if (s->type != INPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT && + inst->domain == ENCODER) { ret = -EINVAL; + goto unlock; } + if (inst->domain == DECODER) { + switch (s->target) { + case V4L2_SEL_TGT_CROP_BOUNDS: + case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP: + case V4L2_SEL_TGT_COMPOSE_BOUNDS: + case V4L2_SEL_TGT_COMPOSE_PADDED: + case V4L2_SEL_TGT_COMPOSE_DEFAULT: + case V4L2_SEL_TGT_COMPOSE: + s->r.left = inst->crop.left; + s->r.top = inst->crop.top; + s->r.width = inst->crop.width; + s->r.height = inst->crop.height; + break; + default: + ret = -EINVAL; + break; + } + } else if (inst->domain == ENCODER) { + switch (s->target) { + case V4L2_SEL_TGT_CROP_BOUNDS: + case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP: + s->r.left = inst->crop.left; + s->r.top = inst->crop.top; + s->r.width = inst->crop.width; + s->r.height = inst->crop.height; + break; + case V4L2_SEL_TGT_COMPOSE_BOUNDS: + case V4L2_SEL_TGT_COMPOSE_PADDED: + case V4L2_SEL_TGT_COMPOSE_DEFAULT: + case V4L2_SEL_TGT_COMPOSE: + s->r.left = inst->compose.left; + s->r.top = inst->compose.top; + s->r.width = inst->compose.width; + s->r.height = inst->compose.height; + break; + default: + ret = -EINVAL; + break; + } + } + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_s_selection(struct file *filp, void *fh, struct v4l2_selection *s) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !s) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + if (inst->domain == DECODER) { + ret = -EINVAL; + goto unlock; + } else if (inst->domain == ENCODER) { + ret = venc_s_selection(inst, s); + } + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_s_parm(struct file *filp, void *fh, struct v4l2_streamparm *a) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !a) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + ret = -EINVAL; + goto unlock; + } + + if (inst->domain == ENCODER) + ret = venc_s_param(inst, a); + else + ret = -EINVAL; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int vidc_g_parm(struct file *filp, void *fh, struct v4l2_streamparm *a) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !a) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + ret = -EINVAL; + goto unlock; + } + + if (inst->domain == ENCODER) + ret = venc_g_param(inst, a); + else + ret = -EINVAL; + unlock: mutex_unlock(&inst->lock); @@ -955,6 +1189,39 @@ static int vidc_try_dec_cmd(struct file *filp, void *fh, return ret; } +static int vidc_try_enc_cmd(struct file *filp, void *fh, + struct v4l2_encoder_cmd *enc) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !enc) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (inst->domain != ENCODER) { + ret = -ENOTTY; + goto unlock; + } + + if (enc->cmd != V4L2_ENC_CMD_STOP && enc->cmd != V4L2_ENC_CMD_START) { + ret = -EINVAL; + goto unlock; + } + enc->flags = 0; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static int vidc_dec_cmd(struct file *filp, void *fh, struct v4l2_decoder_cmd *dec) { @@ -1002,6 +1269,60 @@ static int vidc_dec_cmd(struct file *filp, void *fh, return ret; } +static int vidc_enc_cmd(struct file *filp, void *fh, + struct v4l2_encoder_cmd *enc) +{ + struct iris_inst *inst; + int ret = 0; + + inst = get_vidc_inst(filp, fh); + if (!inst || !enc) + return -EINVAL; + + mutex_lock(&inst->lock); + if (IS_SESSION_ERROR(inst)) { + ret = -EBUSY; + goto unlock; + } + + if (inst->domain != ENCODER) { + ret = -ENOTTY; + goto unlock; + } + + if (enc->cmd != V4L2_ENC_CMD_START && + enc->cmd != V4L2_ENC_CMD_STOP) { + ret = -EINVAL; + goto unlock; + } + + if (enc->cmd == V4L2_ENC_CMD_STOP && inst->state == IRIS_INST_OPEN) { + ret = 0; + goto unlock; + } + + if (!allow_cmd(inst, enc->cmd)) { + ret = -EBUSY; + goto unlock; + } + + ret = iris_pm_get(inst->core); + if (ret) + goto unlock; + + if (enc->cmd == V4L2_ENC_CMD_START) + ret = venc_start_cmd(inst); + else if (enc->cmd == V4L2_ENC_CMD_STOP) + ret = venc_stop_cmd(inst); + + iris_pm_put(inst->core, true); + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static struct v4l2_file_operations v4l2_file_ops = { .owner = THIS_MODULE, .open = vidc_open, @@ -1027,7 +1348,7 @@ static struct vb2_mem_ops iris_vb2_mem_ops = { .unmap_dmabuf = iris_vb2_unmap_dmabuf, }; -static const struct v4l2_ioctl_ops v4l2_ioctl_ops = { +static const struct v4l2_ioctl_ops v4l2_ioctl_ops_dec = { .vidioc_enum_fmt_vid_cap = vidc_enum_fmt, .vidioc_enum_fmt_vid_out = vidc_enum_fmt, .vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt, @@ -1055,12 +1376,44 @@ static const struct v4l2_ioctl_ops v4l2_ioctl_ops = { .vidioc_decoder_cmd = vidc_dec_cmd, }; +static const struct v4l2_ioctl_ops v4l2_ioctl_ops_enc = { + .vidioc_enum_fmt_vid_cap = vidc_enum_fmt, + .vidioc_enum_fmt_vid_out = vidc_enum_fmt, + .vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt, + .vidioc_try_fmt_vid_out_mplane = vidc_try_fmt, + .vidioc_s_fmt_vid_cap_mplane = vidc_s_fmt, + .vidioc_s_fmt_vid_out_mplane = vidc_s_fmt, + .vidioc_g_fmt_vid_cap_mplane = vidc_g_fmt, + .vidioc_g_fmt_vid_out_mplane = vidc_g_fmt, + .vidioc_enum_framesizes = vidc_enum_framesizes, + .vidioc_enum_frameintervals = vidc_enum_frameintervals, + .vidioc_reqbufs = vidc_reqbufs, + .vidioc_querybuf = vidc_querybuf, + .vidioc_create_bufs = vidc_create_bufs, + .vidioc_prepare_buf = vidc_prepare_buf, + .vidioc_qbuf = vidc_qbuf, + .vidioc_dqbuf = vidc_dqbuf, + .vidioc_streamon = vidc_streamon, + .vidioc_streamoff = vidc_streamoff, + .vidioc_querycap = vidc_querycap, + .vidioc_queryctrl = vidc_queryctrl, + .vidioc_querymenu = vidc_querymenu, + .vidioc_subscribe_event = vidc_subscribe_event, + .vidioc_unsubscribe_event = vidc_unsubscribe_event, + .vidioc_g_selection = vidc_g_selection, + .vidioc_s_selection = vidc_s_selection, + .vidioc_s_parm = vidc_s_parm, + .vidioc_g_parm = vidc_g_parm, + .vidioc_try_encoder_cmd = vidc_try_enc_cmd, + .vidioc_encoder_cmd = vidc_enc_cmd, +}; + int init_ops(struct iris_core *core) { core->v4l2_file_ops = &v4l2_file_ops; core->vb2_ops = &iris_vb2_ops; core->vb2_mem_ops = &iris_vb2_mem_ops; - core->v4l2_ioctl_ops = &v4l2_ioctl_ops; - + core->v4l2_ioctl_ops_dec = &v4l2_ioctl_ops_dec; + core->v4l2_ioctl_ops_enc = &v4l2_ioctl_ops_enc; return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 443894c..effecbb 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -115,6 +115,7 @@ struct iris_inst_power { enum plat_core_cap_type { CORE_CAP_NONE = 0, DEC_CODECS, + ENC_CODECS, MAX_SESSION_COUNT, MAX_MBPF, MAX_MBPS, @@ -249,6 +250,7 @@ struct plat_inst_cap { struct plat_inst_caps { enum codec_type codec; + enum domain_type domain; struct plat_inst_cap cap[INST_CAP_MAX + 1]; }; diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 6d5192a..ef0aad7 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -63,6 +63,7 @@ static struct color_format_info color_format_data_sm8550[] = { static struct plat_core_cap core_data_sm8550[] = { {DEC_CODECS, H264 | HEVC | VP9}, + {ENC_CODECS, H264 | HEVC}, {MAX_SESSION_COUNT, 16}, {MAX_MBPF, 278528}, /* ((8192x4352)/256) * 2 */ {MAX_MBPS, 7833600}, /* max_load 7680x4320@60fps */ From patchwork Mon Dec 18 11:32:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755541 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CE8839863; Mon, 18 Dec 2023 11:40:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="BwZrPQCo" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAto4o007911; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=DjeCHraDg5N5KWkgTBSLo6OX0aOTajhJqdyfe0iWaX8=; b=Bw ZrPQCo3m2x7GTbnlBE5tsR4gOI7b2JwjOcFjV4XV/iWzQWLs2G5K6Jgw0Wwo4MVo Ut/I85lEOtytVKtVOorkPTqlIDJ1lUX6xKpcs66SQAab4Tsafmz34Wqxc8HZHijC F4jEntymbQ8xpq0iFXkYUJdy3uZGepHqmMGVP7/COmdm2a247xwj2ZuvdJSlvVb1 574hqMRLrZjB/mS5r5K68CxIrCn8TV7QyFrAGmPtb/B8jVreKG0MpLoJwvPow0jB 0YnyWDcAqBQXdgsL9YFVlh1YlSy0mJhHUx/13wqQxazBAtM+oAa3uNyM1vOvtZtO gtz5VnTqCJCQaMxRbiBQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4xx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0wv004854; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ag-1; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX8re030072; Mon, 18 Dec 2023 11:36:10 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8rs030066; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 71B33238E; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 33/34] media: iris: add vb2 streaming and buffer ops for encoder Date: Mon, 18 Dec 2023 17:02:28 +0530 Message-Id: <1702899149-21321-34-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: bnmkmXwdjsNr_IO3z2ssx7KfZtGHpn73 X-Proofpoint-GUID: bnmkmXwdjsNr_IO3z2ssx7KfZtGHpn73 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: During stream on, set some mandatory properties to firmware to start a session. Set all v4l2 properties set by client, to firmware prepared with the dependency graph. Also, configure the hardware internal buffers required for frame processing. Send HFI_CMD_START on capture and output planes to start the processing on respective planes. The instance state is changed accordingly. During stream off, send HFI_CMD_STOP to firmware which is a synchronous command. After the response is received from firmware, the session is closed on firmware and instance state is changed accordingly. Buffers are submitted and received via HFI_CMD_BUFFER. Add Internal buffer size calculations for Encoder. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 11 + .../media/platform/qcom/vcodec/iris/iris_buffer.c | 296 ++++++++-- .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 49 +- .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 1 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 169 +++++- .../media/platform/qcom/vcodec/iris/iris_helpers.h | 6 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 22 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 137 +++-- .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 9 +- .../platform/qcom/vcodec/iris/iris_hfi_response.c | 143 +++-- drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 62 +- .../media/platform/qcom/vcodec/iris/iris_vdec.c | 137 +---- .../media/platform/qcom/vcodec/iris/iris_venc.c | 423 ++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_venc.h | 3 + .../platform/qcom/vcodec/iris/platform_common.h | 5 + .../platform/qcom/vcodec/iris/platform_sm8550.c | 2 + .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 198 ++++++- .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 624 +++++++++++++++++++++ .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 2 +- 19 files changed, 1992 insertions(+), 307 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index bc32c99..59765de 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -284,6 +284,10 @@ enum hfi_picture_type { #define HFI_PROP_NO_OUTPUT 0x0300016a +#define HFI_PROP_BUFFER_MARK 0x0300016c + +#define HFI_PROP_RAW_RESOLUTION 0x03000178 + #define HFI_PROP_DPB_LIST 0x0300017A #define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C @@ -372,4 +376,11 @@ enum hfi_buffer_firmware_flags { HFI_BUF_FW_FLAG_PSC_LAST = 0x20000000, }; +enum hfi_hevc_profile_type { + HFI_H265_PROFILE_MAIN = 0, + HFI_H265_PROFILE_MAIN_STILL_PICTURE = 1, + HFI_H265_PROFILE_MAIN_10 = 2, + HFI_H265_PROFILE_MAIN_10_STILL_PICTURE = 3, +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c index f85b52c..18418e4e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -10,6 +10,7 @@ #include "hfi_defines.h" #include "iris_hfi_packet.h" #include "iris_instance.h" +#include "vpu_iris3_buffer.h" static const u32 dec_ip_int_buf_type[] = { BUF_BIN, @@ -22,15 +23,51 @@ static const u32 dec_op_int_buf_type[] = { BUF_DPB, }; +static const u32 enc_ip_int_buf_type[] = { + BUF_VPSS, +}; + +static const u32 enc_op_int_buf_type[] = { + BUF_BIN, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, + BUF_DPB, +}; + static int input_min_count(struct iris_inst *inst) { - return MIN_BUFFERS; + u32 input_min_count = 0; + u32 total_hb_layer = 0; + + if (inst->domain == DECODER) { + input_min_count = MIN_BUFFERS; + } else if (inst->domain == ENCODER) { + total_hb_layer = is_hierb_type_requested(inst) ? + inst->cap[ENH_LAYER_COUNT].value + 1 : 0; + if (inst->codec == H264 && + !inst->cap[LAYER_ENABLE].value) { + total_hb_layer = 0; + } + input_min_count = + hfi_iris3_enc_min_input_buf_count(total_hb_layer); + } else { + return 0; + } + + return input_min_count; } static int output_min_count(struct iris_inst *inst) { int output_min_count; + if (inst->domain != DECODER && inst->domain != ENCODER) + return 0; + + if (inst->domain == ENCODER) + return MIN_BUFFERS; + /* fw_min_count > 0 indicates reconfig event has already arrived */ if (inst->fw_min_count) { if (is_split_mode_enabled(inst) && inst->codec == VP9) @@ -83,16 +120,21 @@ static u32 internal_buffer_count(struct iris_inst *inst, { u32 count = 0; - if (buffer_type == BUF_BIN || buffer_type == BUF_LINE || - buffer_type == BUF_PERSIST) { - count = 1; - } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) { - if (inst->codec == H264 || inst->codec == HEVC) + if (inst->domain == ENCODER) + return 1; + + if (inst->domain == DECODER) { + if (buffer_type == BUF_BIN || buffer_type == BUF_LINE || + buffer_type == BUF_PERSIST) { count = 1; - else + } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) { + if (inst->codec == H264 || inst->codec == HEVC) + count = 1; + else + count = 0; + } else { count = 0; - } else { - count = 0; + } } return count; @@ -102,6 +144,9 @@ static int dpb_count(struct iris_inst *inst) { int count = 0; + if (inst->domain == ENCODER) + return get_recon_buf_count(inst); + if (is_split_mode_enabled(inst)) { count = inst->fw_min_count ? inst->fw_min_count : inst->buffers.output.min_count; @@ -123,6 +168,7 @@ int iris_get_buf_min_count(struct iris_inst *inst, case BUF_NON_COMV: case BUF_LINE: case BUF_PERSIST: + case BUF_ARP: return internal_buffer_count(inst, buffer_type); case BUF_DPB: return dpb_count(inst); @@ -131,7 +177,7 @@ int iris_get_buf_min_count(struct iris_inst *inst, } } -static u32 input_buffer_size(struct iris_inst *inst) +static u32 dec_input_buffer_size(struct iris_inst *inst) { u32 base_res_mbs = NUM_MBS_4k; u32 frame_size, num_mbs; @@ -163,7 +209,7 @@ static u32 input_buffer_size(struct iris_inst *inst) return ALIGN(frame_size, SZ_4K); } -static u32 output_buffer_size(struct iris_inst *inst) +static u32 dec_output_buffer_size(struct iris_inst *inst) { struct v4l2_format *f; u32 size; @@ -175,17 +221,97 @@ static u32 output_buffer_size(struct iris_inst *inst) return size; } +static u32 enc_input_buffer_size(struct iris_inst *inst) +{ + struct v4l2_format *f; + u32 size; + + f = inst->fmt_src; + + size = video_raw_buffer_size(f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width, + f->fmt.pix_mp.height); + return size; +} + +static inline +u32 hfi_buffer_bitstream_enc(u32 frame_width, u32 frame_height, + u32 rc_type, bool is_ten_bit) +{ + u32 aligned_width, aligned_height, bitstream_size, yuv_size; + + aligned_width = ALIGN(frame_width, 32); + aligned_height = ALIGN(frame_height, 32); + bitstream_size = aligned_width * aligned_height * 3; + yuv_size = (aligned_width * aligned_height * 3) >> 1; + if (aligned_width * aligned_height > (4096 * 2176)) + /* bitstream_size = 0.25 * yuv_size; */ + bitstream_size = (bitstream_size >> 3); + else if (aligned_width * aligned_height > (1280 * 720)) + /* bitstream_size = 0.5 * yuv_size; */ + bitstream_size = (bitstream_size >> 2); + + if ((rc_type == HFI_RC_CQ || rc_type == HFI_RC_OFF) && + bitstream_size < yuv_size) + bitstream_size = (bitstream_size << 1); + + if (is_ten_bit) + bitstream_size = (bitstream_size) + (bitstream_size >> 2); + + return ALIGN(bitstream_size, HFI_ALIGNMENT_4096); +} + +static u32 enc_output_buffer_size(struct iris_inst *inst) +{ + u32 hfi_rc_type = HFI_RC_VBR_CFR; + enum codec_type codec; + int bitrate_mode, frame_rc; + bool is_ten_bit = false; + struct v4l2_format *f; + u32 frame_size; + + f = inst->fmt_dst; + codec = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat); + if (codec == HEVC) + is_ten_bit = true; + + bitrate_mode = inst->cap[BITRATE_MODE].value; + frame_rc = inst->cap[FRAME_RC_ENABLE].value; + if (!frame_rc) + hfi_rc_type = HFI_RC_OFF; + else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) + hfi_rc_type = HFI_RC_CQ; + + frame_size = hfi_buffer_bitstream_enc(f->fmt.pix_mp.width, + f->fmt.pix_mp.height, + hfi_rc_type, is_ten_bit); + + return frame_size; +} + int iris_get_buffer_size(struct iris_inst *inst, enum iris_buffer_type buffer_type) { - switch (buffer_type) { - case BUF_INPUT: - return input_buffer_size(inst); - case BUF_OUTPUT: - return output_buffer_size(inst); - default: - return 0; + if (inst->domain == DECODER) { + switch (buffer_type) { + case BUF_INPUT: + return dec_input_buffer_size(inst); + case BUF_OUTPUT: + return dec_output_buffer_size(inst); + default: + break; + } + } else if (inst->domain == ENCODER) { + switch (buffer_type) { + case BUF_INPUT: + return enc_input_buffer_size(inst); + case BUF_OUTPUT: + return enc_output_buffer_size(inst); + default: + break; + } } + + return 0; } struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst, @@ -300,14 +426,30 @@ int iris_get_internal_buffers(struct iris_inst *inst, int ret = 0; u32 i = 0; - if (plane == INPUT_MPLANE) { - for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { - ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]); - if (ret) - return ret; + if (inst->domain == DECODER) { + if (plane == INPUT_MPLANE) { + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } + } else { + return iris_get_internal_buf_info(inst, BUF_DPB); + } + } else if (inst->domain == ENCODER) { + if (plane == INPUT_MPLANE) { + for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) { + ret = iris_get_internal_buf_info(inst, enc_ip_int_buf_type[i]); + if (ret) + return ret; + } + } else { + for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) { + ret = iris_get_internal_buf_info(inst, enc_op_int_buf_type[i]); + if (ret) + return ret; + } } - } else { - return iris_get_internal_buf_info(inst, BUF_DPB); } return ret; @@ -377,10 +519,18 @@ int iris_create_input_internal_buffers(struct iris_inst *inst) int ret = 0; u32 i = 0; - for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { - ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]); - if (ret) - return ret; + if (inst->domain == DECODER) { + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } + } else if (inst->domain == ENCODER) { + for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) { + ret = iris_create_internal_buffers(inst, enc_ip_int_buf_type[i]); + if (ret) + return ret; + } } return ret; @@ -388,7 +538,20 @@ int iris_create_input_internal_buffers(struct iris_inst *inst) int iris_create_output_internal_buffers(struct iris_inst *inst) { - return iris_create_internal_buffers(inst, BUF_DPB); + int ret = 0; + u32 i = 0; + + if (inst->domain == DECODER) { + return iris_create_internal_buffers(inst, BUF_DPB); + } else if (inst->domain == ENCODER) { + for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) { + ret = iris_create_internal_buffers(inst, enc_op_int_buf_type[i]); + if (ret) + return ret; + } + } + + return ret; } static int set_num_comv(struct iris_inst *inst) @@ -413,7 +576,7 @@ static int iris_queue_internal_buffers(struct iris_inst *inst, struct iris_buffers *buffers; int ret = 0; - if (buffer_type == BUF_COMV) { + if (inst->domain == DECODER && buffer_type == BUF_COMV) { ret = set_num_comv(inst); if (ret) return ret; @@ -442,10 +605,18 @@ int iris_queue_input_internal_buffers(struct iris_inst *inst) int ret = 0; u32 i; - for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { - ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]); - if (ret) - return ret; + if (inst->domain == DECODER) { + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } + } else if (inst->domain == ENCODER) { + for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) { + ret = iris_queue_internal_buffers(inst, enc_ip_int_buf_type[i]); + if (ret) + return ret; + } } return ret; @@ -453,7 +624,20 @@ int iris_queue_input_internal_buffers(struct iris_inst *inst) int iris_queue_output_internal_buffers(struct iris_inst *inst) { - return iris_queue_internal_buffers(inst, BUF_DPB); + int ret = 0; + u32 i = 0; + + if (inst->domain == DECODER) { + return iris_queue_internal_buffers(inst, BUF_DPB); + } else if (inst->domain == ENCODER) { + for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) { + ret = iris_queue_internal_buffers(inst, enc_op_int_buf_type[i]); + if (ret) + return ret; + } + } + + return ret; } int iris_destroy_internal_buffer(struct iris_inst *inst, @@ -489,16 +673,26 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, { struct iris_buffer *buf, *dummy; struct iris_buffers *buffers; - const u32 *internal_buf_type; + const u32 *internal_buf_type = NULL; int ret = 0; - u32 i, len; - - if (plane == INPUT_MPLANE) { - internal_buf_type = dec_ip_int_buf_type; - len = ARRAY_SIZE(dec_ip_int_buf_type); - } else { - internal_buf_type = dec_op_int_buf_type; - len = ARRAY_SIZE(dec_op_int_buf_type); + u32 i, len = 0; + + if (inst->domain == DECODER) { + if (plane == INPUT_MPLANE) { + internal_buf_type = dec_ip_int_buf_type; + len = ARRAY_SIZE(dec_ip_int_buf_type); + } else { + internal_buf_type = dec_op_int_buf_type; + len = ARRAY_SIZE(dec_op_int_buf_type); + } + } else if (inst->domain == ENCODER) { + if (plane == INPUT_MPLANE) { + internal_buf_type = enc_ip_int_buf_type; + len = ARRAY_SIZE(enc_ip_int_buf_type); + } else { + internal_buf_type = enc_op_int_buf_type; + len = ARRAY_SIZE(enc_op_int_buf_type); + } } for (i = 0; i < len; i++) { @@ -552,10 +746,12 @@ static int iris_release_input_internal_buffers(struct iris_inst *inst) int ret = 0; u32 i = 0; - for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { - ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]); - if (ret) - return ret; + if (inst->domain == DECODER) { + for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) { + ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]); + if (ret) + return ret; + } } return ret; @@ -566,7 +762,7 @@ int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst, { int ret; - if (buffer_type != BUF_PERSIST) + if (buffer_type != BUF_ARP && buffer_type != BUF_PERSIST) return -EINVAL; ret = iris_get_internal_buf_info(inst, buffer_type); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c index af99ac73..559b0dd 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -723,11 +723,20 @@ int set_stage(struct iris_inst *inst, hfi_id = inst->cap[cap_id].hfi_id; - inp_f = inst->fmt_src; - height = inp_f->fmt.pix_mp.height; - width = inp_f->fmt.pix_mp.width; - if (res_is_less_than(width, height, 1280, 720)) - work_mode = STAGE_1; + if (inst->domain == DECODER) { + inp_f = inst->fmt_src; + height = inp_f->fmt.pix_mp.height; + width = inp_f->fmt.pix_mp.width; + if (res_is_less_than(width, height, 1280, 720)) + work_mode = STAGE_1; + } else if (inst->domain == ENCODER) { + if (inst->cap[SLICE_MODE].value == + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES) + work_mode = STAGE_1; + + if (!inst->cap[GOP_SIZE].value) + work_mode = STAGE_2; + } return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, get_port_info(inst, cap_id), @@ -743,6 +752,12 @@ int set_pipe(struct iris_inst *inst, work_route = inst->cap[cap_id].value; hfi_id = inst->cap[cap_id].hfi_id; + if (inst->domain == ENCODER) { + if (inst->cap[SLICE_MODE].value == + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES) + work_route = PIPE_1; + } + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, get_port_info(inst, cap_id), HFI_PAYLOAD_U32, @@ -763,6 +778,30 @@ int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id) &hfi_value, sizeof(u32)); } +int decide_quality_mode(struct iris_inst *inst) +{ + u32 fps, mbpf, mbps, max_hq_mbpf, max_hq_mbps; + u32 mode = POWER_SAVE_MODE; + struct iris_core *core; + + if (inst->domain != ENCODER) + return 0; + + mbpf = NUM_MBS_PER_FRAME(inst->crop.height, inst->crop.width); + fps = max3(inst->cap[QUEUED_RATE].value >> 16, + inst->cap[FRAME_RATE].value >> 16, + inst->cap[OPERATING_RATE].value >> 16); + mbps = mbpf * fps; + core = inst->core; + max_hq_mbpf = core->cap[MAX_MBPF_HQ].value; + max_hq_mbps = core->cap[MAX_MBPS_HQ].value; + + if (mbpf <= max_hq_mbpf && mbps <= max_hq_mbps) + mode = MAX_QUALITY_MODE; + + return mode; +} + int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id) { u32 hfi_id, hfi_val; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h index 5421d9f..404d98c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h @@ -33,6 +33,7 @@ int adjust_v4l2_properties(struct iris_inst *inst); int ctrls_init(struct iris_inst *inst, bool init); int set_q16(struct iris_inst *inst, enum plat_inst_cap_type cap_id); int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int decide_quality_mode(struct iris_inst *inst); int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id); int set_flip(struct iris_inst *inst, enum plat_inst_cap_type cap_id); int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index c84bb51..a770157 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -44,15 +44,15 @@ u32 get_port_info(struct iris_inst *inst, if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT && inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) { if (inst->vb2q_dst->streaming) - return get_hfi_port(INPUT_MPLANE); + return get_hfi_port(inst, INPUT_MPLANE); else - return get_hfi_port(OUTPUT_MPLANE); + return get_hfi_port(inst, OUTPUT_MPLANE); } if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT) - return get_hfi_port(INPUT_MPLANE); + return get_hfi_port(inst, INPUT_MPLANE); else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) - return get_hfi_port(OUTPUT_MPLANE); + return get_hfi_port(inst, OUTPUT_MPLANE); else return HFI_PORT_NONE; } @@ -86,9 +86,14 @@ int get_mbpf(struct iris_inst *inst) int height = 0, width = 0; struct v4l2_format *inp_f; - inp_f = inst->fmt_src; - width = max(inp_f->fmt.pix_mp.width, inst->crop.width); - height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + if (inst->domain == DECODER) { + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + } else if (inst->domain == ENCODER) { + width = inst->crop.width; + height = inst->crop.height; + } return NUM_MBS_PER_FRAME(height, width); } @@ -100,6 +105,9 @@ inline bool is_linear_colorformat(u32 colorformat) bool is_split_mode_enabled(struct iris_inst *inst) { + if (inst->domain != DECODER) + return false; + if (is_linear_colorformat(inst->fmt_dst->fmt.pix_mp.pixelformat)) return true; @@ -118,6 +126,24 @@ inline bool is_8bit_colorformat(enum colorformat_type colorformat) colorformat == FMT_NV21; } +inline bool is_scaling_enabled(struct iris_inst *inst) +{ + return inst->crop.left != inst->compose.left || + inst->crop.top != inst->compose.top || + inst->crop.width != inst->compose.width || + inst->crop.height != inst->compose.height; +} + +inline bool is_hierb_type_requested(struct iris_inst *inst) +{ + return (inst->codec == H264 && + inst->cap[LAYER_TYPE].value == + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) || + (inst->codec == HEVC && + inst->cap[LAYER_TYPE].value == + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B); +} + u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec) { const struct codec_info *codec_info; @@ -319,17 +345,24 @@ static int check_resolution_supported(struct iris_inst *inst) u32 width = 0, height = 0, min_width, min_height, max_width, max_height; - width = inst->fmt_src->fmt.pix_mp.width; - height = inst->fmt_src->fmt.pix_mp.height; + if (inst->domain == DECODER) { + width = inst->fmt_src->fmt.pix_mp.width; + height = inst->fmt_src->fmt.pix_mp.height; + } else if (inst->domain == ENCODER) { + width = inst->crop.width; + height = inst->crop.height; + } min_width = inst->cap[FRAME_WIDTH].min; max_width = inst->cap[FRAME_WIDTH].max; min_height = inst->cap[FRAME_HEIGHT].min; max_height = inst->cap[FRAME_HEIGHT].max; - if (!(min_width <= width && width <= max_width) || - !(min_height <= height && height <= max_height)) - return -EINVAL; + if (inst->domain == DECODER || inst->domain == ENCODER) { + if (!(min_width <= width && width <= max_width) || + !(min_height <= height && height <= max_height)) + return -EINVAL; + } return 0; } @@ -470,7 +503,7 @@ int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf) { int ret; - if (buf->type == BUF_OUTPUT) + if (inst->domain == DECODER && buf->type == BUF_OUTPUT) process_requeued_readonly_buffers(inst, buf); ret = iris_hfi_queue_buffer(inst, buf); @@ -653,7 +686,7 @@ static int iris_flush_read_only_buffers(struct iris_inst *inst, { struct iris_buffer *ro_buf, *dummy; - if (type != BUF_OUTPUT) + if (inst->domain != DECODER || type != BUF_OUTPUT) return 0; list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { @@ -899,6 +932,114 @@ int codec_change(struct iris_inst *inst, u32 v4l2_codec) return ret; } +int process_streamon_input(struct iris_inst *inst) +{ + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; + int ret; + + iris_scale_power(inst); + + ret = iris_hfi_start(inst, INPUT_MPLANE); + if (ret) + return ret; + + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0); + if (ret) + return ret; + } + + if (inst->sub_state & IRIS_INST_SUB_DRC || + inst->sub_state & IRIS_INST_SUB_DRAIN) { + if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) { + ret = iris_hfi_pause(inst, INPUT_MPLANE); + if (ret) + return ret; + set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; + } + } + + ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE); + if (ret) + return ret; + + ret = iris_inst_change_sub_state(inst, 0, set_sub_state); + + return ret; +} + +int process_streamon_output(struct iris_inst *inst) +{ + enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; + bool drain_pending = false; + int ret; + + iris_scale_power(inst); + + if (inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST) + clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + + if (inst->domain == DECODER && inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_alloc_and_queue_input_int_bufs(inst); + if (ret) + return ret; + ret = set_stage(inst, STAGE); + if (ret) + return ret; + ret = set_pipe(inst, PIPE); + if (ret) + return ret; + } + + drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST; + + if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) { + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + } + + ret = iris_hfi_start(inst, OUTPUT_MPLANE); + if (ret) + return ret; + + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + + ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE); + if (ret) + return ret; + + ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); + + return ret; +} + +int vb2_buffer_to_driver(struct vb2_buffer *vb2, struct iris_buffer *buf) +{ + struct vb2_v4l2_buffer *vbuf; + + if (!vb2 || !buf) + return -EINVAL; + + vbuf = to_vb2_v4l2_buffer(vb2); + + buf->fd = vb2->planes[0].m.fd; + buf->data_offset = vb2->planes[0].data_offset; + buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset; + buf->buffer_size = vb2->planes[0].length; + buf->timestamp = vb2->timestamp; + buf->flags = vbuf->flags; + buf->attr = 0; + + return 0; +} + int iris_pm_get(struct iris_core *core) { int ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 39cec8c..bf8448a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -31,7 +31,9 @@ int close_session(struct iris_inst *inst); bool is_linear_colorformat(u32 colorformat); bool is_10bit_colorformat(enum colorformat_type colorformat); bool is_8bit_colorformat(enum colorformat_type colorformat); +bool is_scaling_enabled(struct iris_inst *inst); bool is_split_mode_enabled(struct iris_inst *inst); +bool is_hierb_type_requested(struct iris_inst *inst); int signal_session_msg_receipt(struct iris_inst *inst, enum signal_session_response cmd); struct iris_inst *to_instance(struct iris_core *core, u32 session_id); @@ -54,6 +56,10 @@ void iris_destroy_buffers(struct iris_inst *inst); int session_streamoff(struct iris_inst *inst, u32 plane); int process_resume(struct iris_inst *inst); int codec_change(struct iris_inst *inst, u32 v4l2_codec); +int process_streamon_input(struct iris_inst *inst); +int process_streamon_output(struct iris_inst *inst); +int vb2_buffer_to_driver(struct vb2_buffer *vb2, + struct iris_buffer *buf); int iris_pm_get(struct iris_core *core); int iris_pm_put(struct iris_core *core, bool autosuspend); int iris_pm_get_put(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 00e598d..4a54293 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -356,7 +356,7 @@ int iris_hfi_session_set_default_header(struct iris_inst *inst) ret = hfi_packet_session_property(inst, HFI_PROP_DEC_DEFAULT_HEADER, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32, &default_header, sizeof(u32)); @@ -394,7 +394,7 @@ int iris_hfi_start(struct iris_inst *inst, u32 plane) HFI_CMD_START, (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED), - get_hfi_port(plane), + get_hfi_port(inst, plane), inst->session_id, HFI_PAYLOAD_NONE, NULL, @@ -434,7 +434,7 @@ int iris_hfi_stop(struct iris_inst *inst, u32 plane) (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED | HFI_HOST_FLAGS_NON_DISCARDABLE), - get_hfi_port(plane), + get_hfi_port(inst, plane), inst->session_id, HFI_PAYLOAD_NONE, NULL, @@ -472,7 +472,7 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst, cmd, (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED), - get_hfi_port(plane), + get_hfi_port(inst, plane), inst->session_id, payload_type, payload, @@ -511,7 +511,7 @@ int iris_hfi_pause(struct iris_inst *inst, u32 plane) HFI_CMD_PAUSE, (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED), - get_hfi_port(plane), + get_hfi_port(inst, plane), inst->session_id, HFI_PAYLOAD_NONE, NULL, @@ -550,7 +550,7 @@ int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 payload) HFI_CMD_RESUME, (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED), - get_hfi_port(plane), + get_hfi_port(inst, plane), inst->session_id, HFI_PAYLOAD_U32, &payload, @@ -590,7 +590,7 @@ int iris_hfi_drain(struct iris_inst *inst, u32 plane) (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED | HFI_HOST_FLAGS_NON_DISCARDABLE), - get_hfi_port(plane), + get_hfi_port(inst, plane), inst->session_id, HFI_PAYLOAD_NONE, NULL, @@ -739,14 +739,14 @@ int iris_hfi_queue_buffer(struct iris_inst *inst, goto unlock; } - ret = get_hfi_buffer(buffer, &hfi_buffer); + ret = get_hfi_buffer(inst, buffer, &hfi_buffer); if (ret) goto unlock; ret = hfi_packet_session_command(inst, HFI_CMD_BUFFER, HFI_HOST_FLAGS_INTR_REQUIRED, - get_hfi_port_from_buffer_type(buffer->type), + get_hfi_port_from_buffer_type(inst, buffer->type), inst->session_id, HFI_PAYLOAD_STRUCTURE, &hfi_buffer, @@ -780,7 +780,7 @@ int iris_hfi_release_buffer(struct iris_inst *inst, goto unlock; } - ret = get_hfi_buffer(buffer, &hfi_buffer); + ret = get_hfi_buffer(inst, buffer, &hfi_buffer); if (ret) goto unlock; @@ -790,7 +790,7 @@ int iris_hfi_release_buffer(struct iris_inst *inst, HFI_CMD_BUFFER, (HFI_HOST_FLAGS_RESPONSE_REQUIRED | HFI_HOST_FLAGS_INTR_REQUIRED), - get_hfi_port_from_buffer_type(buffer->type), + get_hfi_port_from_buffer_type(inst, buffer->type), inst->session_id, HFI_PAYLOAD_STRUCTURE, &hfi_buffer, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index dd27fa4..e9f3749 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -3,64 +3,108 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include "iris_common.h" #include "iris_core.h" #include "iris_helpers.h" #include "iris_hfi_packet.h" #include "hfi_defines.h" -u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type) +u32 get_hfi_port_from_buffer_type(struct iris_inst *inst, enum iris_buffer_type buffer_type) { u32 hfi_port = HFI_PORT_NONE; - switch (buffer_type) { - case BUF_INPUT: - case BUF_BIN: - case BUF_COMV: - case BUF_NON_COMV: - case BUF_LINE: - hfi_port = HFI_PORT_BITSTREAM; - break; - case BUF_OUTPUT: - case BUF_DPB: - hfi_port = HFI_PORT_RAW; - break; - case BUF_PERSIST: - hfi_port = HFI_PORT_NONE; - break; - default: - break; + if (inst->domain == DECODER) { + switch (buffer_type) { + case BUF_INPUT: + case BUF_BIN: + case BUF_COMV: + case BUF_NON_COMV: + case BUF_LINE: + hfi_port = HFI_PORT_BITSTREAM; + break; + case BUF_OUTPUT: + case BUF_DPB: + hfi_port = HFI_PORT_RAW; + break; + case BUF_PERSIST: + hfi_port = HFI_PORT_NONE; + break; + default: + break; + } + } else if (inst->domain == ENCODER) { + switch (buffer_type) { + case BUF_INPUT: + case BUF_VPSS: + hfi_port = HFI_PORT_RAW; + break; + case BUF_OUTPUT: + case BUF_BIN: + case BUF_COMV: + case BUF_NON_COMV: + case BUF_LINE: + case BUF_DPB: + hfi_port = HFI_PORT_BITSTREAM; + break; + case BUF_ARP: + hfi_port = HFI_PORT_NONE; + break; + default: + break; + } } return hfi_port; } -u32 get_hfi_port(u32 plane) +u32 get_hfi_port(struct iris_inst *inst, u32 plane) { u32 hfi_port = HFI_PORT_NONE; - switch (plane) { - case INPUT_MPLANE: - hfi_port = HFI_PORT_BITSTREAM; - break; - case OUTPUT_MPLANE: - hfi_port = HFI_PORT_RAW; - break; - default: - break; + if (inst->domain == DECODER) { + switch (plane) { + case INPUT_MPLANE: + hfi_port = HFI_PORT_BITSTREAM; + break; + case OUTPUT_MPLANE: + hfi_port = HFI_PORT_RAW; + break; + default: + break; + } + } else if (inst->domain == ENCODER) { + switch (plane) { + case INPUT_MPLANE: + hfi_port = HFI_PORT_RAW; + break; + case OUTPUT_MPLANE: + hfi_port = HFI_PORT_BITSTREAM; + break; + default: + break; + } } return hfi_port; } -static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) +static u32 hfi_buf_type_from_driver(enum domain_type domain, enum iris_buffer_type buffer_type) { switch (buffer_type) { case BUF_INPUT: - return HFI_BUFFER_BITSTREAM; + if (domain == DECODER) + return HFI_BUFFER_BITSTREAM; + else + return HFI_BUFFER_RAW; case BUF_OUTPUT: - return HFI_BUFFER_RAW; + if (domain == DECODER) + return HFI_BUFFER_RAW; + else + return HFI_BUFFER_BITSTREAM; case BUF_BIN: return HFI_BUFFER_BIN; + case BUF_ARP: + return HFI_BUFFER_ARP; case BUF_COMV: return HFI_BUFFER_COMV; case BUF_NON_COMV: @@ -76,13 +120,19 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) } } -u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type) +u32 hfi_buf_type_to_driver(enum domain_type domain, enum hfi_buffer_type buf_type) { switch (buf_type) { case HFI_BUFFER_BITSTREAM: - return BUF_INPUT; + if (domain == DECODER) + return BUF_INPUT; + else + return BUF_OUTPUT; case HFI_BUFFER_RAW: - return BUF_OUTPUT; + if (domain == DECODER) + return BUF_OUTPUT; + else + return BUF_INPUT; case HFI_BUFFER_BIN: return BUF_BIN; case HFI_BUFFER_ARP: @@ -108,9 +158,15 @@ u32 get_hfi_codec(struct iris_inst *inst) { switch (inst->codec) { case H264: - return HFI_CODEC_DECODE_AVC; + if (inst->domain == ENCODER) + return HFI_CODEC_ENCODE_AVC; + else + return HFI_CODEC_DECODE_AVC; case HEVC: - return HFI_CODEC_DECODE_HEVC; + if (inst->domain == ENCODER) + return HFI_CODEC_ENCODE_HEVC; + else + return HFI_CODEC_DECODE_HEVC; case VP9: return HFI_CODEC_DECODE_VP9; default: @@ -337,10 +393,11 @@ u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients) return coefficients; } -int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) +int get_hfi_buffer(struct iris_inst *inst, + struct iris_buffer *buffer, struct hfi_buffer *buf) { - memset(buf, 0, sizeof(*buf)); - buf->type = hfi_buf_type_from_driver(buffer->type); + memset(buf, 0, sizeof(struct hfi_buffer)); + buf->type = hfi_buf_type_from_driver(inst->domain, buffer->type); buf->index = buffer->index; buf->base_address = buffer->device_addr; buf->addr_offset = 0; @@ -350,7 +407,7 @@ int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) * buffer size otherwise it will truncate or ignore the data after 256 * aligned size which may lead to error concealment */ - if (buffer->type == BUF_INPUT) + if (inst->domain == DECODER && buffer->type == BUF_INPUT) buf->buffer_size = ALIGN(buffer->buffer_size, 256); buf->data_offset = buffer->data_offset; buf->data_size = buffer->data_size; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index 82148b7..734a070 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -80,8 +80,8 @@ enum hfi_packet_port_type { HFI_PORT_RAW = 0x00000002, }; -u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type); -u32 get_hfi_port(u32 plane); +u32 get_hfi_port_from_buffer_type(struct iris_inst *inst, enum iris_buffer_type buffer_type); +u32 get_hfi_port(struct iris_inst *inst, u32 plane); u32 get_hfi_colorformat(u32 colorformat); u32 get_hfi_codec(struct iris_inst *inst); u32 get_hfi_color_primaries(u32 primaries); @@ -90,8 +90,9 @@ u32 get_hfi_matrix_coefficients(u32 coefficients); u32 get_v4l2_color_primaries(u32 hfi_primaries); u32 get_v4l2_transfer_char(u32 hfi_characterstics); u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients); -u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type); -int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); +u32 hfi_buf_type_to_driver(enum domain_type domain, enum hfi_buffer_type buf_type); +int get_hfi_buffer(struct iris_inst *inst, + struct iris_buffer *buffer, struct hfi_buffer *buf); int hfi_packet_sys_init(struct iris_core *core, u8 *pkt, u32 pkt_size); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index 1b667a5..08bdb8f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -71,7 +71,8 @@ static bool is_valid_hfi_buffer_type(u32 buffer_type) static bool is_valid_hfi_port(u32 port, u32 buffer_type) { if (port == HFI_PORT_NONE && - buffer_type != HFI_BUFFER_PERSIST) + buffer_type != HFI_BUFFER_ARP && + buffer_type != HFI_BUFFER_PERSIST) return false; if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW) @@ -103,8 +104,9 @@ static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags) if (inst->hfi_frame_info.overflow) driver_flags |= BUF_FLAG_ERROR; - if (hfi_flags & HFI_BUF_FW_FLAG_LAST || - hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST) + if ((inst->domain == ENCODER && (hfi_flags & HFI_BUF_FW_FLAG_LAST)) || + (inst->domain == DECODER && (hfi_flags & HFI_BUF_FW_FLAG_LAST || + hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST))) driver_flags |= BUF_FLAG_LAST; return driver_flags; @@ -309,9 +311,12 @@ static int handle_session_close(struct iris_inst *inst, static int handle_read_only_buffer(struct iris_inst *inst, struct iris_buffer *buf) { - struct iris_buffer *ro_buf, *iter; + struct iris_buffer *ro_buf, *iter = NULL; bool found = false; + if (inst->domain != DECODER && inst->domain != ENCODER) + return 0; + list_for_each_entry(iter, &inst->buffers.read_only.list, list) { if (iter->device_addr == buf->device_addr) { found = true; @@ -344,6 +349,9 @@ static int handle_non_read_only_buffer(struct iris_inst *inst, { struct iris_buffer *ro_buf; + if (inst->domain != DECODER && inst->domain != ENCODER) + return 0; + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { if (ro_buf->device_addr == buffer->base_address) { ro_buf->attr &= ~BUF_ATTR_READ_ONLY; @@ -414,9 +422,9 @@ static int handle_input_buffer(struct iris_inst *inst, static int handle_output_buffer(struct iris_inst *inst, struct hfi_buffer *hfi_buffer) { - struct iris_buffers *buffers; struct iris_buffer *buf, *iter; - bool found; + struct iris_buffers *buffers; + bool found, fatal = false; int ret = 0; if (hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) { @@ -425,17 +433,19 @@ static int handle_output_buffer(struct iris_inst *inst, return ret; } - if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE) - return handle_release_output_buffer(inst, hfi_buffer); + if (inst->domain == DECODER) { + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE) + return handle_release_output_buffer(inst, hfi_buffer); - if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) { - ret = iris_inst_sub_state_change_drc_last(inst); - if (ret) - return ret; - } + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) { + ret = iris_inst_sub_state_change_drc_last(inst); + if (ret) + return ret; + } - if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)) - ret = handle_non_read_only_buffer(inst, hfi_buffer); + if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)) + ret = handle_non_read_only_buffer(inst, hfi_buffer); + } buffers = iris_get_buffer_list(inst, BUF_OUTPUT); if (!buffers) @@ -446,9 +456,12 @@ static int handle_output_buffer(struct iris_inst *inst, if (!(iter->attr & BUF_ATTR_QUEUED)) continue; - found = (iter->index == hfi_buffer->index && - iter->device_addr == hfi_buffer->base_address && - iter->data_offset == hfi_buffer->data_offset); + if (inst->domain == DECODER) + found = (iter->index == hfi_buffer->index && + iter->device_addr == hfi_buffer->base_address && + iter->data_offset == hfi_buffer->data_offset); + else + found = iter->index == hfi_buffer->index; if (found) { buf = iter; @@ -465,14 +478,27 @@ static int handle_output_buffer(struct iris_inst *inst, buf->attr &= ~BUF_ATTR_QUEUED; buf->attr |= BUF_ATTR_DEQUEUED; - if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) - iris_inst_change_state(inst, IRIS_INST_ERROR); + if (inst->domain == ENCODER) { + if (inst->hfi_frame_info.data_corrupt) + fatal = true; + if (inst->hfi_frame_info.overflow) { + if (!hfi_buffer->data_size && inst->hfi_rc_type == HFI_RC_CBR_CFR) + fatal = true; + } + if (fatal) + iris_inst_change_state(inst, IRIS_INST_ERROR); + } + + if (inst->domain == DECODER) { + if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) + iris_inst_change_state(inst, IRIS_INST_ERROR); - if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) { - buf->attr |= BUF_ATTR_READ_ONLY; - ret = handle_read_only_buffer(inst, buf); - } else { - buf->attr &= ~BUF_ATTR_READ_ONLY; + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) { + buf->attr |= BUF_ATTR_READ_ONLY; + ret = handle_read_only_buffer(inst, buf); + } else { + buf->attr &= ~BUF_ATTR_READ_ONLY; + } } buf->flags = get_driver_buffer_flags(inst, hfi_buffer->flags); @@ -521,7 +547,7 @@ static int handle_release_internal_buffer(struct iris_inst *inst, int ret = 0; bool found; - buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(buffer->type)); + buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(inst->domain, buffer->type)); if (!buffers) return -EINVAL; @@ -550,12 +576,22 @@ static int handle_session_stop(struct iris_inst *inst, int ret = 0; enum signal_session_response signal_type = -1; - if (pkt->port == HFI_PORT_RAW) { - signal_type = SIGNAL_CMD_STOP_OUTPUT; - ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE); - } else if (pkt->port == HFI_PORT_BITSTREAM) { - signal_type = SIGNAL_CMD_STOP_INPUT; - ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE); + if (inst->domain == DECODER) { + if (pkt->port == HFI_PORT_RAW) { + signal_type = SIGNAL_CMD_STOP_OUTPUT; + ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE); + } else if (pkt->port == HFI_PORT_BITSTREAM) { + signal_type = SIGNAL_CMD_STOP_INPUT; + ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE); + } + } else if (inst->domain == ENCODER) { + if (pkt->port == HFI_PORT_RAW) { + signal_type = SIGNAL_CMD_STOP_INPUT; + ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE); + } else if (pkt->port == HFI_PORT_BITSTREAM) { + signal_type = SIGNAL_CMD_STOP_OUTPUT; + ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE); + } } if (signal_type != -1) @@ -571,7 +607,7 @@ static int handle_session_buffer(struct iris_inst *inst, u32 hfi_handle_size = 0; int i, ret = 0; const struct iris_hfi_buffer_handle *hfi_handle_arr = NULL; - static const struct iris_hfi_buffer_handle input_hfi_handle[] = { + static const struct iris_hfi_buffer_handle dec_input_hfi_handle[] = { {HFI_BUFFER_BITSTREAM, handle_input_buffer }, {HFI_BUFFER_BIN, handle_release_internal_buffer }, {HFI_BUFFER_COMV, handle_release_internal_buffer }, @@ -579,10 +615,24 @@ static int handle_session_buffer(struct iris_inst *inst, {HFI_BUFFER_LINE, handle_release_internal_buffer }, {HFI_BUFFER_PERSIST, handle_release_internal_buffer }, }; - static const struct iris_hfi_buffer_handle output_hfi_handle[] = { + static const struct iris_hfi_buffer_handle dec_output_hfi_handle[] = { {HFI_BUFFER_RAW, handle_output_buffer }, {HFI_BUFFER_DPB, handle_release_internal_buffer }, }; + static const struct iris_hfi_buffer_handle enc_input_hfi_handle[] = { + {HFI_BUFFER_RAW, handle_input_buffer }, + {HFI_BUFFER_VPSS, handle_release_internal_buffer }, + }; + static const struct iris_hfi_buffer_handle enc_output_hfi_handle[] = { + {HFI_BUFFER_BITSTREAM, handle_output_buffer }, + {HFI_BUFFER_BIN, handle_release_internal_buffer }, + {HFI_BUFFER_COMV, handle_release_internal_buffer }, + {HFI_BUFFER_NON_COMV, handle_release_internal_buffer }, + {HFI_BUFFER_LINE, handle_release_internal_buffer }, + {HFI_BUFFER_ARP, handle_release_internal_buffer }, + {HFI_BUFFER_DPB, handle_release_internal_buffer }, + }; + if (pkt->payload_info == HFI_PAYLOAD_NONE) return 0; @@ -599,12 +649,22 @@ static int handle_session_buffer(struct iris_inst *inst, if (!is_valid_hfi_port(pkt->port, buffer->type)) return 0; - if (pkt->port == HFI_PORT_BITSTREAM) { - hfi_handle_size = ARRAY_SIZE(input_hfi_handle); - hfi_handle_arr = input_hfi_handle; - } else if (pkt->port == HFI_PORT_RAW) { - hfi_handle_size = ARRAY_SIZE(output_hfi_handle); - hfi_handle_arr = output_hfi_handle; + if (inst->domain == DECODER) { + if (pkt->port == HFI_PORT_BITSTREAM) { + hfi_handle_size = ARRAY_SIZE(dec_input_hfi_handle); + hfi_handle_arr = dec_input_hfi_handle; + } else if (pkt->port == HFI_PORT_RAW) { + hfi_handle_size = ARRAY_SIZE(dec_output_hfi_handle); + hfi_handle_arr = dec_output_hfi_handle; + } + } else if (inst->domain == ENCODER) { + if (pkt->port == HFI_PORT_RAW) { + hfi_handle_size = ARRAY_SIZE(enc_input_hfi_handle); + hfi_handle_arr = enc_input_hfi_handle; + } else if (pkt->port == HFI_PORT_BITSTREAM) { + hfi_handle_size = ARRAY_SIZE(enc_output_hfi_handle); + hfi_handle_arr = enc_output_hfi_handle; + } } if (!hfi_handle_arr || !hfi_handle_size) @@ -693,6 +753,9 @@ static int handle_dpb_list_property(struct iris_inst *inst, u8 *payload_start; u32 payload_size; + if (inst->domain != DECODER) + return -EINVAL; + payload_size = pkt->size - sizeof(*pkt); payload_start = (u8 *)((u8 *)pkt + sizeof(*pkt)); memset(inst->dpb_list_payload, 0, MAX_DPB_LIST_ARRAY_SIZE); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index c0878f1..1a434ff 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -13,6 +13,7 @@ #include "iris_power.h" #include "iris_vb2.h" #include "iris_vdec.h" +#include "iris_venc.h" int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, @@ -58,7 +59,8 @@ int iris_vb2_queue_setup(struct vb2_queue *q, return ret; } - if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + if ((inst->domain == DECODER && q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) || + (inst->domain == ENCODER && q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) { ret = adjust_v4l2_properties(inst); if (ret) return ret; @@ -112,6 +114,11 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) goto error; } + if (inst->domain != DECODER && inst->domain != ENCODER) { + ret = -EINVAL; + goto error; + } + ret = iris_pm_get(inst->core); if (ret) goto error; @@ -122,21 +129,34 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) if (ret) goto err_pm_get; - ret = iris_hfi_session_set_default_header(inst); - if (ret) - goto err_pm_get; - - ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST); - if (ret) - goto err_pm_get; + if (inst->domain == ENCODER) { + ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_ARP); + if (ret) + goto err_pm_get; + } else if (inst->domain == DECODER) { + ret = iris_hfi_session_set_default_header(inst); + if (ret) + goto err_pm_get; + + ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST); + if (ret) + goto err_pm_get; + } } iris_scale_power(inst); - if (q->type == INPUT_MPLANE) - ret = vdec_streamon_input(inst); - else if (q->type == OUTPUT_MPLANE) - ret = vdec_streamon_output(inst); + if (q->type == INPUT_MPLANE) { + if (inst->domain == DECODER) + ret = vdec_streamon_input(inst); + else if (inst->domain == ENCODER) + ret = venc_streamon_input(inst); + } else if (q->type == OUTPUT_MPLANE) { + if (inst->domain == DECODER) + ret = vdec_streamon_output(inst); + else if (inst->domain == ENCODER) + ret = venc_streamon_output(inst); + } if (ret) goto err_pm_get; @@ -179,6 +199,11 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE) goto error; + if (inst->domain != DECODER && inst->domain != ENCODER) { + ret = -EINVAL; + goto error; + } + ret = iris_pm_get_put(inst->core); if (ret) goto error; @@ -225,7 +250,10 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2) if (ret) goto exit; - ret = vdec_qbuf(inst, vb2); + if (inst->domain == DECODER) + ret = vdec_qbuf(inst, vb2); + else if (inst->domain == ENCODER) + ret = venc_qbuf(inst, vb2); exit: if (ret) { @@ -270,7 +298,7 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, buf->inst = inst; buf->dmabuf = dbuf; - if (buf->type == BUF_OUTPUT) { + if (inst->domain == DECODER && buf->type == BUF_OUTPUT) { list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { if (ro_buf->dmabuf != buf->dmabuf) continue; @@ -307,7 +335,7 @@ int iris_vb2_map_dmabuf(void *buf_priv) return -EINVAL; } - if (buf->type == BUF_OUTPUT) { + if (inst->domain == DECODER && buf->type == BUF_OUTPUT) { list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { if (ro_buf->dmabuf != buf->dmabuf) continue; @@ -356,7 +384,7 @@ void iris_vb2_unmap_dmabuf(void *buf_priv) return; } - if (buf->type == BUF_OUTPUT) { + if (inst->domain == DECODER && buf->type == BUF_OUTPUT) { list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { if (ro_buf->dmabuf != buf->dmabuf) continue; @@ -393,7 +421,7 @@ void iris_vb2_detach_dmabuf(void *buf_priv) buf->sg_table = NULL; } - if (buf->type == BUF_OUTPUT) { + if (inst->domain == DECODER && buf->type == BUF_OUTPUT) { list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { if (ro_buf->dmabuf != buf->dmabuf) continue; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 300d0e9..b2d4739 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -362,7 +362,7 @@ static int vdec_set_bitstream_resolution(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_BITSTREAM_RESOLUTION, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32, &resolution, sizeof(u32)); @@ -388,7 +388,7 @@ static int vdec_set_crop_offsets(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_CROP_OFFSETS, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_64_PACKED, &payload, sizeof(u64)); @@ -409,7 +409,7 @@ static int vdec_set_bit_depth(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_LUMA_CHROMA_BIT_DEPTH, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32, &bitdepth, sizeof(u32)); @@ -426,7 +426,7 @@ static int vdec_set_coded_frames(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_CODED_FRAMES, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32, &coded_frames, sizeof(u32)); @@ -442,7 +442,7 @@ static int vdec_set_min_output_count(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32, &min_output, sizeof(u32)); @@ -457,7 +457,7 @@ static int vdec_set_picture_order_count(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_PIC_ORDER_CNT_TYPE, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32, &poc, sizeof(u32)); @@ -509,7 +509,7 @@ static int vdec_set_colorspace(struct iris_inst *inst) ret = iris_hfi_set_property(inst, HFI_PROP_SIGNAL_COLOR_INFO, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_32_PACKED, &color_info, sizeof(u32)); @@ -527,7 +527,7 @@ static int vdec_set_profile(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_PROFILE, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32_ENUM, &profile, sizeof(u32)); @@ -543,7 +543,7 @@ static int vdec_set_level(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_LEVEL, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32_ENUM, &level, sizeof(u32)); @@ -559,7 +559,7 @@ static int vdec_set_tier(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_TIER, HFI_HOST_FLAGS_NONE, - get_hfi_port(INPUT_MPLANE), + get_hfi_port(inst, INPUT_MPLANE), HFI_PAYLOAD_U32_ENUM, &tier, sizeof(u32)); @@ -811,7 +811,7 @@ static int vdec_set_colorformat(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_COLOR_FORMAT, HFI_HOST_FLAGS_NONE, - get_hfi_port(OUTPUT_MPLANE), + get_hfi_port(inst, OUTPUT_MPLANE), HFI_PAYLOAD_U32, &hfi_colorformat, sizeof(u32)); @@ -839,7 +839,7 @@ static int vdec_set_linear_stride_scanline(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_LINEAR_STRIDE_SCANLINE, HFI_HOST_FLAGS_NONE, - get_hfi_port(OUTPUT_MPLANE), + get_hfi_port(inst, OUTPUT_MPLANE), HFI_PAYLOAD_U64, &payload, sizeof(u64)); @@ -887,7 +887,7 @@ static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst) return iris_hfi_set_property(inst, HFI_PROP_UBWC_STRIDE_SCANLINE, HFI_HOST_FLAGS_NONE, - get_hfi_port(OUTPUT_MPLANE), + get_hfi_port(inst, OUTPUT_MPLANE), HFI_PAYLOAD_U32_ARRAY, &payload[0], sizeof(u32) * 4); @@ -1020,7 +1020,7 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst) ret = iris_hfi_set_property(inst, prop_type, HFI_HOST_FLAGS_NONE, - get_hfi_port(OUTPUT_MPLANE), + get_hfi_port(inst, OUTPUT_MPLANE), payload_type, &payload, payload_size); @@ -1032,42 +1032,6 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst) return ret; } -static int process_streamon_input(struct iris_inst *inst) -{ - enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; - int ret; - - iris_scale_power(inst); - - ret = iris_hfi_start(inst, INPUT_MPLANE); - if (ret) - return ret; - - if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { - ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0); - if (ret) - return ret; - } - - if (inst->sub_state & IRIS_INST_SUB_DRC || - inst->sub_state & IRIS_INST_SUB_DRAIN) { - if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) { - ret = iris_hfi_pause(inst, INPUT_MPLANE); - if (ret) - return ret; - set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; - } - } - - ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE); - if (ret) - return ret; - - ret = iris_inst_change_sub_state(inst, 0, set_sub_state); - - return ret; -} - int vdec_streamon_input(struct iris_inst *inst) { int ret; @@ -1114,58 +1078,6 @@ int vdec_streamon_input(struct iris_inst *inst) return ret; } -static int process_streamon_output(struct iris_inst *inst) -{ - enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; - bool drain_pending = false; - int ret; - - iris_scale_power(inst); - - if (inst->sub_state & IRIS_INST_SUB_DRC && - inst->sub_state & IRIS_INST_SUB_DRC_LAST) - clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; - - if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { - ret = iris_alloc_and_queue_input_int_bufs(inst); - if (ret) - return ret; - ret = set_stage(inst, STAGE); - if (ret) - return ret; - ret = set_pipe(inst, PIPE); - if (ret) - return ret; - } - - drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN && - inst->sub_state & IRIS_INST_SUB_DRAIN_LAST; - - if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) { - if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { - ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE); - if (ret) - return ret; - clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; - } - } - - ret = iris_hfi_start(inst, OUTPUT_MPLANE); - if (ret) - return ret; - - if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) - clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; - - ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE); - if (ret) - return ret; - - ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); - - return ret; -} - int vdec_streamon_output(struct iris_inst *inst) { int ret; @@ -1220,27 +1132,6 @@ int vdec_streamon_output(struct iris_inst *inst) return ret; } -static int vb2_buffer_to_driver(struct vb2_buffer *vb2, - struct iris_buffer *buf) -{ - struct vb2_v4l2_buffer *vbuf; - - if (!vb2 || !buf) - return -EINVAL; - - vbuf = to_vb2_v4l2_buffer(vb2); - - buf->fd = vb2->planes[0].m.fd; - buf->data_offset = vb2->planes[0].data_offset; - buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset; - buf->buffer_size = vb2->planes[0].length; - buf->timestamp = vb2->timestamp; - buf->flags = vbuf->flags; - buf->attr = 0; - - return 0; -} - int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) { struct iris_buffer *buf = NULL; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c index 802db40..28c6b20 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c @@ -13,6 +13,23 @@ #include "iris_power.h" #include "iris_venc.h" +#define SCALE_FACTOR 8 +#define UNSPECIFIED_COLOR_FORMAT 5 + +static const u32 enc_input_properties[] = { + HFI_PROP_NO_OUTPUT, +}; + +static const u32 enc_output_properties[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_BUFFER_MARK, +}; + +struct venc_prop_type_handle { + u32 type; + int (*handle)(struct iris_inst *inst); +}; + int venc_inst_init(struct iris_inst *inst) { struct v4l2_format *f; @@ -523,3 +540,409 @@ int venc_stop_cmd(struct iris_inst *inst) return ret; } + +int venc_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) +{ + struct iris_buffer *buf = NULL; + int ret; + + buf = get_driver_buf(inst, vb2->type, vb2->index); + if (!buf) + return -EINVAL; + + ret = vb2_buffer_to_driver(vb2, buf); + if (ret) + return ret; + + if (!allow_qbuf(inst, vb2->type)) { + buf->attr |= BUF_ATTR_DEFERRED; + return 0; + } + + iris_scale_power(inst); + + return queue_buffer(inst, buf); +} + +static int check_scaling_supported(struct iris_inst *inst) +{ + u32 iwidth, owidth, iheight, oheight; + + if (!(inst->crop.left != inst->compose.left || + inst->crop.top != inst->compose.top || + inst->crop.width != inst->compose.width || + inst->crop.height != inst->compose.height)) + return 0; + + iwidth = inst->crop.width; + iheight = inst->crop.height; + owidth = inst->compose.width; + oheight = inst->compose.height; + + if (owidth > iwidth || oheight > iheight) + return -EINVAL; + + if (iwidth > owidth * SCALE_FACTOR || iheight > oheight * SCALE_FACTOR) + return -EINVAL; + + return 0; +} + +static int venc_set_colorformat(struct iris_inst *inst) +{ + u32 hfi_colorformat; + u32 pixelformat; + + pixelformat = inst->fmt_src->fmt.pix_mp.pixelformat; + hfi_colorformat = get_hfi_colorformat(pixelformat); + + return iris_hfi_set_property(inst, + HFI_PROP_COLOR_FORMAT, + HFI_HOST_FLAGS_NONE, + get_hfi_port(inst, INPUT_MPLANE), + HFI_PAYLOAD_U32_ENUM, + &hfi_colorformat, + sizeof(u32)); +} + +static int venc_set_stride_scanline(struct iris_inst *inst) +{ + u32 color_format, stride_y, scanline_y; + u32 stride_uv = 0, scanline_uv = 0; + u32 payload[2]; + + color_format = inst->cap[PIX_FMTS].value; + if (!is_linear_colorformat(color_format)) + return 0; + + stride_y = color_format == FMT_TP10C ? + ALIGN(inst->fmt_src->fmt.pix_mp.width, 192) : + ALIGN(inst->fmt_src->fmt.pix_mp.width, 128); + scanline_y = color_format == FMT_TP10C ? + ALIGN(inst->fmt_src->fmt.pix_mp.height, 16) : + ALIGN(inst->fmt_src->fmt.pix_mp.height, 32); + + if (color_format == FMT_NV12 || + color_format == FMT_NV21) { + stride_uv = stride_y; + scanline_uv = scanline_y / 2; + } + + payload[0] = stride_y << 16 | scanline_y; + payload[1] = stride_uv << 16 | scanline_uv; + + return iris_hfi_set_property(inst, + HFI_PROP_LINEAR_STRIDE_SCANLINE, + HFI_HOST_FLAGS_NONE, + get_hfi_port(inst, INPUT_MPLANE), + HFI_PAYLOAD_64_PACKED, + &payload, + sizeof(u64)); +} + +static int venc_set_raw_resolution(struct iris_inst *inst) +{ + u32 resolution; + + resolution = (inst->fmt_src->fmt.pix_mp.width << 16) | + inst->fmt_src->fmt.pix_mp.height; + + return iris_hfi_set_property(inst, + HFI_PROP_RAW_RESOLUTION, + HFI_HOST_FLAGS_NONE, + get_hfi_port(inst, INPUT_MPLANE), + HFI_PAYLOAD_32_PACKED, + &resolution, + sizeof(u32)); +} + +static int venc_set_bitstream_resolution(struct iris_inst *inst) +{ + u32 resolution; + + resolution = (inst->fmt_dst->fmt.pix_mp.width << 16) | + inst->fmt_dst->fmt.pix_mp.height; + + return iris_hfi_set_property(inst, + HFI_PROP_BITSTREAM_RESOLUTION, + HFI_HOST_FLAGS_NONE, + get_hfi_port(inst, OUTPUT_MPLANE), + HFI_PAYLOAD_32_PACKED, + &resolution, + sizeof(u32)); +} + +static int venc_set_inp_crop_offsets(struct iris_inst *inst) +{ + u32 left_offset, top_offset, right_offset, bottom_offset; + u32 crop[2] = {0}; + u32 width, height; + + left_offset = inst->crop.left; + top_offset = inst->crop.top; + width = inst->crop.width; + height = inst->crop.height; + + right_offset = (inst->fmt_src->fmt.pix_mp.width - width); + bottom_offset = (inst->fmt_src->fmt.pix_mp.height - height); + + crop[0] = left_offset << 16 | top_offset; + crop[1] = right_offset << 16 | bottom_offset; + + return iris_hfi_set_property(inst, + HFI_PROP_CROP_OFFSETS, + HFI_HOST_FLAGS_NONE, + get_hfi_port(inst, INPUT_MPLANE), + HFI_PAYLOAD_64_PACKED, + &crop, + sizeof(u64)); +} + +static int venc_set_out_crop_offsets(struct iris_inst *inst) +{ + u32 left_offset, top_offset, right_offset, bottom_offset; + u32 crop[2] = {0}; + u32 width, height; + + left_offset = inst->compose.left; + top_offset = inst->compose.top; + width = inst->compose.width; + height = inst->compose.height; + if (inst->cap[ROTATION].value == 90 || inst->cap[ROTATION].value == 270) { + width = inst->compose.height; + height = inst->compose.width; + } + + right_offset = (inst->fmt_dst->fmt.pix_mp.width - width); + bottom_offset = (inst->fmt_dst->fmt.pix_mp.height - height); + + crop[0] = left_offset << 16 | top_offset; + crop[1] = right_offset << 16 | bottom_offset; + + return iris_hfi_set_property(inst, + HFI_PROP_CROP_OFFSETS, + HFI_HOST_FLAGS_NONE, + get_hfi_port(inst, OUTPUT_MPLANE), + HFI_PAYLOAD_64_PACKED, + &crop, + sizeof(u64)); +} + +static int venc_set_colorspace(struct iris_inst *inst) +{ + u32 video_signal_type_present_flag = 0, payload = 0; + u32 matrix_coeff = HFI_MATRIX_COEFF_RESERVED; + u32 video_format = UNSPECIFIED_COLOR_FORMAT; + struct v4l2_pix_format_mplane *pixmp = NULL; + u32 transfer_char = HFI_TRANSFER_RESERVED; + u32 colour_description_present_flag = 0; + u32 primaries = HFI_PRIMARIES_RESERVED; + u32 full_range = 0; + + pixmp = &inst->fmt_src->fmt.pix_mp; + if (pixmp->colorspace != V4L2_COLORSPACE_DEFAULT || + pixmp->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT || + pixmp->xfer_func != V4L2_XFER_FUNC_DEFAULT) { + colour_description_present_flag = 1; + video_signal_type_present_flag = 1; + primaries = get_hfi_color_primaries(pixmp->colorspace); + matrix_coeff = get_hfi_matrix_coefficients(pixmp->ycbcr_enc); + transfer_char = get_hfi_transfer_char(pixmp->xfer_func); + } + + if (pixmp->quantization != V4L2_QUANTIZATION_DEFAULT) { + video_signal_type_present_flag = 1; + full_range = pixmp->quantization == + V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0; + } + + payload = (matrix_coeff & 0xFF) | + ((transfer_char << 8) & 0xFF00) | + ((primaries << 16) & 0xFF0000) | + ((colour_description_present_flag << 24) & 0x1000000) | + ((full_range << 25) & 0x2000000) | + ((video_format << 26) & 0x1C000000) | + ((video_signal_type_present_flag << 29) & 0x20000000); + + return iris_hfi_set_property(inst, + HFI_PROP_SIGNAL_COLOR_INFO, + HFI_HOST_FLAGS_NONE, + get_hfi_port(inst, INPUT_MPLANE), + HFI_PAYLOAD_32_PACKED, + &payload, + sizeof(u32)); +} + +static int venc_set_quality_mode(struct iris_inst *inst) +{ + u32 mode; + + mode = decide_quality_mode(inst); + return iris_hfi_set_property(inst, + HFI_PROP_QUALITY_MODE, + HFI_HOST_FLAGS_NONE, + HFI_PORT_BITSTREAM, + HFI_PAYLOAD_U32_ENUM, + &mode, + sizeof(u32)); +} + +static int venc_set_input_properties(struct iris_inst *inst) +{ + int j, ret; + static const struct venc_prop_type_handle prop_type_handle_arr[] = { + {HFI_PROP_COLOR_FORMAT, venc_set_colorformat }, + {HFI_PROP_RAW_RESOLUTION, venc_set_raw_resolution }, + {HFI_PROP_CROP_OFFSETS, venc_set_inp_crop_offsets }, + {HFI_PROP_LINEAR_STRIDE_SCANLINE, venc_set_stride_scanline }, + {HFI_PROP_SIGNAL_COLOR_INFO, venc_set_colorspace }, + }; + + for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) { + ret = prop_type_handle_arr[j].handle(inst); + if (ret) + return ret; + } + + return ret; +} + +static int venc_property_subscription(struct iris_inst *inst, u32 plane) +{ + u32 payload[32] = {0}; + u32 payload_size = 0; + u32 i; + + payload[0] = HFI_MODE_PROPERTY; + if (plane == INPUT_MPLANE) { + for (i = 0; i < ARRAY_SIZE(enc_input_properties); i++) + payload[i + 1] = enc_input_properties[i]; + payload_size = (ARRAY_SIZE(enc_input_properties) + 1) * + sizeof(u32); + } else if (plane == OUTPUT_MPLANE) { + for (i = 0; i < ARRAY_SIZE(enc_output_properties); i++) + payload[i + 1] = enc_output_properties[i]; + payload_size = (ARRAY_SIZE(enc_output_properties) + 1) * + sizeof(u32); + } else { + return -EINVAL; + } + + return iris_hfi_session_subscribe_mode(inst, + HFI_CMD_SUBSCRIBE_MODE, + plane, + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + payload_size); +} + +int venc_streamon_input(struct iris_inst *inst) +{ + int ret; + + ret = check_session_supported(inst); + if (ret) + goto error; + + ret = check_scaling_supported(inst); + if (ret) + goto error; + + ret = venc_set_input_properties(inst); + if (ret) + goto error; + + ret = iris_get_internal_buffers(inst, INPUT_MPLANE); + if (ret) + goto error; + + ret = iris_destroy_internal_buffers(inst, INPUT_MPLANE); + if (ret) + goto error; + + ret = iris_create_input_internal_buffers(inst); + if (ret) + goto error; + + ret = iris_queue_input_internal_buffers(inst); + if (ret) + goto error; + + ret = venc_property_subscription(inst, INPUT_MPLANE); + if (ret) + goto error; + + ret = process_streamon_input(inst); + if (ret) + goto error; + + return ret; + +error: + return ret; +} + +static int venc_set_output_properties(struct iris_inst *inst) +{ + int j, ret; + static const struct venc_prop_type_handle prop_type_handle_arr[] = { + {HFI_PROP_BITSTREAM_RESOLUTION, venc_set_bitstream_resolution }, + {HFI_PROP_CROP_OFFSETS, venc_set_out_crop_offsets }, + }; + + for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) { + ret = prop_type_handle_arr[j].handle(inst); + if (ret) + return ret; + } + + return ret; +} + +int venc_streamon_output(struct iris_inst *inst) +{ + int ret; + + ret = venc_set_output_properties(inst); + if (ret) + goto error; + + ret = set_v4l2_properties(inst); + if (ret) + goto error; + + ret = venc_set_quality_mode(inst); + if (ret) + goto error; + + ret = iris_get_internal_buffers(inst, OUTPUT_MPLANE); + if (ret) + goto error; + + ret = iris_destroy_internal_buffers(inst, OUTPUT_MPLANE); + if (ret) + goto error; + + ret = iris_create_output_internal_buffers(inst); + if (ret) + goto error; + + ret = iris_queue_output_internal_buffers(inst); + if (ret) + goto error; + + ret = venc_property_subscription(inst, OUTPUT_MPLANE); + if (ret) + goto error; + + ret = process_streamon_output(inst); + if (ret) + goto error; + + return ret; + +error: + session_streamoff(inst, OUTPUT_MPLANE); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h index 24da63f..b289750 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h @@ -20,5 +20,8 @@ int venc_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); int venc_start_cmd(struct iris_inst *inst); int venc_stop_cmd(struct iris_inst *inst); +int venc_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2); +int venc_streamon_input(struct iris_inst *inst); +int venc_streamon_output(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index effecbb..42ac662 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -71,6 +71,11 @@ enum pipe_type { PIPE_4 = 4, }; +enum quality_mode { + MAX_QUALITY_MODE = 0x1, + POWER_SAVE_MODE = 0x2, +}; + extern struct platform_data sm8550_data; struct bw_info { diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index ef0aad7..17c5856 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -74,6 +74,8 @@ static struct plat_core_cap core_data_sm8550[] = { {CP_SIZE, 0x25800000}, {CP_NONPIXEL_START, 0x01000000}, {CP_NONPIXEL_SIZE, 0x24800000}, + {MAX_MBPF_HQ, 8160}, /* ((1920x1088)/256) */ + {MAX_MBPS_HQ, 489600}, /* ((1920x1088)/256)@60fps */ }; static struct plat_inst_cap instance_cap_data_sm8550[] = { diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c index 44f9342..9882f00 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c @@ -163,6 +163,186 @@ static u32 dec_dpb_size_iris3(struct iris_inst *inst) return size; } +static u32 enc_bin_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_vpp_pipes, stage, profile; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + + core = inst->core; + + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + stage = inst->cap[STAGE].value; + f = inst->fmt_dst; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + profile = inst->cap[PROFILE].value; + + if (inst->codec == H264) + size = hfi_buffer_bin_h264e(inst->hfi_rc_type, width, height, + stage, num_vpp_pipes, profile); + else if (inst->codec == HEVC) + size = hfi_buffer_bin_h265e(inst->hfi_rc_type, width, height, + stage, num_vpp_pipes, profile); + + return size; +} + +u32 get_recon_buf_count(struct iris_inst *inst) +{ + s32 n_bframe, ltr_count, hp_layers = 0, hb_layers = 0; + bool is_hybrid_hp = false; + u32 num_buf_recon = 0; + u32 hfi_codec = 0; + + n_bframe = inst->cap[B_FRAME].value; + ltr_count = inst->cap[LTR_COUNT].value; + + if (inst->hfi_layer_type == HFI_HIER_B) { + hb_layers = inst->cap[ENH_LAYER_COUNT].value + 1; + } else { + hp_layers = inst->cap[ENH_LAYER_COUNT].value + 1; + if (inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR) + is_hybrid_hp = true; + } + + if (inst->codec == H264) + hfi_codec = HFI_CODEC_ENCODE_AVC; + else if (inst->codec == HEVC) + hfi_codec = HFI_CODEC_ENCODE_HEVC; + + num_buf_recon = hfi_iris3_enc_recon_buf_count(n_bframe, ltr_count, + hp_layers, hb_layers, + is_hybrid_hp, hfi_codec); + + return num_buf_recon; +} + +static u32 enc_comv_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_recon = 0; + struct v4l2_format *f; + u32 size = 0; + + f = inst->fmt_dst; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + num_recon = get_recon_buf_count(inst); + if (inst->codec == H264) + size = hfi_buffer_comv_h264e(width, height, num_recon); + else if (inst->codec == HEVC) + size = hfi_buffer_comv_h265e(width, height, num_recon); + + return size; +} + +static u32 enc_non_comv_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_vpp_pipes; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + + core = inst->core; + + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + f = inst->fmt_dst; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (inst->codec == H264) + size = hfi_buffer_non_comv_h264e(width, height, num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_non_comv_h265e(width, height, num_vpp_pipes); + + return size; +} + +static u32 enc_line_size_iris3(struct iris_inst *inst) +{ + u32 width, height, pixfmt, num_vpp_pipes; + struct iris_core *core; + bool is_tenbit = false; + struct v4l2_format *f; + u32 size = 0; + + core = inst->core; + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + pixfmt = inst->cap[PIX_FMTS].value; + + f = inst->fmt_dst; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + is_tenbit = (pixfmt == FMT_TP10C); + + if (inst->codec == H264) + size = hfi_buffer_line_h264e(width, height, is_tenbit, num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_line_h265e(width, height, is_tenbit, num_vpp_pipes); + + return size; +} + +static u32 enc_dpb_size_iris3(struct iris_inst *inst) +{ + u32 width, height, pixfmt; + struct v4l2_format *f; + bool is_tenbit; + u32 size = 0; + + f = inst->fmt_dst; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + pixfmt = inst->cap[PIX_FMTS].value; + is_tenbit = (pixfmt == FMT_TP10C); + + if (inst->codec == H264) + size = hfi_buffer_dpb_h264e(width, height); + else if (inst->codec == HEVC) + size = hfi_buffer_dpb_h265e(width, height, is_tenbit); + + return size; +} + +static u32 enc_arp_size_iris3(struct iris_inst *inst) +{ + u32 size = 0; + + HFI_BUFFER_ARP_ENC(size); + + return size; +} + +static u32 enc_vpss_size_iris3(struct iris_inst *inst) +{ + bool ds_enable = false, is_tenbit = false; + struct v4l2_format *f; + u32 width, height; + u32 size = 0; + + ds_enable = is_scaling_enabled(inst); + + f = inst->fmt_dst; + if (inst->cap[ROTATION].value == 90 || + inst->cap[ROTATION].value == 270) { + width = f->fmt.pix_mp.height; + height = f->fmt.pix_mp.width; + } else { + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + } + + f = inst->fmt_src; + is_tenbit = is_10bit_colorformat(f->fmt.pix_mp.pixelformat); + + size = hfi_buffer_vpss_enc(width, height, ds_enable, 0, is_tenbit); + + return size; +} + struct iris_buf_type_handle { enum iris_buffer_type type; u32 (*handle)(struct iris_inst *inst); @@ -183,9 +363,23 @@ int iris_int_buf_size_iris3(struct iris_inst *inst, {BUF_PERSIST, dec_persist_size_iris3 }, {BUF_DPB, dec_dpb_size_iris3 }, }; + static const struct iris_buf_type_handle enc_internal_buf_type_handle[] = { + {BUF_BIN, enc_bin_size_iris3 }, + {BUF_COMV, enc_comv_size_iris3 }, + {BUF_NON_COMV, enc_non_comv_size_iris3 }, + {BUF_LINE, enc_line_size_iris3 }, + {BUF_DPB, enc_dpb_size_iris3 }, + {BUF_ARP, enc_arp_size_iris3 }, + {BUF_VPSS, enc_vpss_size_iris3 }, + }; - buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle); - buf_type_handle_arr = dec_internal_buf_type_handle; + if (inst->domain == DECODER) { + buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle); + buf_type_handle_arr = dec_internal_buf_type_handle; + } else if (inst->domain == ENCODER) { + buf_type_handle_size = ARRAY_SIZE(enc_internal_buf_type_handle); + buf_type_handle_arr = enc_internal_buf_type_handle; + } if (!buf_type_handle_arr || !buf_type_handle_size) return size; diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h index b520c79..c63b3ac 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h @@ -14,8 +14,10 @@ #define DMA_ALIGNMENT 256 +#define BUFFER_ALIGNMENT_4096_BYTES 4096 #define BUFFER_ALIGNMENT_512_BYTES 512 #define BUFFER_ALIGNMENT_256_BYTES 256 +#define BUFFER_ALIGNMENT_128_BYTES 128 #define BUFFER_ALIGNMENT_64_BYTES 64 #define BUFFER_ALIGNMENT_32_BYTES 32 #define BUFFER_ALIGNMENT_16_BYTES 16 @@ -26,6 +28,8 @@ #define HFI_COL_FMT_NV12C_Y_TILE_WIDTH (32) #define HFI_COL_FMT_NV12C_UV_TILE_HEIGHT (8) #define HFI_COL_FMT_NV12C_UV_TILE_WIDTH (16) +#define HFI_COL_FMT_TP10C_Y_TILE_HEIGHT (4) +#define HFI_COL_FMT_TP10C_Y_TILE_WIDTH (48) #define NUM_HW_PIC_BUF 32 #define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf)) @@ -102,6 +106,27 @@ #define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE (64 * 2 * 3) #define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640 +#define HFI_BUFFER_ARP_ENC(size) (size = 204800) + +#define HFI_MAX_COL_FRAME 6 +#define HFI_VENUS_WIDTH_ALIGNMENT 128 +#define HFI_VENUS_HEIGHT_ALIGNMENT 32 +#define VENUS_METADATA_STRIDE_MULTIPLE 64 +#define VENUS_METADATA_HEIGHT_MULTIPLE 16 + +#ifndef SYSTEM_LAL_TILE10 +#define SYSTEM_LAL_TILE10 192 +#endif + +#define SIZE_SLICE_CMD_BUFFER (ALIGN(20480, BUFFER_ALIGNMENT_256_BYTES)) +#define SIZE_SPS_PPS_SLICE_HDR (2048 + 4096) + +#define SIZE_BSE_SLICE_CMD_BUF ((((8192 << 2) + 7) & (~7)) * 3) +#define SIZE_LAMBDA_LUT (256 * 11) + +#define HFI_WORKMODE_1 1 +#define HFI_WORKMODE_2 2 + static inline u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height, u32 num_vpp_pipes) @@ -842,4 +867,603 @@ u32 hfi_yuv420_tp10_ubwc_calc_buf_size(u32 y_stride, u32 y_buf_height, int iris_int_buf_size_iris3(struct iris_inst *inst, enum iris_buffer_type buffer_type); +static inline +u32 hfi_iris3_enc_recon_buf_count(s32 n_bframe, s32 ltr_count, + s32 _total_hp_layers, + s32 _total_hb_layers, + bool hybrid_hp, + u32 codec_standard) +{ + u32 num_ref = 1; + + if (n_bframe) + num_ref = 2; + + if (_total_hp_layers > 1) { + if (hybrid_hp) + num_ref = (_total_hp_layers + 1) >> 1; + else if (codec_standard == HFI_CODEC_ENCODE_HEVC) + num_ref = (_total_hp_layers + 1) >> 1; + else if (codec_standard == HFI_CODEC_ENCODE_AVC && + _total_hp_layers < 4) + num_ref = (_total_hp_layers - 1); + else + num_ref = _total_hp_layers; + } + + if (ltr_count) + num_ref = num_ref + ltr_count; + + if (_total_hb_layers > 1) { + if (codec_standard == HFI_CODEC_ENCODE_HEVC) + num_ref = (_total_hb_layers); + else if (codec_standard == HFI_CODEC_ENCODE_AVC) + num_ref = (1 << (_total_hb_layers - 2)) + 1; + } + + return num_ref + 1; +} + +static inline +u32 size_bin_bitstream_enc(u32 rc_type, u32 frame_width, u32 frame_height, + u32 work_mode, u32 lcu_size, u32 profile) +{ + u32 size_aligned_width = 0, size_aligned_height = 0; + u32 bitstream_size_eval = 0; + + size_aligned_width = ALIGN((frame_width), lcu_size); + size_aligned_height = ALIGN((frame_height), lcu_size); + if (work_mode == HFI_WORKMODE_2) { + if (rc_type == HFI_RC_CQ || rc_type == HFI_RC_OFF) { + bitstream_size_eval = (((size_aligned_width) * + (size_aligned_height) * 3) >> 1); + } else { + bitstream_size_eval = ((size_aligned_width) * + (size_aligned_height) * 3); + if ((size_aligned_width * size_aligned_height) > (4096 * 2176)) + bitstream_size_eval >>= 3; + else if ((size_aligned_width * size_aligned_height) > (480 * 320)) + bitstream_size_eval >>= 2; + + if (profile == HFI_H265_PROFILE_MAIN_10 || + profile == HFI_H265_PROFILE_MAIN_10_STILL_PICTURE) + bitstream_size_eval = (bitstream_size_eval * 5 >> 2); + } + } else { + bitstream_size_eval = size_aligned_width * size_aligned_height * 3; + } + + return ALIGN(bitstream_size_eval, BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_enc_single_pipe(u32 rc_type, u32 bitbin_size, u32 num_vpp_pipes, + u32 frame_width, u32 frame_height, u32 lcu_size) +{ + u32 size_single_pipe_eval = 0, sao_bin_buffer_size = 0; + u32 size_aligned_width = 0, size_aligned_height = 0; + u32 _padded_bin_sz = 0; + + size_aligned_width = ALIGN((frame_width), lcu_size); + size_aligned_height = ALIGN((frame_height), lcu_size); + if ((size_aligned_width * size_aligned_height) > (3840 * 2160)) + size_single_pipe_eval = (bitbin_size / num_vpp_pipes); + else if (num_vpp_pipes > 2) + size_single_pipe_eval = bitbin_size / 2; + else + size_single_pipe_eval = bitbin_size; + + sao_bin_buffer_size = + (64 * ((((frame_width) + BUFFER_ALIGNMENT_32_BYTES) * + ((frame_height) + BUFFER_ALIGNMENT_32_BYTES)) >> 10)) + 384; + _padded_bin_sz = ALIGN(size_single_pipe_eval, BUFFER_ALIGNMENT_256_BYTES); + size_single_pipe_eval = sao_bin_buffer_size + _padded_bin_sz; + size_single_pipe_eval = ALIGN(size_single_pipe_eval, BUFFER_ALIGNMENT_256_BYTES); + + return size_single_pipe_eval; +} + +static inline +u32 hfi_buffer_bin_enc(u32 rc_type, u32 frame_width, u32 frame_height, u32 lcu_size, + u32 work_mode, u32 num_vpp_pipes, u32 profile) +{ + u32 bitstream_size = 0, total_bitbin_buffers = 0; + u32 size_single_pipe = 0, bitbin_size = 0; + u32 _size = 0; + + bitstream_size = size_bin_bitstream_enc(rc_type, frame_width, + frame_height, work_mode, + lcu_size, profile); + + if (work_mode == HFI_WORKMODE_2) { + total_bitbin_buffers = 3; + bitbin_size = bitstream_size * 12 / 10; + bitbin_size = ALIGN(bitbin_size, BUFFER_ALIGNMENT_256_BYTES); + } else if ((lcu_size == 16) || (num_vpp_pipes > 1)) { + total_bitbin_buffers = 1; + bitbin_size = bitstream_size; + } + + if (total_bitbin_buffers > 0) { + size_single_pipe = size_enc_single_pipe(rc_type, bitbin_size, + num_vpp_pipes, frame_width, + frame_height, lcu_size); + bitbin_size = size_single_pipe * num_vpp_pipes; + _size = ALIGN(bitbin_size, BUFFER_ALIGNMENT_256_BYTES) * + total_bitbin_buffers + 512; + } else { + /* Avoid 512 Bytes allocation in case of 1Pipe HEVC Direct Mode*/ + _size = 0; + } + + return _size; +} + +static inline +u32 hfi_buffer_bin_h264e(u32 rc_type, u32 frame_width, u32 frame_height, + u32 work_mode, u32 num_vpp_pipes, u32 profile) +{ + return hfi_buffer_bin_enc(rc_type, frame_width, frame_height, 16, + work_mode, num_vpp_pipes, profile); +} + +static inline +u32 hfi_buffer_bin_h265e(u32 rc_type, u32 frame_width, u32 frame_height, + u32 work_mode, u32 num_vpp_pipes, u32 profile) +{ + return hfi_buffer_bin_enc(rc_type, frame_width, frame_height, 32, + work_mode, num_vpp_pipes, profile); +} + +static inline +u32 size_enc_slice_info_buf(u32 num_lcu_in_frame) +{ + return ALIGN((256 + (num_lcu_in_frame << 4)), BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_line_buf_ctrl(u32 frame_width_coded) +{ + return ALIGN(frame_width_coded, BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_line_buf_ctrl_id2(u32 frame_width_coded) +{ + return ALIGN(frame_width_coded, BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_linebuff_data(bool is_ten_bit, u32 frame_width_coded) +{ + u32 _size = 0; + + _size = is_ten_bit ? + (((((10 * (frame_width_coded) + 1024) + (BUFFER_ALIGNMENT_256_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 1) + + (((((10 * (frame_width_coded) + 1024) >> 1) + (BUFFER_ALIGNMENT_256_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 2)) : + (((((8 * (frame_width_coded) + 1024) + (BUFFER_ALIGNMENT_256_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 1) + + (((((8 * (frame_width_coded) + 1024) >> 1) + (BUFFER_ALIGNMENT_256_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 2)); + + return _size; +} + +static inline +u32 size_left_linebuff_ctrl(u32 standard, u32 frame_height_coded, + u32 num_vpp_pipes_enc) +{ + u32 _size = 0; + + _size = standard == HFI_CODEC_ENCODE_HEVC ? + (((frame_height_coded) + + (BUFFER_ALIGNMENT_32_BYTES)) / BUFFER_ALIGNMENT_32_BYTES * 4 * 16) : + (((frame_height_coded) + 15) / 16 * 5 * 16); + + if ((num_vpp_pipes_enc) > 1) { + _size += BUFFER_ALIGNMENT_512_BYTES; + _size = ALIGN(_size, BUFFER_ALIGNMENT_512_BYTES) * + num_vpp_pipes_enc; + } + + return ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_left_linebuff_recon_pix(bool is_ten_bit, u32 frame_height_coded, + u32 num_vpp_pipes_enc) +{ + return (((is_ten_bit + 1) * 2 * (frame_height_coded) + BUFFER_ALIGNMENT_256_BYTES) + + (BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1) & + (~((BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1)) * 1; +} + +static inline +u32 size_top_linebuff_ctrl_fe(u32 frame_width_coded, u32 standard) +{ + u32 _size = 0; + + _size = standard == HFI_CODEC_ENCODE_HEVC ? + (64 * ((frame_width_coded) >> 5)) : + (BUFFER_ALIGNMENT_256_BYTES + 16 * ((frame_width_coded) >> 4)); + + return ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_left_linebuff_ctrl_fe(u32 frame_height_coded, u32 num_vpp_pipes_enc) +{ + return (((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) >> 4)) + + (BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1) & + (~((BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1)) * 1) * + num_vpp_pipes_enc; +} + +static inline +u32 size_left_linebuff_metadata_recon_y(u32 frame_height_coded, + bool is_ten_bit, + u32 num_vpp_pipes_enc) +{ + u32 _size = 0; + + _size = ((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) / + (8 * (is_ten_bit ? 4 : 8))))); + _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES); + + return (_size * num_vpp_pipes_enc); +} + +static inline +u32 size_left_linebuff_metadata_recon_uv(u32 frame_height_coded, + bool is_ten_bit, + u32 num_vpp_pipes_enc) +{ + u32 _size = 0; + + _size = ((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) / + (4 * (is_ten_bit ? 4 : 8))))); + _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES); + + return (_size * num_vpp_pipes_enc); +} + +static inline +u32 size_linebuff_recon_pix(bool is_ten_bit, u32 frame_width_coded) +{ + return ALIGN(((is_ten_bit ? 3 : 2) * (frame_width_coded)), + BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_frame_rc_buf_size(u32 standard, u32 frame_height_coded, + u32 num_vpp_pipes_enc) +{ + u32 _size = 0; + + _size = (standard == HFI_CODEC_ENCODE_HEVC) ? + (256 + 16 * (14 + ((((frame_height_coded) >> 5) + 7) >> 3))) : + (256 + 16 * (14 + ((((frame_height_coded) >> 4) + 7) >> 3))); + _size *= 11; + if (num_vpp_pipes_enc > 1) + _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES) * num_vpp_pipes_enc; + + return ALIGN(_size, BUFFER_ALIGNMENT_512_BYTES) * HFI_MAX_COL_FRAME; +} + +static inline u32 enc_bitcnt_buf_size(u32 num_lcu_in_frame) +{ + return ALIGN((256 + (4 * (num_lcu_in_frame))), BUFFER_ALIGNMENT_256_BYTES); +} + +static inline u32 enc_bitmap_buf_size(u32 num_lcu_in_frame) +{ + return ALIGN((256 + ((num_lcu_in_frame) >> 3)), BUFFER_ALIGNMENT_256_BYTES); +} + +static inline u32 size_line_buf_sde(u32 frame_width_coded) +{ + return ALIGN((256 + (16 * ((frame_width_coded) >> 4))), BUFFER_ALIGNMENT_256_BYTES); +} + +static inline u32 size_override_buf(u32 num_lcumb) +{ + return ALIGN(((16 * (((num_lcumb) + 7) >> 3))), BUFFER_ALIGNMENT_256_BYTES) * 2; +} + +static inline u32 size_ir_buf(u32 num_lcu_in_frame) +{ + return ALIGN((((((num_lcu_in_frame) << 1) + 7) & (~7)) * 3), BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_vpss_line_buf(u32 num_vpp_pipes_enc, u32 frame_height_coded, + u32 frame_width_coded) +{ + return ALIGN(((((((8192) >> 2) << 5) * (num_vpp_pipes_enc)) + 64) + + (((((max_t(u32, (frame_width_coded), + (frame_height_coded)) + 3) >> 2) << 5) + 256) * 16)), + BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 size_top_line_buf_first_stg_sao(u32 frame_width_coded) +{ + return ALIGN((16 * ((frame_width_coded) >> 5)), BUFFER_ALIGNMENT_256_BYTES); +} + +static inline +u32 hfi_buffer_line_enc(u32 frame_width, u32 frame_height, bool is_ten_bit, + u32 num_vpp_pipes_enc, u32 lcu_size, u32 standard) +{ + u32 line_buff_data_size = 0, left_line_buff_ctrl_size = 0; + u32 frame_width_coded = 0, frame_height_coded = 0; + u32 left_line_buff_metadata_recon__uv__size = 0; + u32 left_line_buff_metadata_recon__y__size = 0; + u32 width_in_lcus = 0, height_in_lcus = 0; + u32 left_line_buff_recon_pix_size = 0; + u32 top_line_buff_ctrl_fe_size = 0; + u32 line_buff_recon_pix_size = 0; + + width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size); + height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size); + frame_width_coded = width_in_lcus * (lcu_size); + frame_height_coded = height_in_lcus * (lcu_size); + line_buff_data_size = size_linebuff_data(is_ten_bit, frame_width_coded); + left_line_buff_ctrl_size = + size_left_linebuff_ctrl(standard, frame_height_coded, num_vpp_pipes_enc); + left_line_buff_recon_pix_size = + size_left_linebuff_recon_pix(is_ten_bit, frame_height_coded, + num_vpp_pipes_enc); + top_line_buff_ctrl_fe_size = + size_top_linebuff_ctrl_fe(frame_width_coded, standard); + left_line_buff_metadata_recon__y__size = + size_left_linebuff_metadata_recon_y(frame_height_coded, is_ten_bit, + num_vpp_pipes_enc); + left_line_buff_metadata_recon__uv__size = + size_left_linebuff_metadata_recon_uv(frame_height_coded, is_ten_bit, + num_vpp_pipes_enc); + line_buff_recon_pix_size = size_linebuff_recon_pix(is_ten_bit, frame_width_coded); + + return size_line_buf_ctrl(frame_width_coded) + + size_line_buf_ctrl_id2(frame_width_coded) + + line_buff_data_size + + left_line_buff_ctrl_size + + left_line_buff_recon_pix_size + + top_line_buff_ctrl_fe_size + + left_line_buff_metadata_recon__y__size + + left_line_buff_metadata_recon__uv__size + + line_buff_recon_pix_size + + size_left_linebuff_ctrl_fe(frame_height_coded, + num_vpp_pipes_enc) + + size_line_buf_sde(frame_width_coded) + + size_vpss_line_buf(num_vpp_pipes_enc, frame_height_coded, + frame_width_coded) + + size_top_line_buf_first_stg_sao(frame_width_coded); +} + +static inline +u32 hfi_buffer_line_h264e(u32 frame_width, u32 frame_height, bool is_ten_bit, + u32 num_vpp_pipes) +{ + return hfi_buffer_line_enc(frame_width, frame_height, 0, + num_vpp_pipes, 16, + HFI_CODEC_ENCODE_AVC); +} + +static inline +u32 hfi_buffer_line_h265e(u32 frame_width, u32 frame_height, bool is_ten_bit, + u32 num_vpp_pipes) +{ + return hfi_buffer_line_enc(frame_width, frame_height, + is_ten_bit, num_vpp_pipes, 32, + HFI_CODEC_ENCODE_HEVC); +} + +static inline +u32 hfi_buffer_comv_enc(u32 frame_width, u32 frame_height, u32 lcu_size, + u32 num_recon, u32 standard) +{ + u32 width_in_lcus, height_in_lcus, num_lcu_in_frame; + u32 size_colloc_mv = 0, size_colloc_rc = 0; + u32 mb_height = ((frame_height) + 15) >> 4; + u32 mb_width = ((frame_width) + 15) >> 4; + + width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size); + height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size); + num_lcu_in_frame = width_in_lcus * height_in_lcus; + size_colloc_mv = (standard == HFI_CODEC_ENCODE_HEVC) ? + (16 * ((num_lcu_in_frame << 2) + BUFFER_ALIGNMENT_32_BYTES)) : + (3 * 16 * (width_in_lcus * height_in_lcus + BUFFER_ALIGNMENT_32_BYTES)); + size_colloc_mv = ALIGN(size_colloc_mv, BUFFER_ALIGNMENT_256_BYTES) * num_recon; + size_colloc_rc = (((mb_width + 7) >> 3) * 16 * 2 * mb_height); + size_colloc_rc = ALIGN(size_colloc_rc, BUFFER_ALIGNMENT_256_BYTES) * HFI_MAX_COL_FRAME; + + return size_colloc_mv + size_colloc_rc; +} + +static inline +u32 hfi_buffer_comv_h264e(u32 frame_width, u32 frame_height, u32 num_recon) +{ + return hfi_buffer_comv_enc(frame_width, frame_height, 16, + num_recon, HFI_CODEC_ENCODE_AVC); +} + +static inline +u32 hfi_buffer_comv_h265e(u32 frame_width, u32 frame_height, u32 num_recon) +{ + return hfi_buffer_comv_enc(frame_width, frame_height, 32, + num_recon, HFI_CODEC_ENCODE_HEVC); +} + +static inline +u32 hfi_buffer_non_comv_enc(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes_enc, u32 lcu_size, u32 standard) +{ + u32 frame_width_coded = 0, frame_height_coded = 0; + u32 width_in_lcus = 0, height_in_lcus = 0; + u32 num_lcu_in_frame = 0, num_lcumb = 0; + u32 frame_rc_buf_size = 0; + + width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size); + height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size); + num_lcu_in_frame = width_in_lcus * height_in_lcus; + frame_width_coded = width_in_lcus * (lcu_size); + frame_height_coded = height_in_lcus * (lcu_size); + num_lcumb = (frame_height_coded / lcu_size) * + ((frame_width_coded + lcu_size * 8) / lcu_size); + frame_rc_buf_size = + size_frame_rc_buf_size(standard, frame_height_coded, + num_vpp_pipes_enc); + return size_enc_slice_info_buf(num_lcu_in_frame) + + SIZE_SLICE_CMD_BUFFER + + SIZE_SPS_PPS_SLICE_HDR + + frame_rc_buf_size + + enc_bitcnt_buf_size(num_lcu_in_frame) + + enc_bitmap_buf_size(num_lcu_in_frame) + + SIZE_BSE_SLICE_CMD_BUF + + SIZE_LAMBDA_LUT + + size_override_buf(num_lcumb) + + size_ir_buf(num_lcu_in_frame); +} + +static inline +u32 hfi_buffer_non_comv_h264e(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes_enc) +{ + return hfi_buffer_non_comv_enc(frame_width, frame_height, + num_vpp_pipes_enc, 16, + HFI_CODEC_ENCODE_AVC); +} + +static inline +u32 hfi_buffer_non_comv_h265e(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes_enc) +{ + return hfi_buffer_non_comv_enc(frame_width, frame_height, + num_vpp_pipes_enc, 32, + HFI_CODEC_ENCODE_HEVC); +} + +static inline +u32 size_enc_ref_buffer(u32 frame_width, u32 frame_height) +{ + u32 u_buffer_width = 0, u_buffer_height = 0; + u32 u_chroma_buffer_height = 0; + + u_buffer_height = ALIGN(frame_height, HFI_VENUS_HEIGHT_ALIGNMENT); + u_chroma_buffer_height = frame_height >> 1; + u_chroma_buffer_height = ALIGN(u_chroma_buffer_height, HFI_VENUS_HEIGHT_ALIGNMENT); + u_buffer_width = ALIGN(frame_width, HFI_VENUS_WIDTH_ALIGNMENT); + + return (u_buffer_height + u_chroma_buffer_height) * u_buffer_width; +} + +static inline +u32 size_enc_ten_bit_ref_buffer(u32 frame_width, u32 frame_height) +{ + u32 ref_buf_height = 0, ref_luma_stride_in_bytes = 0; + u32 chroma_size = 0, ref_buf_size = 0; + u32 u_ref_stride = 0, luma_size = 0; + u32 ref_chrm_height_in_bytes = 0; + + ref_buf_height = (frame_height + (HFI_VENUS_HEIGHT_ALIGNMENT - 1)) & + (~(HFI_VENUS_HEIGHT_ALIGNMENT - 1)); + ref_luma_stride_in_bytes = ((frame_width + SYSTEM_LAL_TILE10 - 1) / SYSTEM_LAL_TILE10) * + SYSTEM_LAL_TILE10; + u_ref_stride = 4 * (ref_luma_stride_in_bytes / 3); + u_ref_stride = (u_ref_stride + (BUFFER_ALIGNMENT_128_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_128_BYTES - 1)); + luma_size = ref_buf_height * u_ref_stride; + ref_chrm_height_in_bytes = (((frame_height + 1) >> 1) + (BUFFER_ALIGNMENT_32_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_32_BYTES - 1)); + chroma_size = u_ref_stride * ref_chrm_height_in_bytes; + luma_size = (luma_size + (BUFFER_ALIGNMENT_4096_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_4096_BYTES - 1)); + chroma_size = (chroma_size + (BUFFER_ALIGNMENT_4096_BYTES - 1)) & + (~(BUFFER_ALIGNMENT_4096_BYTES - 1)); + ref_buf_size = luma_size + chroma_size; + + return ref_buf_size; +} + +static inline +u32 hfi_buffer_dpb_enc(u32 frame_width, u32 frame_height, bool is_ten_bit) +{ + u32 metadata_stride, metadata_buf_height, meta_size_y, meta_size_c; + u32 ten_bit_ref_buf_size = 0, ref_buf_size = 0; + u32 _size = 0; + + if (!is_ten_bit) { + ref_buf_size = size_enc_ref_buffer(frame_width, frame_height); + metadata_stride = + hfi_ubwc_calc_metadata_plane_stride(frame_width, 64, + HFI_COL_FMT_NV12C_Y_TILE_WIDTH); + metadata_buf_height = + hfi_ubwc_metadata_plane_bufheight(frame_height, 16, + HFI_COL_FMT_NV12C_Y_TILE_HEIGHT); + meta_size_y = + hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height); + meta_size_c = + hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height); + _size = ref_buf_size + meta_size_y + meta_size_c; + } else { + ten_bit_ref_buf_size = size_enc_ten_bit_ref_buffer(frame_width, frame_height); + metadata_stride = + hfi_ubwc_calc_metadata_plane_stride(frame_width, + VENUS_METADATA_STRIDE_MULTIPLE, + HFI_COL_FMT_TP10C_Y_TILE_WIDTH); + metadata_buf_height = + hfi_ubwc_metadata_plane_bufheight(frame_height, + VENUS_METADATA_HEIGHT_MULTIPLE, + HFI_COL_FMT_TP10C_Y_TILE_HEIGHT); + meta_size_y = + hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height); + meta_size_c = + hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height); + _size = ten_bit_ref_buf_size + meta_size_y + meta_size_c; + } + + return _size; +} + +static inline +u32 hfi_buffer_dpb_h264e(u32 frame_width, u32 frame_height) +{ + return hfi_buffer_dpb_enc(frame_width, frame_height, 0); +} + +static inline +u32 hfi_buffer_dpb_h265e(u32 frame_width, u32 frame_height, bool is_ten_bit) +{ + return hfi_buffer_dpb_enc(frame_width, frame_height, is_ten_bit); +} + +static inline +u32 hfi_buffer_vpss_enc(u32 dswidth, u32 dsheight, bool ds_enable, + u32 blur, bool is_ten_bit) +{ + u32 vpss_size = 0; + + if (ds_enable || blur) + vpss_size = hfi_buffer_dpb_enc(dswidth, dsheight, is_ten_bit); + + return vpss_size; +} + +static inline +u32 hfi_iris3_enc_min_input_buf_count(u32 totalhblayers) +{ + u32 numinput = 3; + + if (totalhblayers >= 2) + numinput = (1 << (totalhblayers - 1)) + 2; + + return numinput; +} + +u32 enc_output_buffer_size_iris3(struct iris_inst *inst); +u32 get_recon_buf_count(struct iris_inst *inst); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c index 58498af..2828227 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c @@ -54,7 +54,7 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size) if (inst->cap[STAGE].value == STAGE_1) vsp_cycles = vsp_cycles * 3; - vsp_cycles += mbs_per_second * base_cycles; + vsp_cycles += mbs_per_second * base_cycles; freq = max3(vpp_cycles, vsp_cycles, fw_cycles); From patchwork Mon Dec 18 11:32:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755548 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54FD21C6A7; Mon, 18 Dec 2023 11:40:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="hr88ExrJ" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIB0Cqb001111; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=2RhQtL+sFcwJZo/VmDn+5n1VU9Q22OgEKdRfJvEjC8g=; b=hr 88ExrJvBVxjn2q3J3LUJWIXKZhX/7712NvhJWDqGTbHyI9P1iDKbmDw1adr1Gq5X iZus0Cv9NYgL1fkkZRVrxSSjIegrra7xaQ5nMTOOsrFlDRD1Gmk8RjtZDYWR/pCX LtRQ9y7eL5u6R5QyEoWch+XPWvZARUpIjmZtCq5q8aL11FhKTR5zhRnXc+a1CRVM nDy90MaIZV1WfteTK0i6IqhpcV3BWgoUHL+oQbqeM26DeLxtV0QYiCmv9ZbHbbEd dWyOw0aHMEYeSKxeHE2PzDxnDym0DUlyEBY4R4bbJUf2ZcDSMZeC2LwiqMwRvpQN 7MflnMAkQRxI4rn3mW0w== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2md2r4n6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5KW029944; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00af-1; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBa9rW000378; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX864030065; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 7432D2390; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 34/34] media: iris: add power management for encoder Date: Mon, 18 Dec 2023 17:02:29 +0530 Message-Id: <1702899149-21321-35-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 5IZyREQe7H_wDHPhZJZUdD50bi2vuaba X-Proofpoint-GUID: 5IZyREQe7H_wDHPhZJZUdD50bi2vuaba X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 suspectscore=0 lowpriorityscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Hardware specific power sequence include programming specific sequence of registers. Implements this sequence for iris3. Also, implement iris3 Encoder specific calculation for clock and bus bandwidth which depends on hardware configuration, codec format, resolution and frame rate. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 2 + .../media/platform/qcom/vcodec/iris/iris_power.c | 15 ++-- .../platform/qcom/vcodec/iris/platform_common.h | 3 + .../platform/qcom/vcodec/iris/platform_sm8550.c | 14 +++ .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 100 +++++++++++++++------ 5 files changed, 103 insertions(+), 31 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c index 559b0dd..1a2e305 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -799,6 +799,8 @@ int decide_quality_mode(struct iris_inst *inst) if (mbpf <= max_hq_mbpf && mbps <= max_hq_mbps) mode = MAX_QUALITY_MODE; + inst->cap[QUALITY_MODE].value = mode; + return mode; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.c b/drivers/media/platform/qcom/vcodec/iris/iris_power.c index 77439e3..be71751 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_power.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.c @@ -48,11 +48,16 @@ static int iris_vote_buses(struct iris_inst *inst) vote_data->height = inp_f->fmt.pix_mp.height; vote_data->fps = inst->max_rate; - if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) { - vote_data->color_formats[0] = V4L2_PIX_FMT_NV12; - vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat; - } else { - vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat; + if (inst->domain == ENCODER) { + vote_data->color_formats[0] = + v4l2_colorformat_to_driver(inst, inst->fmt_src->fmt.pix_mp.pixelformat); + } else if (inst->domain == DECODER) { + if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) { + vote_data->color_formats[0] = V4L2_PIX_FMT_NV12; + vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat; + } else { + vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat; + } } call_session_op(core, calc_bw, inst, vote_data); diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 42ac662..e9b5c99 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -174,6 +174,7 @@ enum plat_inst_cap_type { INPUT_BUF_HOST_MAX_COUNT, STAGE, PIPE, + QUALITY_MODE, POC, CODED_FRAMES, BIT_DEPTH, @@ -279,6 +280,8 @@ struct format_capability { struct platform_data { const struct bus_info *bus_tbl; unsigned int bus_tbl_size; + const struct bw_info *bw_tbl_enc; + unsigned int bw_tbl_enc_size; const struct bw_info *bw_tbl_dec; unsigned int bw_tbl_dec_size; const char * const *pd_tbl; diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 17c5856..d47c985 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -1005,6 +1005,11 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_pipe}, + {QUALITY_MODE, ENC, CODECS_ALL, + MAX_QUALITY_MODE, + POWER_SAVE_MODE, 1, + POWER_SAVE_MODE}, + {POC, DEC, H264, 0, 2, 1, 1, 0, HFI_PROP_PIC_ORDER_CNT_TYPE}, @@ -1051,6 +1056,13 @@ static const char * const sm8550_pd_table[] = { "iris-ctl", "vcodec", NULL }; static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx", NULL }; +static const struct bw_info sm8550_bw_table_enc[] = { + { 1944000, 1491000, 2693000 }, /* 3840x2160@60 */ + { 972000, 765000, 1366000 }, /* 3840x2160@30 */ + { 489600, 557000, 780000 }, /* 1920x1080@60 */ + { 244800, 288000, 399000 }, /* 1920x1080@30 */ +}; + static const struct bw_info sm8550_bw_table_dec[] = { { 2073600, 1608000, 2742000 }, /* 4096x2160@60 */ { 1036800, 826000, 1393000 }, /* 4096x2160@30 */ @@ -1133,6 +1145,8 @@ struct platform_data sm8550_data = { .clk_rst_tbl = sm8550_clk_reset_table, .clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table), + .bw_tbl_enc = sm8550_bw_table_enc, + .bw_tbl_enc_size = ARRAY_SIZE(sm8550_bw_table_enc), .bw_tbl_dec = sm8550_bw_table_dec, .bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec), diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c index 2828227..8497b84 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c @@ -11,12 +11,13 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size) { + u32 operating_rate, vsp_factor_num = 1, vsp_factor_den = 1; u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0; u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0; + u32 vpp_cycles_per_mb, mbs_per_second; u32 base_cycles = 0, fps, mbpf; u32 height = 0, width = 0; struct v4l2_format *inp_f; - u32 mbs_per_second; inp_f = inst->fmt_src; width = max(inp_f->fmt.pix_mp.width, inst->crop.width); @@ -29,32 +30,74 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size) fw_cycles = fps * inst->cap[MB_CYCLES_FW].value; fw_vpp_cycles = fps * inst->cap[MB_CYCLES_FW_VPP].value; - vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value / - inst->cap[PIPE].value; - vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles); - - if (inst->cap[PIPE].value > 1) - vpp_cycles += div_u64(vpp_cycles * 59, 1000); - - base_cycles = inst->cap[MB_CYCLES_VSP].value; - bitrate = fps * data_size * 8; - vsp_cycles = bitrate; - - if (inst->codec == VP9) { - vsp_cycles = div_u64(vsp_cycles * 170, 100); - } else if (inst->cap[ENTROPY_MODE].value == - V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) { - vsp_cycles = div_u64(vsp_cycles * 135, 100); - } else { - base_cycles = 0; - vsp_cycles = div_u64(vsp_cycles, 2); - } - vsp_cycles = div_u64(vsp_cycles * 21, 20); + if (inst->domain == ENCODER) { + vpp_cycles_per_mb = + inst->cap[QUALITY_MODE].value == POWER_SAVE_MODE ? + inst->cap[MB_CYCLES_LP].value : + inst->cap[MB_CYCLES_VPP].value; + + vpp_cycles = mbs_per_second * vpp_cycles_per_mb / inst->cap[PIPE].value; + + if (inst->cap[B_FRAME].value > 1) + vpp_cycles += (vpp_cycles / 4) + (vpp_cycles / 8); + else if (inst->cap[B_FRAME].value) + vpp_cycles += vpp_cycles / 4; + + vpp_cycles += max(div_u64(vpp_cycles, 20), fw_vpp_cycles); + if (inst->cap[PIPE].value > 1) + vpp_cycles += div_u64(vpp_cycles, 100); + + operating_rate = inst->cap[OPERATING_RATE].value >> 16; + if (operating_rate > (inst->cap[FRAME_RATE].value >> 16) && + (inst->cap[FRAME_RATE].value >> 16)) { + vsp_factor_num = operating_rate; + vsp_factor_den = inst->cap[FRAME_RATE].value >> 16; + } + vsp_cycles = div_u64(((u64)inst->cap[BIT_RATE].value * vsp_factor_num), + vsp_factor_den); + + base_cycles = inst->cap[MB_CYCLES_VSP].value; + if (inst->cap[ENTROPY_MODE].value == + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) { + vsp_cycles = div_u64(vsp_cycles * 135, 100); + } else { + base_cycles = 0; + vsp_cycles = div_u64(vsp_cycles, 2); + } + vsp_cycles = div_u64(vsp_cycles * 21, 20); + + if (inst->cap[STAGE].value == STAGE_1) + vsp_cycles = vsp_cycles * 3; - if (inst->cap[STAGE].value == STAGE_1) - vsp_cycles = vsp_cycles * 3; + vsp_cycles += mbs_per_second * base_cycles; + } else if (inst->domain == DECODER) { + vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value / + inst->cap[PIPE].value; + vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles); + + if (inst->cap[PIPE].value > 1) + vpp_cycles += div_u64(vpp_cycles * 59, 1000); + + base_cycles = inst->cap[MB_CYCLES_VSP].value; + bitrate = fps * data_size * 8; + vsp_cycles = bitrate; + + if (inst->codec == VP9) { + vsp_cycles = div_u64(vsp_cycles * 170, 100); + } else if (inst->cap[ENTROPY_MODE].value == + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) { + vsp_cycles = div_u64(vsp_cycles * 135, 100); + } else { + base_cycles = 0; + vsp_cycles = div_u64(vsp_cycles, 2); + } + vsp_cycles = div_u64(vsp_cycles * 21, 20); + + if (inst->cap[STAGE].value == STAGE_1) + vsp_cycles = vsp_cycles * 3; vsp_cycles += mbs_per_second * base_cycles; + } freq = max3(vpp_cycles, vsp_cycles, fw_cycles); @@ -78,8 +121,13 @@ int iris_calc_bw_iris3(struct iris_inst *inst, struct bus_vote_data *data) if (mbps == 0) return 0; - bw_tbl = core->platform_data->bw_tbl_dec; - num_rows = core->platform_data->bw_tbl_dec_size; + if (inst->domain == DECODER) { + bw_tbl = core->platform_data->bw_tbl_dec; + num_rows = core->platform_data->bw_tbl_dec_size; + } else if (inst->domain == ENCODER) { + bw_tbl = core->platform_data->bw_tbl_enc; + num_rows = core->platform_data->bw_tbl_enc_size; + } if (!bw_tbl || num_rows == 0) return 0;