From patchwork Wed Aug 11 08:42:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 495320 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB64EC4320A for ; Wed, 11 Aug 2021 08:44:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CA3E60527 for ; Wed, 11 Aug 2021 08:44:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236232AbhHKIoi (ORCPT ); Wed, 11 Aug 2021 04:44:38 -0400 Received: from mail-am6eur05on2078.outbound.protection.outlook.com ([40.107.22.78]:49598 "EHLO EUR05-AM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235282AbhHKIoc (ORCPT ); Wed, 11 Aug 2021 04:44:32 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AoDZjQ3g0t8XdP4MnxDxbY+Ic8fvZhHGKJPko8Tzc9xG8pmETGogc+ufNB5hZnv6bVWWtpaMCAKLiA/c+Pn4g5iNKHsAWF1JIHVGT9bkarIWtgYA65klyOKW2Dw7gYheKHyEmYMUorvDKxApIThN3EeKWcLXvywOkTS8VObpDQ0ORsdixX2sSPdyi6uqAI0tLoKw1+1djFdfhGJSwF2MdIlRQnRoF6AtVBRQHRflDnHEGOfrYBThOhX87CfhnxvlToUVQMQJn0gL7N4KF2v8++3lQed3FAjBBQ2l/SwMbpmcBlowhWtd8DZeu5z2XcZptrDyjRtfzxZWv8q8+ME2bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OhxRUPnEwlV4CgYdBTvE2xDk5TIbD3YkypRNoc0XgM4=; b=PiteA9zbmzr0o7Eu1OW4ewOAmnNVeMlEG5DhpkbQKQptKayirGrDQnDzLgkbuiME5Pw48yUKnuqE55faCrV5fZ4WpNta8HelMjtd0Zv7/08YqhGkmF2ny92EQG+QpkpMCj/bMLm1D4DOHrsghF5LoVSaDkC3hdaPIs5OHOVEhjn5RfDJXYkmjRCL+I+nqvJbDdN1915HEJr8PLD54omta2Sl7LCKBJ5PUAqyE1FxZ60pnq8feJOFjGmoGCa0q09PKnuRS/ykB/jTJddZ3iz1FJ6ZKA2jBQBzxYb/wgn3qMqZLP43OA14j8fr2uduLcSRIK1I4EGCVR/oCCBxWb0o5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OhxRUPnEwlV4CgYdBTvE2xDk5TIbD3YkypRNoc0XgM4=; b=ZKkmfMXn07nt9RD9SL1j8pIkjekvuZyi8T91unlhSjpdAwHF6SBei6K1MNhaWq2WyYSRGcdZ17DQOWaMjB03kw7Y58WIL/bxb7DlFPwaqr8MysRbTbn2Xr5f5wAt2nHHtjNG3rvyfyRo3AxVqyBZU4uFuOanB7VVtvvW5phPEPw= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM6PR04MB5126.eurprd04.prod.outlook.com (2603:10a6:20b:4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.19; Wed, 11 Aug 2021 08:44:05 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384%6]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 08:44:05 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de Cc: hverkuil-cisco@xs4all.nl, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 02/14] media:Add nt8 and nt10 video format. Date: Wed, 11 Aug 2021 16:42:58 +0800 Message-Id: <981eee1270fb47658f7821d59525ff4c53c30644.1628671163.git.ming.qian@nxp.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-ClientProxiedBy: SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.4 via Frontend Transport; Wed, 11 Aug 2021 08:44:01 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e10345b6-31ed-4ff8-aee8-08d95ca42cfb X-MS-TrafficTypeDiagnostic: AM6PR04MB5126: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NXeqF3C+i+E+Jdhg1pBV2A9u2YvEPF7GnsjP4lBHZlX3dx6QAA+LDZHjl+3cMdTaoB0HsthLbjkY9diNmjLWkZIPt3tB+NI/7bS5J2Gf2x88dRmjfU50d1vAXs0BX324UjjWRJN6ZLId35u2w99Nis88dQxCw+28kX0uzV6kiObxS3yL4eXwNP5cT0OF5Kj+nIPLbjdIzbMRpICnFHB/G6NgZQ1p7JWH3eA0kCVUNqQXRdODin2XQF1rbRy8CGHLDhMx319SK4gLQ6+xaGfJHYmO+AEki/4+qPgWd6tsjdwxsF/t8xBtgTGtZL974xEQVEDaqX+flGFo0xa3xSYkfd66jVqkJhhatIF/i2fupKJFK5jUJo4QLMqsPzfguvwy9GO4FuK/zcmw+937IZeVHeGfsvjc6Xkn+Sr7+q4PXURw10HZagEmk3udX+rKSf6LXsh0kLbA/ut4vBBwSa8F3G8xp5J6z0XfcbXoyKIgvISTqWqE8UC3roSXdqyCZe3Xxxy5QZCDfL9GEAWyrA7SXiaUcJE2eD+vD2Ta0aP9NemSlA8v90ISoVK0ItXyRoO1W1bklPX0MX1kLs40jIc7hRNji4YNr0529JVi9wwwDoG5fuNKDifbEhjmM6s12AHxOhpG1f6i0Qs2qV+mbJuoRmi+wlEpQTMTU9vN0FAUszhcE+vFe9n03tlbxl9usd/wfOy1npDr1/uWfydjcp11c7G7z9yZaKLJ4NeIvpqPDmylFwOt9wDTGOrS8ZlxE74No/WPDLIDLKyViu2hQmV5KzoOHQktX7EzDhuYujjMVRY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(136003)(396003)(346002)(366004)(8676002)(4326008)(38100700002)(2906002)(6486002)(966005)(86362001)(6666004)(38350700002)(8936002)(478600001)(186003)(36756003)(316002)(66476007)(26005)(44832011)(2616005)(66946007)(7696005)(66556008)(5660300002)(83380400001)(7416002)(52116002)(956004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: W3m54KMQRe2fbQ6Gw59/ho6zIXJjatz8UCQ13NxQi4EAic6AUlYNLpaykdT/4d4nlprTpi5dShNQdpt0/KjtkI/o+KDO3OkDzIjp7kILg3YmX4nuaYc1ygHm7oTdCemF+NLk7y4S8fq/3ViJNqTYZqRDfEjKw7tNQepkZBp3aEsOBSSpidIWV533GbVdVQhiqne7730y49cQkLQ5Cxv0Pfcwc9iU0B8/z84dXhopyNxS71oHLWyFwLAIwISkgp16U0Tg1JFnr3jIXFAKsAoj9LcllYYBbXB8GMGu8WojNJFdNsKeQOkwVWOKgzxVywZyHBA5gevZdFNJlcPL5Bm5I4p3KgzwTVXmbIaj17Ml9fYNxGJ70smSavIWvKP0GcFMWtbuwmIstS54TMenJEIfrg6yf5lXNRHDxr1OqCdgv2zt2PhVoWK/eE3/h/9XpcPT6YwUpZLurCR/8uIBKWRxyGS6S8dOryZ8HIXZ9Yo9qSteC69KILZC9uSqDiEm5H8vE+ROeRYajkrm16sWC6wK7FSpcP9nbCgpC3ZcHeeIJf2btdXc7G4161e1uUnlteOC/hzOhlBXhyf1LxE0a3PNphr23txinPMAjYJfffYfw4RatxNyKvGeHCAtyIcr6I7ywOeWldxq6NgJfhz4YU3gnFrf2diqtSpU6BgmtaYOEMa4oRO6qeoi28tGGgP863Ni5E4tpJlHNY6Fc0RXy85TOiUq6DYJ84MuFiIEpoDi4RhmpLDbXFR+40aH2TFOwjOyZG3grSK9XsHAuFM2Q1erxEitXoFzZ7kZWW80Sy+AsmV0I5/5JLtxD/RBL3kC7/roqWsjupPnjAEYZWx1Nzr2YBZJBzYlO6rdR85r5LitF2YW5KrMTYmdjHImgDh46bZVN2bW7YiZzqnsW59jAL/LIbLj14x2TsVIaLwfXw79J0ondNAMGaaEhGkWBTKR3zXTA2yOAn4nScGYdp9e/M0TskP6KHvS68ZA2q9UiV88IXQcmjCQAVAvfD55+jy41aArkYRGy0gHQ9vqNrf0lkNnnvBIxO/GIpTzV34+neUjcyh8O3yNWFyucLtHjpm+XNd6y6WDJMxtgJBrazydQZMVUhI9TQscX+w/98WgrtqjWbh1yLmEBIkZRD6ZiyZ5HzxDRaIEy7UmGNAaPWik3FNdV6naf6KGRedO6L0QO91hjX+sA6Rz9AROqHFBShbKnyx/gkv592evRB+uEkueAqI5ibzwYPmr/zq0KqkgFL8Ba+52VLvWm7dlSqk7FKf/T0FS4be1TWlufHPQscod/4oRTBe9qaL6BpzM+2LKtCyqNOt8WP4wiuLDXE199zr9N7Y3 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: e10345b6-31ed-4ff8-aee8-08d95ca42cfb X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 08:44:05.0448 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gztnfpGUd0Y6V2z91Cov/48k+ZnNBDZtLS0In3LKvaAnb4Pd1eFx5Z9K9eHPmL7NkqSYItF3+8mkjZ502+aW7g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB5126 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org NT8 is 8-bit tiled nv12 format used by amphion decoder. NT10 is 10-bit tiled format used by amphion decoder. The tile size is 8x128 Signed-off-by: Ming Qian Signed-off-by: Shijie Qin Signed-off-by: Zhou Peng --- .../userspace-api/media/v4l/pixfmt-yuv-planar.rst | 15 +++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++ include/uapi/linux/videodev2.h | 4 ++++ 3 files changed, 21 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst index 090c091affd2..ce3419d95691 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst @@ -254,6 +254,8 @@ of the luma plane. .. _V4L2-PIX-FMT-NV12MT: .. _V4L2-PIX-FMT-NV12MT-16X16: +.. _V4L2_PIX_FMT_NV12_8L128: +.. _V4L2_PIX_FMT_NV12_10BE_8L128: NV12MT and MV12MT_16X16 ----------------------- @@ -276,6 +278,19 @@ If the vertical resolution is an odd number of macroblocks, the last row of macroblocks is stored in linear order. The layouts of the luma and chroma planes are identical. +``V4L2_PIX_FMT_NV12_8L128`` stores pixel in 2D 8x128 tiles, and stores +tiles linearly in memory. The line stride must be aligned to a multiple of 256. +The image height must be aligned to a multiple of 128. The layouts of the +luma and chroma planes are identical. + +``V4L2_PIX_FMT_NV12_10BE_8L128`` stores 10 bits pixel in 2D 8x128 tiles, +and stores tiles linearly in memory. the data is arranged at the big end. +The line stride must be aligned to a multiple of 256 bytes. The image +height must be aligned to a multiple of 128. The layouts of the luma +and chroma planes are identical. Note the tile size is 8bytes multiplied +by 128 bytes, it means that the low bits and high bits of one pixel +may be in differnt tiles. + .. _nv12mt: .. kernel-figure:: nv12mt.svg diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 05d5db3d85e5..22f97a1cfbeb 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1367,6 +1367,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break; case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A Parameters"; break; case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A Statistics"; break; + case V4L2_PIX_FMT_NV12_8L128: descr = "Y/CbCr 4:2:0 (8x128 Linear)"; break; + case V4L2_PIX_FMT_NV12_10BE_8L128: descr = "Y/CbCr 4:2:0 10BE(8x128 Linear)"; break; default: /* Compressed formats */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 9260791b8438..167c0e40ec06 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -627,6 +627,10 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_YUV444M v4l2_fourcc('Y', 'M', '2', '4') /* 24 YUV444 planar */ #define V4L2_PIX_FMT_YVU444M v4l2_fourcc('Y', 'M', '4', '2') /* 24 YVU444 planar */ +/* Tiled YUV formats */ +#define V4L2_PIX_FMT_NV12_8L128 v4l2_fourcc('N', 'A', '1', '2') /* Y/CbCr 4:2:0 8x128 tiles */ +#define V4L2_PIX_FMT_NV12_10BE_8L128 v4l2_fourcc('N', 'T', '1', '2') /* Y/CbCr 4:2:0 10-bit 8x128 tiles */ + /* Bayer formats - see http://www.siliconimaging.com/RGB%20Bayer.htm */ #define V4L2_PIX_FMT_SBGGR8 v4l2_fourcc('B', 'A', '8', '1') /* 8 BGBG.. GRGR.. */ #define V4L2_PIX_FMT_SGBRG8 v4l2_fourcc('G', 'B', 'R', 'G') /* 8 GBGB.. RGRG.. */ From patchwork Wed Aug 11 08:42:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 495319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57503C00143 for ; Wed, 11 Aug 2021 08:44:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3A9C160FE6 for ; Wed, 11 Aug 2021 08:44:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236263AbhHKIol (ORCPT ); Wed, 11 Aug 2021 04:44:41 -0400 Received: from mail-am6eur05on2078.outbound.protection.outlook.com ([40.107.22.78]:49598 "EHLO EUR05-AM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236183AbhHKIoj (ORCPT ); Wed, 11 Aug 2021 04:44:39 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GzPzFmom2FEpzNv7/ETS2rS16gTdDvr+fkXovw9n1ulS7sHksvSf3SyOMPukUAGRjFMuM//TTwEQsPen4Tn543s931ED0u6uRFhuQ4+KNtrpWILfsdUt/lm/QgT4a1ta8KjxhckRP7kDcBOT4vW+uJgGrEAekCu3I816eRUrIRthn6osrsxeXDtn83Y3kZnlRSeTrCBSQHmWmMvRsIQygcwIrqSOabmIDSygYLoQoffDW71EOFj8XUVGAMaN/Q+Iahw9S8fFK7xZc8U4kFLMYC0W695WGSMGfKYUd4txlF3V+/v9xBNEHDTnhe8YKrOpRjYuZ1fifJML4yeiAtq9uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nuGAhvg+ZR0HNx8MmJlLyP+rZrtnojGtIfP4DZRJkyo=; b=g0T92MS9L+/Uy8ykg3C/Rh4bYaxiEj/AAHZJNGbcTp5Fjwcayx77w7Qk/pSUC8paKe9z1Ki2Accl0IVuvEhTAvsx6L6IlhQWpMp4qwJFY+3SzO4wLNBtTW/wEzwOU95xI4NQPE3MIkgUAadnUVDaHE3AlD/i7U9PsBOpaqPjb3Nmjxnx8rwJZkyw6oj+96oU87phF8NAGIHZYc/q0CJgdW8/0LgRgXJxL3gLjK9WRvsSf159QcDmHfMC8bmRRjUWI1xO2c5obXk3uhsO3mYgt7rOIoKosaLlAG/FXR0ElVg4Mh/ggrheJXudQKKK8fj4DY7jwEvyx9kMm85xvWI++g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nuGAhvg+ZR0HNx8MmJlLyP+rZrtnojGtIfP4DZRJkyo=; b=RQrUulVfhtsVvmPGgA/GE01RDqHqSHhqcjrpZ36jvlaJDLhn0nqwn43Df7+2jrIX7D1FI7XGvLlHxCZe4LEO2e6zzDE/8u53HlKLQD5K5SiGtuPCAoJp3ahgR/Bm5vLK/QJxueiLt9Fd47u2qejM5kjniGQa9cYQ5eCIvanO5Io= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM6PR04MB5126.eurprd04.prod.outlook.com (2603:10a6:20b:4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.19; Wed, 11 Aug 2021 08:44:09 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384%6]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 08:44:09 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de Cc: hverkuil-cisco@xs4all.nl, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 03/14] media:Add v4l2 buf flag codec data. Date: Wed, 11 Aug 2021 16:42:59 +0800 Message-Id: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-ClientProxiedBy: SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.4 via Frontend Transport; Wed, 11 Aug 2021 08:44:05 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8115f49e-e23b-4957-9192-08d95ca42f6b X-MS-TrafficTypeDiagnostic: AM6PR04MB5126: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3383; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CkbGnIrcSpWwHQ/G8Y+AusfAewtU3InLItemp4xcXVGMFJsByKe51FEMMujP5Rl0RxlQ3B/rLK1Ohk6WMypPu7V9+uFUgyJm26ADOb8ymO47UzaeLkMRN6E9dC4021/A1kxwb/kpVUDCdjDpTltZlHqIwaCeZIpifru4CXGoDUuEjwfNhzAhDWziGXRjglIctYi9EYEEJ8w5MvoDX9XDXwz2CnffJLSqojXQyfEiDaFXek7qcGdxoe3sR54aBmQW6kPx8M3Fi7jpPSSw1o7lDg52GagEOwWAB76pEQp+GHNu+LZ7Z3P+lLJhbNkcgy2HyFf6uTTTO+wAkBGO8MdyjVqhsL0cN2SBqvYF16S6X4Yl2TZLwn++jJ/oLcmU0ovlPIx5oqeSPTWIj8gZ1oqvuubFG04Z6u4uHJxmeFAiIHzG9/GKzdMZZgcxkkhq7ZeZQFJBWvAnR1Mru4DIdgTde1qWIFcD9ZUMOmT3h9pZ/y/u56pn+1nU/HJlfeds8BvzpNLPcfBpw+aIFioHzYrAFAf8+Mon5eec7nKZdzvRlUMJNby4kQBlti+QsA22FB/zahFajXafLRlM9mZC3hV4dFkGUwkaFoNtShkwl1+zAqQ+tPgzQA96IRZzKEJxnRXJ82QdsVWgdokHY3q8Q0t7VsvDKw+X+hjNjU0tcjhANKfiShbzVlM7TtjaHpHJMAOVB1/2o9SNwTI1y0T1INXSgQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(136003)(396003)(346002)(366004)(8676002)(4326008)(38100700002)(2906002)(6486002)(86362001)(6666004)(38350700002)(8936002)(478600001)(186003)(36756003)(316002)(66476007)(26005)(44832011)(2616005)(66946007)(7696005)(66556008)(5660300002)(83380400001)(7416002)(52116002)(956004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: UFRGzuRtHgnCKy39UBC7ZueOtDmsU2bBJ6subGOcsVG3F/TNiWHvevLEvyq34HkRcULCuszN2fBWA8fT0GGr3X3TfWdjHnC60FuZo+EW+5pKxdBAQLZmn2LbBw7yBiylJ+asGtuIVUpiZ2wcgfJ7RgRbVyMyPYcWyFQmjyf0rPlkeBDaUwuNrf3mGAgSUrDWiNur0Cy+jqBeX44Fw+SYUOgsr+ZwW+38sJsQs5qz9SxTJkI53FGf7PkqdCq7wTwNV2OlzT5oqIzWUruKCRNWWPbMlaxy6BxRsa2uRlyOR6GYEhjHWTKuYCneqM/x0xqq1KoYlL63Zu1CmWwewyi9KtlMvNtRu1S2mH1DrXK3qsOxyip62IS/kyecyus0DVDPwwe34Dh47GYstpDnEncNOFApXVkkWnHsscgJK0N9Wt0X5VTbyfRN+0u+XNAd0pYbDRFnvOyD7jTUuCaO81EBX+KlPc1JbfOhM9JXTlbocWxSSfoa3u4QgKUdBi5FYiztTCDkAhcLqIEciibZn0gHJmHg9TnDAQeJFDk0nbBVsZMBLuTUQgyN5t7PsLsYPRFBUPqrVWn/mhkUEctb10kkqVWg/aWgR+GC1kzS2o5psuaN5wtxLgzwfY+POloPUVpDEnKvGJmVuUcfrXTuJy18TqblHIDpe0H0CF9zlizmC48MaR6kUgqXJZfX6DfAXOxdY7Bv5CKZ/hjaf1fnN8lVoHXANb29C+vlWgpaXoNxZMobBvYJL03jusNYZCD2EHPlI3qizAWH1buq8K6RoD/wyCp41wmbrWdcLurbcgFj+jqNvqaeQNmjQ2wI4PWqgtt4nkH9LN8A8Lv29rVXItWSByJyJC93r9G1SyAXwYffOQAX9pIcslKhreSOMlrqe8I+HRLc9lAPE/cigihkNvtsxspCGpobSaDRpUSuq2CJBvqu2GspgWoCj7zHGy1Z0KJg2H8gTxFgzQUKfHAmAhaJ0y0BiNZkI7PWFi2r1PXM3txIG3bQmswjJwoic16lEpJdJoRZu/CEPMIdN/ll5mCrsR3NqXGFubRJcqiqeqjYAcqHxIITSWMPOjHZQHhqaRqi8gaYaQp1zzZXrkplJ5YZuESvRxxYQestE/sHG6hXhNDqXktuWn3sVu2Mm6hmMG0ZoX1vqdxOaAEABbp11e2bH5YJM+SKdhe3+03l0QZWQgWcg5ki07U0BQ2hrTOR/QDldrTjiuaM7pGtD/T2k0YxmNHf4niEj1gi6zwMqAFIn+v1M/pFCsqteD1yToJVBzJ9OIQIwunS/DExOM44nGcGWtHtfheQus7DsImm/g6+T1tiK13oZu9YErHfKnWn64u+ X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8115f49e-e23b-4957-9192-08d95ca42f6b X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 08:44:09.1225 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PnCRLslz9ddZGsXaFG8MgYxsdwtEo4NqGfRy/RTQ50NqUgyvm0GLzrjwX/jM3xhWNfN+whR+2D01y9mF1xV0oQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB5126 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org In some decoing scenarios, application may queue a buffer that only contains codec config data, and the driver needs to know whether is it a frame or not. So we add a buf flag to tell this case. Signed-off-by: Ming Qian Signed-off-by: Shijie Qin Signed-off-by: Zhou Peng --- Documentation/userspace-api/media/v4l/buffer.rst | 7 +++++++ include/uapi/linux/videodev2.h | 1 + 2 files changed, 8 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst index e991ba73d873..11013bcf8a41 100644 --- a/Documentation/userspace-api/media/v4l/buffer.rst +++ b/Documentation/userspace-api/media/v4l/buffer.rst @@ -607,6 +607,13 @@ Buffer Flags the format. Any subsequent call to the :ref:`VIDIOC_DQBUF ` ioctl will not block anymore, but return an ``EPIPE`` error code. + * .. _`V4L2-BUF-FLAG-CODECCONFIG`: + + - ``V4L2_BUF_FLAG_CODECCONFIG`` + - 0x00200000 + - The buffer only contains codec config data, eg. sps and pps. + Applications can set this bit when ``type`` refers to an output + stream, this flag is usually used by v4l2 decoder. * .. _`V4L2-BUF-FLAG-REQUEST-FD`: - ``V4L2_BUF_FLAG_REQUEST_FD`` diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 167c0e40ec06..5bb0682b4a23 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1119,6 +1119,7 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv) #define V4L2_BUF_FLAG_TSTAMP_SRC_SOE 0x00010000 /* mem2mem encoder/decoder */ #define V4L2_BUF_FLAG_LAST 0x00100000 +#define V4L2_BUF_FLAG_CODECCONFIG 0x00200000 /* request_fd is valid */ #define V4L2_BUF_FLAG_REQUEST_FD 0x00800000 From patchwork Wed Aug 11 08:43:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 495317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFEA8C43214 for ; Wed, 11 Aug 2021 08:44:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2C5D60FD9 for ; Wed, 11 Aug 2021 08:44:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236396AbhHKIo6 (ORCPT ); Wed, 11 Aug 2021 04:44:58 -0400 Received: from mail-eopbgr80089.outbound.protection.outlook.com ([40.107.8.89]:28293 "EHLO EUR04-VI1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S234568AbhHKIos (ORCPT ); Wed, 11 Aug 2021 04:44:48 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XFoyDpdx0RFC4Ad95tslShUm8tEG1nNuxPn1v52mDAR/Fm9K5L8lWlBJ0a7nfaLYcTheTXTgPaAsGinQlIaegxcCbUvzxyTgqyVbZK4kcAaVMzBurq2MR23ymSzx7TxEt6geeTMuieskVTNy2qKkC7+sor2kDyg9MunvGoSo/GN+pgECaJt5Hs1VuVVHvJAmTMklnTIvL7hD5UQwDPgPL9kph4CVKOVHiqFk6YWeJaFTEBLFMBPmJ4YK834OSVAbg7IrHY4SyLXk+EeLkzREWTeTvylacz46nPG4Y9+WUyPHNtCJC4I5J5YDy6mT7XbDXlOJaxovhqZE080lLOmMvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mQeRSWW7FoAIEwr0aeP3Ns+ovnuU51PiNjiIi0Bzyw0=; b=UCkIFcom7g6qbVBv5vIJXOdY5wBsGzLifkA1vvwk5h1uQngvl6oL5Z9ERZ+C4xFlzQvEBMQL7KZ2bjjkGtbi4IwjPnSEK66AUdYRp5hsNL1dfHvTh9lcs9WQXzMeGSI1MgPZCxUGxRpshnPSR2GH5jjcJwXP1vOVrViioYb2gjxfvBQqW0mbpPkciqiVcCa5tyQJl648/TSsoXsup0YgJCQfpf0kjNpVWgxLA/+WodmTPnNhtSJuFbmF2XD1PKGV9nBlFgBXGelRuKP6futqVKQb28459/oEDtGfID7y3actMhoK5+ITyGanSu44df14ZCGk6QjWDVcpTSh5ztmb9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mQeRSWW7FoAIEwr0aeP3Ns+ovnuU51PiNjiIi0Bzyw0=; b=jcVMRnCp5lzFHWNhfK092eqYt6BnBZvl+Oj7uk6eku5lopPNUrwMtrH/wwDEVx/STbT4Il1pvTZ8tmOOByckH4r/lhrXUkVIdTbtw1FZ+BFKc9NNZuWnlfuzxwOoARvfRQjorfout25pMR+6FVB4n+mesd34LypsH5BKjN2pTuQ= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM5PR0401MB2546.eurprd04.prod.outlook.com (2603:10a6:203:38::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.15; Wed, 11 Aug 2021 08:44:21 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384%6]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 08:44:21 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de Cc: hverkuil-cisco@xs4all.nl, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 06/14] media: imx: imx8q: implement vpu core communication based on mailbox Date: Wed, 11 Aug 2021 16:43:02 +0800 Message-Id: <7a945b82a4ce89d1fd8ded0b76b7c4ac5754ab6f.1628671163.git.ming.qian@nxp.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-ClientProxiedBy: SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.4 via Frontend Transport; Wed, 11 Aug 2021 08:44:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ad4813c1-36a6-45f7-0363-08d95ca436dc X-MS-TrafficTypeDiagnostic: AM5PR0401MB2546: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:94; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YKGUTlQpucSngP5xYMS8VRkD9ha0RDiTkF05+kIhrkZGKLZ+LxRXDxDURK2WujEU9WNV6i5INRDDyBargRy2VUz3RJc7K1FmzA6UvOcWqhd+aapRkteSo7LGtHS38d97QMTUyHZEE/S/vtBzh/cNbHL8alX1QvaQ0MNgz+LCDb5Q85MUC4fceLd1LMtcn0uUmKDEyYqqXvPysGniyQcr/n37AGXEsL4GGK3GDA/Tlsl6ImHcsFqGxl9U0iWZuil14zkU4XEll1ANf7oruVTHCZ8efIZOE5GW6TlFQ5yO5aruULzlTFwabmjtAQuDG6gC04QScyh//xXSDP44XNZb9Sl9uu7/V9cbQIFI3oNOJWCi1ep8WRljxCjUx3MdfOahyqgY1S8QbUnZL1OoqzyWzU3OXZ5MFFqpeQtSw6wNlJW1FoAYEug71M4e69O36M9VbXhCRUaK5b5IdnTCokRqJ9GVlTj2s4jT+ExPMX5E2KmgrTVf+9YdeVOkhHTwBR3aPCpJlmMjpUcHhZDleeHJ2Ty06otYI8ydhoZT8B1op3G2rnXRFMs3M92McsDr5zNbTJIbie2bpmKdikDEm8+JQLninDg2Vwg5+lXU0/yC4rqKBD0Ai5MHGFA+Zn67W9dgsJIhCV1AHcZH9x85ROvtkY0gX5Qx0HY+rePo/8zNs4SpUyeaXZQZW1gxG/Pfl39h5vb25Dup4GH7rZuEqESZ4jQ9mspdO8xaoTGa6aS5hps= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(346002)(39860400002)(376002)(136003)(83380400001)(6486002)(5660300002)(2616005)(6666004)(38350700002)(66946007)(38100700002)(7416002)(4326008)(2906002)(956004)(36756003)(66556008)(66476007)(15650500001)(8936002)(186003)(30864003)(7696005)(52116002)(44832011)(316002)(86362001)(478600001)(8676002)(26005)(32563001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CC7RXQUWv9rxfgsbljOqYH86Pdm6MwreI1WC9IyiQM0e1uMlhPhYd9Y/VQyY6bBzQyysS8w36sFMsjTwf6rdvI4siSDYFx6967K3fMuhyrMOAMoLdujV+/qGqiNubR85oc7cKqSiDNy4hPa7HKuOYzzKmWo8fbPtdmSCKgnmae395MsPZPgcDpWULPZ7LVoHcWD1zBec2rsqmsWmpc0KBc5sqy8HGivuvoaaDve2h0mvGCuXgRw8hDWNy6iM6XsZIH1s6yeuKFCMfRsELLnVDJ+OOUKUZ5VvC5kZjyPn4oEXgBFeosYdFkV5WRffkqAZJv+UEwXmS+pGUlVvpFkvaaWFQY4/HbuSl8jwFqNQUdTQQQHTCJoW1sQmjsBD3viN0uVbRgZpOuNGXaYxYwiRMlTckYdy46BQp5J1NC4Vm9Ox4profiedKLjWWml1cLFiW2g08gQTFmYiWAVEuTBK1PutAJyWGfnUT3WGl+3zmN0XUAxoy1YomycdNlMrMWooXOIQqTOkNG6si2P1ZcH9AO8gGynNHkDgh0/eoi/c/jGwPjSEhCAPwzlq3aqZEqmL5CYyZRPmbo9GnOXh+ve/2lu2RcQZ1n0r5LSYVJZn6FFlYCuGzDgMDoIYyc1hqjjoyPG8zmlQKWmNiXEmPpuHEwViXKDGMPqSmDjiBKpNVWspH5knSuD0GM0jP7iKCoruwof3xxwgcq5JM2TV6eAaT8hhrZRDjj726CZJ5KpA8AuVWeBHveeTOJdx5qlzeESpnASwppm59Ar80Gy8lOfdo03XZ4T10ClBkOxW8aHsi5K1bo7vUmJHE30q2GNnTwTG4ZKP4pD/OVpg3EjZWcMghtc9YO6S4nZMa9trUIJ80N6Z5KaRw/2xFgOedb5iVbaPSXFOMzdSvk4/6ZHUNxcLZ/l/gbp8+uSfJXneWxUDbT0vruqzodcolqYxT8szW4LIBgabxPM1cT3AHTmwqxLaSdLFORf6tTlVHYQ43a6xDdsg2rKmaz3V1EtLLFU+e2sTeld8iAnUCVqNGUFqC13JV3R4A2pNKHppBcLRli/cbLEWJFaZg+YqR3qf8PxrCuZch8mE5Mp2zsCf38Bux/fxnFk/9PbB7JPiNihk6ZAXubPy1bkEoUNhXGegrTiUgbPY/WlUjDbGJVIuI4f/WEF4Ehe9zDnbAZxQpwn7jHX4/9ke4FJ5/1hdvXMOa20LSrL8JhlNtvS4jVx1CKActeWF1ICPL0ScHchYfa4E7MCn31/MB5OG7zeCAqwfv5oopU5RSSUh3uHmDb5sSe5YRihLKcCsgLQpBfrnMFqHMQ9CTVSU05bf+yfJt+ZZCuAcSGhm X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: ad4813c1-36a6-45f7-0363-08d95ca436dc X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 08:44:21.6185 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tfNqwjoLM8iooqUGxY2qTosOZK6I06SDyPiBoSKv6emIUw9bTYAnM/zli6OSUCJE/Ci0yv58FzQhkZ5XRVUN5Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0401MB2546 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org driver use mailbox to communicate with vpu core. and there are a command buffer and a message buffer. driver will write commands to the command buffer, then trigger a vpu core interrupt vpu core will write messages to the message buffer, then trigger a cpu interrupt. Signed-off-by: Ming Qian Signed-off-by: Shijie Qin Signed-off-by: Zhou Peng --- drivers/media/platform/imx/vpu-8q/vpu_cmds.c | 438 +++++++++++++++++++ drivers/media/platform/imx/vpu-8q/vpu_cmds.h | 25 ++ drivers/media/platform/imx/vpu-8q/vpu_mbox.c | 126 ++++++ drivers/media/platform/imx/vpu-8q/vpu_mbox.h | 16 + drivers/media/platform/imx/vpu-8q/vpu_msgs.c | 412 +++++++++++++++++ drivers/media/platform/imx/vpu-8q/vpu_msgs.h | 14 + 6 files changed, 1031 insertions(+) create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_cmds.c create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_cmds.h create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_mbox.c create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_mbox.h create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_msgs.c create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_msgs.h diff --git a/drivers/media/platform/imx/vpu-8q/vpu_cmds.c b/drivers/media/platform/imx/vpu-8q/vpu_cmds.c new file mode 100644 index 000000000000..43dd32c2519d --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_cmds.c @@ -0,0 +1,438 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020-2021 NXP + */ + +#define TAG "CMD" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "vpu.h" +#include "vpu_defs.h" +#include "vpu_cmds.h" +#include "vpu_rpc.h" +#include "vpu_mbox.h" +#include "vpu_log.h" + +struct vpu_cmd_request { + u32 request; + u32 response; + u32 handled; +}; + +struct vpu_cmd_t { + struct list_head list; + u32 id; + struct vpu_cmd_request *request; + struct vpu_rpc_event *pkt; + unsigned long key; +}; + +static struct vpu_cmd_request vpu_cmd_requests[] = { + { + .request = VPU_CMD_ID_CONFIGURE_CODEC, + .response = VPU_MSG_ID_MEM_REQUEST, + .handled = 1, + }, + { + .request = VPU_CMD_ID_START, + .response = VPU_MSG_ID_START_DONE, + .handled = 0, + }, + { + .request = VPU_CMD_ID_STOP, + .response = VPU_MSG_ID_STOP_DONE, + .handled = 0, + }, + { + .request = VPU_CMD_ID_ABORT, + .response = VPU_MSG_ID_ABORT_DONE, + .handled = 0, + }, + { + .request = VPU_CMD_ID_RST_BUF, + .response = VPU_MSG_ID_BUF_RST, + .handled = 1, + }, +}; + +int vpu_cmd_send(struct vpu_core *core, struct vpu_rpc_event *pkt) +{ + int ret = 0; + + WARN_ON(!core || !pkt); + + ret = vpu_iface_send_cmd(core, pkt); + if (ret) + return ret; + + /*write cmd data to cmd buffer before trigger a cmd interrupt*/ + mb(); + vpu_mbox_send_type(core, COMMAND); + + return ret; +} + +struct vpu_cmd_t *vpu_alloc_cmd(struct vpu_inst *inst, u32 id, void *data) +{ + struct vpu_cmd_t *cmd; + int i; + int ret; + + cmd = vzalloc(sizeof(*cmd)); + if (!cmd) + return NULL; + + cmd->pkt = vzalloc(sizeof(*cmd->pkt)); + if (!cmd->pkt) { + vfree(cmd); + return NULL; + } + + cmd->id = id; + ret = vpu_iface_pack_cmd(inst->core, cmd->pkt, inst->id, id, data); + if (ret) { + inst_err(inst, "iface pack cmd(%d) fail\n", id); + vfree(cmd->pkt); + vfree(cmd); + return NULL; + } + for (i = 0; i < ARRAY_SIZE(vpu_cmd_requests); i++) { + if (vpu_cmd_requests[i].request == id) { + cmd->request = &vpu_cmd_requests[i]; + break; + } + } + + return cmd; +} + +void vpu_free_cmd(struct vpu_cmd_t *cmd) +{ + if (!cmd) + return; + if (cmd->pkt) + vfree(cmd->pkt); + vfree(cmd); +} + +int vpu_session_process_cmd(struct vpu_inst *inst, struct vpu_cmd_t *cmd) +{ + int ret; + + if (!inst || !cmd || !cmd->pkt) + return -EINVAL; + + inst_dbg(inst, LVL_CMD, "send cmd(%d)\n", cmd->id); + vpu_iface_pre_send_cmd(inst); + ret = vpu_cmd_send(inst->core, cmd->pkt); + if (!ret) { + vpu_iface_post_send_cmd(inst); + vpu_inst_record_flow(inst, cmd->id); + } else { + inst_err(inst, "iface send cmd(%d) fail\n", cmd->id); + } + + return ret; +} + +void vpu_process_cmd_request(struct vpu_inst *inst) +{ + struct vpu_cmd_t *cmd; + struct vpu_cmd_t *tmp; + + if (!inst || inst->pending) + return; + + list_for_each_entry_safe(cmd, tmp, &inst->cmd_q, list) { + list_del_init(&cmd->list); + if (vpu_session_process_cmd(inst, cmd)) + inst_err(inst, "process cmd(%d) fail\n", cmd->id); + if (cmd->request) { + inst->pending = (void *)cmd; + break; + } + vpu_free_cmd(cmd); + } +} + +int vpu_request_cmd(struct vpu_inst *inst, u32 id, void *data, + unsigned long *key, int *sync) +{ + struct vpu_core *core; + struct vpu_cmd_t *cmd; + + if (!inst || !inst->core) + return -EINVAL; + + core = inst->core; + cmd = vpu_alloc_cmd(inst, id, data); + if (!cmd) + return -ENOMEM; + + mutex_lock(&core->cmd_lock); + cmd->key = core->cmd_seq++; + if (key) + *key = cmd->key; + if (sync) + *sync = cmd->request ? true : false; + list_add_tail(&cmd->list, &inst->cmd_q); + vpu_process_cmd_request(inst); + mutex_unlock(&core->cmd_lock); + + return 0; +} + +void vpu_clear_pending(struct vpu_inst *inst) +{ + if (!inst || !inst->pending) + return; + + vpu_free_cmd(inst->pending); + wake_up_all(&inst->core->ack_wq); + inst->pending = NULL; +} + +bool vpu_check_response(struct vpu_cmd_t *cmd, u32 response, u32 handled) +{ + struct vpu_cmd_request *request; + + if (!cmd || !cmd->request) + return false; + + request = cmd->request; + if (request->response != response) + return false; + if (request->handled != handled) + return false; + + return true; +} + +int vpu_response_cmd(struct vpu_inst *inst, u32 response, u32 handled) +{ + struct vpu_core *core; + + if (!inst || !inst->core) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->cmd_lock); + if (vpu_check_response(inst->pending, response, handled)) + vpu_clear_pending(inst); + + vpu_process_cmd_request(inst); + mutex_unlock(&core->cmd_lock); + + return 0; +} + +void vpu_clear_request(struct vpu_inst *inst) +{ + struct vpu_cmd_t *cmd; + struct vpu_cmd_t *tmp; + + mutex_lock(&inst->core->cmd_lock); + if (inst->pending) + vpu_clear_pending(inst); + + list_for_each_entry_safe(cmd, tmp, &inst->cmd_q, list) { + list_del_init(&cmd->list); + vpu_free_cmd(cmd); + } + mutex_unlock(&inst->core->cmd_lock); +} + +static bool check_is_responsed(struct vpu_inst *inst, unsigned long key) +{ + struct vpu_core *core = inst->core; + struct vpu_cmd_t *cmd; + bool flag = true; + + mutex_lock(&core->cmd_lock); + cmd = inst->pending; + if (cmd && key == cmd->key) { + flag = false; + goto exit; + } + list_for_each_entry(cmd, &inst->cmd_q, list) { + if (key == cmd->key) { + flag = false; + break; + } + } +exit: + mutex_unlock(&core->cmd_lock); + + return flag; +} + +static int sync_session_response(struct vpu_inst *inst, unsigned long key) +{ + struct vpu_core *core; + long ret; + + if (!inst || !inst->core) + return -EINVAL; + + core = inst->core; + + call_vop(inst, wait_prepare); + ret = wait_event_timeout(core->ack_wq, + check_is_responsed(inst, key), + VPU_TIMEOUT); + call_vop(inst, wait_finish); + + if (!check_is_responsed(inst, key)) { + inst_err(inst, "sync session timeout\n"); + set_bit(inst->id, &core->hang_mask); + mutex_lock(&inst->core->cmd_lock); + vpu_clear_pending(inst); + mutex_unlock(&inst->core->cmd_lock); + return -EINVAL; + } + + return 0; +} + +int vpu_session_send_cmd(struct vpu_inst *inst, u32 id, void *data) +{ + struct vpu_core *core; + unsigned long key; + int sync = false; + int ret = -EINVAL; + + WARN_ON(!inst || !inst->core || inst->id < 0); + core = inst->core; + + ret = vpu_request_cmd(inst, id, data, &key, &sync); + if (!ret && sync) + ret = sync_session_response(inst, key); + + if (ret) + inst_err(inst, "send cmd(%d) fail\n", id); + + return ret; +} + +int vpu_session_configure_codec(struct vpu_inst *inst) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_CONFIGURE_CODEC, NULL); +} + +int vpu_session_start(struct vpu_inst *inst) +{ + inst_dbg(inst, LVL_FLOW, "send start cmd\n"); + + return vpu_session_send_cmd(inst, VPU_CMD_ID_START, NULL); +} + +int vpu_session_stop(struct vpu_inst *inst) +{ + inst_dbg(inst, LVL_FLOW, "send stop cmd\n"); + + return vpu_session_send_cmd(inst, VPU_CMD_ID_STOP, NULL); +} + +int vpu_session_encode_frame(struct vpu_inst *inst, s64 timestamp) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_FRAME_ENCODE, ×tamp); +} + +int vpu_session_alloc_fs(struct vpu_inst *inst, struct vpu_fs_info *fs) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_FS_ALLOC, fs); +} + +int vpu_session_release_fs(struct vpu_inst *inst, struct vpu_fs_info *fs) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_FS_RELEASE, fs); +} + +int vpu_session_abort(struct vpu_inst *inst) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_ABORT, NULL); +} + +int vpu_session_rst_buf(struct vpu_inst *inst) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_RST_BUF, NULL); +} + +int vpu_session_fill_timestamp(struct vpu_inst *inst, struct vpu_ts_info *info) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_TIMESTAMP, info); +} + +int vpu_session_update_parameters(struct vpu_inst *inst, void *arg) +{ + if (inst->type & VPU_CORE_TYPE_DEC) + vpu_iface_set_decode_params(inst, arg, 1); + else + vpu_iface_set_encode_params(inst, arg, 1); + + return vpu_session_send_cmd(inst, VPU_CMD_ID_UPDATE_PARAMETER, arg); +} + +int vpu_session_debug(struct vpu_inst *inst) +{ + return vpu_session_send_cmd(inst, VPU_CMD_ID_DEBUG, NULL); +} + +int vpu_core_snapshot(struct vpu_core *core) +{ + struct vpu_inst *inst; + int ret; + + WARN_ON(!core || list_empty(&core->instances)); + + inst = list_first_entry(&core->instances, struct vpu_inst, list); + + reinit_completion(&core->cmp); + ret = vpu_session_send_cmd(inst, VPU_CMD_ID_SNAPSHOT, NULL); + if (ret) + return ret; + ret = wait_for_completion_timeout(&core->cmp, VPU_TIMEOUT); + if (!ret) { + vpu_err("core[%d] snapshot timeout\n", core->id); + return -EINVAL; + } + + return 0; +} + +int vpu_core_sw_reset(struct vpu_core *core) +{ + struct vpu_rpc_event pkt; + int ret; + + WARN_ON(!core); + + memset(&pkt, 0, sizeof(pkt)); + vpu_iface_pack_cmd(core, &pkt, 0, VPU_CMD_ID_FIRM_RESET, NULL); + + reinit_completion(&core->cmp); + mutex_lock(&core->cmd_lock); + ret = vpu_cmd_send(core, &pkt); + mutex_unlock(&core->cmd_lock); + if (ret) + return ret; + ret = wait_for_completion_timeout(&core->cmp, VPU_TIMEOUT); + if (!ret) { + vpu_err("core[%d] sw reset timeout\n", core->id); + return -EINVAL; + } + + return 0; +} diff --git a/drivers/media/platform/imx/vpu-8q/vpu_cmds.h b/drivers/media/platform/imx/vpu-8q/vpu_cmds.h new file mode 100644 index 000000000000..e45f4a67ac37 --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_cmds.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2020-2021 NXP + */ + +#ifndef _IMX_VPU_CMDS_H +#define _IMX_VPU_CMDS_H + +int vpu_session_configure_codec(struct vpu_inst *inst); +int vpu_session_start(struct vpu_inst *inst); +int vpu_session_stop(struct vpu_inst *inst); +int vpu_session_abort(struct vpu_inst *inst); +int vpu_session_rst_buf(struct vpu_inst *inst); +int vpu_session_encode_frame(struct vpu_inst *inst, s64 timestamp); +int vpu_session_alloc_fs(struct vpu_inst *inst, struct vpu_fs_info *fs); +int vpu_session_release_fs(struct vpu_inst *inst, struct vpu_fs_info *fs); +int vpu_session_fill_timestamp(struct vpu_inst *inst, struct vpu_ts_info *info); +int vpu_session_update_parameters(struct vpu_inst *inst, void *arg); +int vpu_core_snapshot(struct vpu_core *core); +int vpu_core_sw_reset(struct vpu_core *core); +int vpu_response_cmd(struct vpu_inst *inst, u32 response, u32 handled); +void vpu_clear_request(struct vpu_inst *inst); +int vpu_session_debug(struct vpu_inst *inst); + +#endif diff --git a/drivers/media/platform/imx/vpu-8q/vpu_mbox.c b/drivers/media/platform/imx/vpu-8q/vpu_mbox.c new file mode 100644 index 000000000000..5f926837932f --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_mbox.c @@ -0,0 +1,126 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020-2021 NXP + */ + +#define TAG "MBOX" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "vpu.h" +#include "vpu_mbox.h" +#include "vpu_msgs.h" +#include "vpu_log.h" + +static void vpu_mbox_rx_callback(struct mbox_client *cl, void *msg) +{ + struct vpu_mbox *rx = container_of(cl, struct vpu_mbox, cl); + struct vpu_core *core = container_of(rx, struct vpu_core, rx); + + vpu_isr(core, *(u32 *)msg); +} + +static int vpu_mbox_request_channel(struct device *dev, struct vpu_mbox *mbox) +{ + struct mbox_chan *ch; + struct mbox_client *cl; + + if (!dev || !mbox) + return -EINVAL; + if (mbox->ch) + return 0; + + cl = &mbox->cl; + cl->dev = dev; + if (mbox->block) { + cl->tx_block = true; + cl->tx_tout = 1000; + } else { + cl->tx_block = false; + } + cl->knows_txdone = false; + cl->rx_callback = vpu_mbox_rx_callback; + + ch = mbox_request_channel_byname(cl, mbox->name); + if (IS_ERR(ch)) { + vpu_err("Failed to request mbox chan %s, ret : %ld\n", + mbox->name, PTR_ERR(ch)); + return PTR_ERR(ch); + } + + mbox->ch = ch; + return 0; +} + +int vpu_mbox_init(struct vpu_core *core) +{ + WARN_ON(!core); + + scnprintf(core->tx_type.name, sizeof(core->tx_type.name) - 1, "tx0"); + core->tx_type.block = true; + + scnprintf(core->tx_data.name, sizeof(core->tx_data.name) - 1, "tx1"); + core->tx_data.block = false; + + scnprintf(core->rx.name, sizeof(core->rx.name) - 1, "rx"); + core->rx.block = true; + + return 0; +} + +int vpu_mbox_request(struct vpu_core *core) +{ + int ret; + + WARN_ON(!core); + + ret = vpu_mbox_request_channel(core->dev, &core->tx_type); + if (ret) + goto error; + ret = vpu_mbox_request_channel(core->dev, &core->tx_data); + if (ret) + goto error; + ret = vpu_mbox_request_channel(core->dev, &core->rx); + if (ret) + goto error; + + vpu_dbg(LVL_DEBUG, "mbox request\n"); + return 0; +error: + vpu_mbox_free(core); + return ret; +} + +void vpu_mbox_free(struct vpu_core *core) +{ + WARN_ON(!core); + + mbox_free_channel(core->tx_type.ch); + mbox_free_channel(core->tx_data.ch); + mbox_free_channel(core->rx.ch); + core->tx_type.ch = NULL; + core->tx_data.ch = NULL; + core->rx.ch = NULL; + vpu_dbg(LVL_DEBUG, "mbox free\n"); +} + +void vpu_mbox_send_type(struct vpu_core *core, u32 type) +{ + mbox_send_message(core->tx_type.ch, &type); +} + +void vpu_mbox_send_msg(struct vpu_core *core, u32 type, u32 data) +{ + mbox_send_message(core->tx_data.ch, &data); + mbox_send_message(core->tx_type.ch, &type); +} + +void vpu_mbox_enable_rx(struct vpu_dev *dev) +{ +} diff --git a/drivers/media/platform/imx/vpu-8q/vpu_mbox.h b/drivers/media/platform/imx/vpu-8q/vpu_mbox.h new file mode 100644 index 000000000000..3893ad3a4f7a --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_mbox.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2020-2021 NXP + */ + +#ifndef _IMX_VPU_MBOX_H +#define _IMX_VPU_MBOX_H + +int vpu_mbox_init(struct vpu_core *core); +int vpu_mbox_request(struct vpu_core *core); +void vpu_mbox_free(struct vpu_core *core); +void vpu_mbox_send_msg(struct vpu_core *core, u32 type, u32 data); +void vpu_mbox_send_type(struct vpu_core *core, u32 type); +void vpu_mbox_enable_rx(struct vpu_dev *dev); + +#endif diff --git a/drivers/media/platform/imx/vpu-8q/vpu_msgs.c b/drivers/media/platform/imx/vpu-8q/vpu_msgs.c new file mode 100644 index 000000000000..d01150758f14 --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_msgs.c @@ -0,0 +1,412 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020-2021 NXP + */ + +#define TAG "MSG" + +#include +#include +#include +#include +#include +#include +#include "vpu.h" +#include "vpu_core.h" +#include "vpu_rpc.h" +#include "vpu_mbox.h" +#include "vpu_defs.h" +#include "vpu_cmds.h" +#include "vpu_v4l2.h" +#include "vpu_log.h" + +#define VPU_PKT_HEADER_LENGTH 3 + +struct vpu_msg_handler { + u32 id; + void (*done)(struct vpu_inst *inst, struct vpu_rpc_event *pkt); +}; + +void vpu_session_handle_start_done(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + WARN_ON(!inst || !inst->core); + + inst_dbg(inst, LVL_FLOW, "start done\n"); +} + +void vpu_session_handle_mem_request(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + struct vpu_pkt_mem_req_data req_data; + + WARN_ON(!inst || !inst->core || !inst->ops); + + vpu_iface_unpack_msg_data(inst->core, pkt, (void *)&req_data); + inst_dbg(inst, LVL_FLOW, "mem request event : %d:%d %d:%d %d:%d\n", + req_data.enc_frame_size, + req_data.enc_frame_num, + req_data.ref_frame_size, + req_data.ref_frame_num, + req_data.act_buf_size, + req_data.act_buf_num); + call_vop(inst, mem_request, + req_data.enc_frame_size, + req_data.enc_frame_num, + req_data.ref_frame_size, + req_data.ref_frame_num, + req_data.act_buf_size, + req_data.act_buf_num); +} + +void vpu_session_handle_stop_done(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + WARN_ON(!inst || !inst->core); + + inst_dbg(inst, LVL_FLOW, "stop done\n"); + + call_vop(inst, stop_done); +} + +void vpu_session_handle_seq_hdr(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + struct vpu_dec_codec_info info; + + WARN_ON(!inst || !inst->core); + + info.stride = inst->core->res->stride; + vpu_iface_unpack_msg_data(inst->core, pkt, (void *)&info); + call_vop(inst, event_notify, VPU_MSG_ID_SEQ_HDR_FOUND, &info); +} + +void vpu_session_handle_resolution_change(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + WARN_ON(!inst || !inst->core); + + call_vop(inst, event_notify, VPU_MSG_ID_RES_CHANGE, NULL); +} + +void vpu_session_handle_enc_frame_done(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + struct vpu_enc_pic_info info; + + WARN_ON(!inst || !inst->core); + + vpu_iface_unpack_msg_data(inst->core, pkt, (void *)&info); + inst_dbg(inst, LVL_DEBUG, "frame id = %d, wptr = 0x%x, size = %d\n", + info.frame_id, info.wptr, info.frame_size); + call_vop(inst, get_one_frame, &info); +} + +void vpu_session_handle_frame_request(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + struct vpu_fs_info fs; + + vpu_iface_unpack_msg_data(inst->core, pkt, &fs); + call_vop(inst, event_notify, VPU_MSG_ID_FRAME_REQ, &fs); +} + +void vpu_session_handle_frame_release(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + + WARN_ON(!inst || !inst->core); + + if (inst->core->type == VPU_CORE_TYPE_ENC) { + struct vpu_frame_info info; + + memset(&info, 0, sizeof(info)); + vpu_iface_unpack_msg_data(inst->core, pkt, (void *)&info.sequence); + inst_dbg(inst, LVL_DEBUG, "frame release:%d\n", info.sequence); + info.type = inst->out_format.type; + call_vop(inst, buf_done, &info); + } else if (inst->core->type == VPU_CORE_TYPE_DEC) { + struct vpu_fs_info fs; + + vpu_iface_unpack_msg_data(inst->core, pkt, &fs); + call_vop(inst, event_notify, VPU_MSG_ID_FRAME_RELEASE, &fs); + } +} + +void vpu_session_handle_input_done(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + + WARN_ON(!inst || !inst->core); + + inst_dbg(inst, LVL_DEBUG, "frame input done\n"); + call_vop(inst, input_done); +} + +void vpu_session_handle_pic_decoded(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + struct vpu_dec_pic_info info; + + WARN_ON(!inst || !inst->core); + + vpu_iface_unpack_msg_data(inst->core, pkt, (void *)&info); + call_vop(inst, get_one_frame, &info); +} + +void vpu_session_handle_pic_done(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + struct vpu_dec_pic_info info; + struct vpu_frame_info frame; + + WARN_ON(!inst || !inst->core); + + memset(&frame, 0, sizeof(frame)); + vpu_iface_unpack_msg_data(inst->core, pkt, (void *)&info); + if (inst->core->type == VPU_CORE_TYPE_DEC) + frame.type = inst->cap_format.type; + frame.id = info.id; + frame.luma = info.luma; + frame.skipped = info.skipped; + frame.timestamp = info.timestamp; + + call_vop(inst, buf_done, &frame); +} + +void vpu_session_handle_eos(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + call_vop(inst, event_notify, VPU_MSG_ID_PIC_EOS, NULL); +} + +void vpu_session_handle_error(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + call_vop(inst, event_notify, VPU_MSG_ID_UNSUPPORTED, NULL); + vpu_notify_codec_error(inst); +} + +void vpu_session_handle_firmware_xcpt(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + char *str = (char *)pkt->data; + + inst_err(inst, "%s firmware xcpt: %s\n", + vpu_core_type_desc(inst->core->type), str); + call_vop(inst, event_notify, VPU_MSG_ID_FIRMWARE_XCPT, NULL); + set_bit(inst->id, &inst->core->hang_mask); + vpu_notify_codec_error(inst); +} + +struct vpu_msg_handler handlers[] = { + {VPU_MSG_ID_START_DONE, vpu_session_handle_start_done}, + {VPU_MSG_ID_STOP_DONE, vpu_session_handle_stop_done}, + {VPU_MSG_ID_MEM_REQUEST, vpu_session_handle_mem_request}, + {VPU_MSG_ID_SEQ_HDR_FOUND, vpu_session_handle_seq_hdr}, + {VPU_MSG_ID_RES_CHANGE, vpu_session_handle_resolution_change}, + {VPU_MSG_ID_FRAME_INPUT_DONE, vpu_session_handle_input_done}, + {VPU_MSG_ID_FRAME_REQ, vpu_session_handle_frame_request}, + {VPU_MSG_ID_FRAME_RELEASE, vpu_session_handle_frame_release}, + {VPU_MSG_ID_ENC_DONE, vpu_session_handle_enc_frame_done}, + {VPU_MSG_ID_PIC_DECODED, vpu_session_handle_pic_decoded}, + {VPU_MSG_ID_DEC_DONE, vpu_session_handle_pic_done}, + {VPU_MSG_ID_PIC_EOS, vpu_session_handle_eos}, + {VPU_MSG_ID_UNSUPPORTED, vpu_session_handle_error}, + {VPU_MSG_ID_FIRMWARE_XCPT, vpu_session_handle_firmware_xcpt}, +}; + +int vpu_session_handle_msg(struct vpu_inst *inst, struct vpu_rpc_event *msg) +{ + int ret; + u32 msg_id; + struct vpu_msg_handler *handler = NULL; + unsigned int i; + + ret = vpu_iface_convert_msg_id(inst->core, msg->hdr.id); + if (ret < 0) + return -EINVAL; + + msg_id = ret; + inst_dbg(inst, LVL_EVT, "receive event(%d)\n", msg_id); + + for (i = 0; i < ARRAY_SIZE(handlers); i++) { + if (handlers[i].id == msg_id) { + handler = &handlers[i]; + break; + } + } + + if (handler && handler->done) + handler->done(inst, msg); + + vpu_response_cmd(inst, msg_id, 1); + + return 0; +} + +static bool vpu_inst_receive_msg(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + u32 bytes = sizeof(struct vpu_rpc_event_header); + u32 ret; + + memset(pkt, 0, sizeof(*pkt)); + if (kfifo_len(&inst->msg_fifo) < bytes) + return false; + + ret = kfifo_out(&inst->msg_fifo, pkt, bytes); + if (ret != bytes) + return false; + + if (pkt->hdr.num > 0) { + bytes = pkt->hdr.num * sizeof(u32); + ret = kfifo_out(&inst->msg_fifo, pkt->data, bytes); + if (ret != bytes) + return false; + } + + return true; +} + +void vpu_inst_run_work(struct work_struct *work) +{ + struct vpu_inst *inst = container_of(work, struct vpu_inst, msg_work); + struct vpu_rpc_event pkt; + + while (vpu_inst_receive_msg(inst, &pkt)) + vpu_session_handle_msg(inst, &pkt); +} + +void vpu_inst_handle_msg(struct vpu_inst *inst, struct vpu_rpc_event *pkt) +{ + u32 bytes; + u32 id = pkt->hdr.id; + int ret; + + if (!inst->workqueue) { + vpu_session_handle_msg(inst, pkt); + return; + } + + bytes = sizeof(pkt->hdr) + pkt->hdr.num * sizeof(u32); + ret = kfifo_in(&inst->msg_fifo, pkt, bytes); + if (ret != bytes) + vpu_err("[%d:%d]overflow: %d\n", inst->core->id, inst->id, id); + queue_work(inst->workqueue, &inst->msg_work); +} + +int vpu_handle_msg(struct vpu_core *core) +{ + struct vpu_rpc_event pkt; + struct vpu_inst *inst; + int ret; + + memset(&pkt, 0, sizeof(pkt)); + while (!vpu_iface_receive_msg(core, &pkt)) { + vpu_dbg(LVL_DEBUG, "event index = %d, id = %d, num = %d\n", + pkt.hdr.index, pkt.hdr.id, pkt.hdr.num); + + ret = vpu_iface_convert_msg_id(core, pkt.hdr.id); + if (ret < 0) + continue; + + inst = vpu_core_find_instance(core, pkt.hdr.index); + if (inst) { + vpu_response_cmd(inst, ret, 0); + mutex_lock(&core->cmd_lock); + vpu_inst_record_flow(inst, ret); + mutex_unlock(&core->cmd_lock); + + vpu_inst_handle_msg(inst, &pkt); + vpu_inst_put(inst); + } + memset(&pkt, 0, sizeof(pkt)); + } + + return 0; +} + +int vpu_isr_thread(struct vpu_core *core, u32 irq_code) +{ + WARN_ON(!core); + + vpu_dbg(LVL_DEBUG, "core[%d] irq code = 0x%x\n", core->id, irq_code); + switch (irq_code) { + case VPU_IRQ_CODE_SYNC: + vpu_mbox_send_msg(core, PRC_BUF_OFFSET, core->rpc.phys - core->fw.phys); + vpu_mbox_send_msg(core, BOOT_ADDRESS, core->fw.phys); + vpu_mbox_send_msg(core, INIT_DONE, 2); + break; + case VPU_IRQ_CODE_BOOT_DONE: + break; + case VPU_IRQ_CODE_SNAPSHOT_DONE: + break; + default: + vpu_handle_msg(core); + break; + } + + return 0; +} + +static void vpu_core_run_msg_work(struct vpu_core *core) +{ + const unsigned int SIZE = sizeof(u32); + + while (kfifo_len(&core->msg_fifo) >= SIZE) { + u32 data; + + if (kfifo_out(&core->msg_fifo, &data, SIZE) == SIZE) + vpu_isr_thread(core, data); + } +} + +void vpu_msg_run_work(struct work_struct *work) +{ + struct vpu_core *core = container_of(work, struct vpu_core, msg_work); + unsigned long delay = msecs_to_jiffies(10); + + vpu_core_run_msg_work(core); + queue_delayed_work(core->workqueue, &core->msg_delayed_work, delay); +} + +void vpu_msg_delayed_work(struct work_struct *work) +{ + struct vpu_core *core; + struct delayed_work *dwork; + u32 bytes = sizeof(bytes); + u32 i; + + if (!work) + return; + + dwork = to_delayed_work(work); + core = container_of(dwork, struct vpu_core, msg_delayed_work); + if (kfifo_len(&core->msg_fifo) >= bytes) + vpu_core_run_msg_work(core); + + bytes = sizeof(struct vpu_rpc_event_header); + for (i = 0; i < core->supported_instance_count; i++) { + struct vpu_inst *inst = vpu_core_find_instance(core, i); + + if (!inst) + continue; + + if (inst->workqueue && kfifo_len(&inst->msg_fifo) >= bytes) + queue_work(inst->workqueue, &inst->msg_work); + + vpu_inst_put(inst); + } +} + + +int vpu_isr(struct vpu_core *core, u32 irq) +{ + WARN_ON(!core); + + vpu_dbg(LVL_DEBUG, "core[%d] irq code = 0x%x\n", core->id, irq); + switch (irq) { + case VPU_IRQ_CODE_SYNC: + break; + case VPU_IRQ_CODE_BOOT_DONE: + complete(&core->cmp); + break; + case VPU_IRQ_CODE_SNAPSHOT_DONE: + complete(&core->cmp); + break; + default: + break; + } + + if (kfifo_in(&core->msg_fifo, &irq, sizeof(irq)) != sizeof(irq)) + vpu_err("[%d]overflow: %d\n", core->id, irq); + queue_work(core->workqueue, &core->msg_work); + + return 0; +} diff --git a/drivers/media/platform/imx/vpu-8q/vpu_msgs.h b/drivers/media/platform/imx/vpu-8q/vpu_msgs.h new file mode 100644 index 000000000000..fa8117cffa5a --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_msgs.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2020-2021 NXP + */ + +#ifndef _IMX_VPU_MSGS_H +#define _IMX_VPU_MSGS_H + +int vpu_isr(struct vpu_core *core, u32 irq); +void vpu_inst_run_work(struct work_struct *work); +void vpu_msg_run_work(struct work_struct *work); +void vpu_msg_delayed_work(struct work_struct *work); + +#endif From patchwork Wed Aug 11 08:43:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 495316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57CBFC4338F for ; Wed, 11 Aug 2021 08:45:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2FD4C60F55 for ; Wed, 11 Aug 2021 08:45:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236526AbhHKIpY (ORCPT ); Wed, 11 Aug 2021 04:45:24 -0400 Received: from mail-eopbgr80048.outbound.protection.outlook.com ([40.107.8.48]:49541 "EHLO EUR04-VI1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236342AbhHKIpP (ORCPT ); Wed, 11 Aug 2021 04:45:15 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FKr7K3IKUcT7yEBKsuFmY52fLns1aKXkrrzVjs0xFy8uazWxSpOqjefnbIeXo9gs7Ga/NEQdMmyNvKv8z3Im9VhpOPDlMWTNaHwQChOARRvobGytH6eTTxg0Uyp9WwX1DgVDG2DFlEA8Dyr7974y3165h3Y71ELeVExcebml4WInbvq/KOOItEnlN5KXWuFnCETb3vB3Tdm3vYUx+p5Jfj/AaggTSyimjl8bReGAL3Dc9BXTntjYON3VfcBU6jFP0pGYyCmm1VaBmHWd3XbwsmH33LxdDHwbFgjf4R0VQ16jgRwVkWnaGqCLj3MtOB9BI3haMSxnP8QsAEClvKDhnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=frjSyKi5+BO6Mg8vGl+5BUPe2fuO/YE9pSPdElkPVl4=; b=WfwNQm6IYHfGI/vhQc81hYYfg6k/NfgWgZKamghaUzvAs2VNnpx88nO/aYwhB3b6npN40qQ3YUGZGiTWZRA+7xijaV5MtZsYBGlfJ+4kGyFgulqsK9YdDMEOuJiHRbFvtkEI8WeSFkqsAZ6k9PLX+wbzB+oiqCQ46pYHyHX/YlLGJgIiBECDapuRSwM/le7niVSttIyxpucjntAkb78OxgsoeVSjE7SJ1UAQMdBEMmh+PAzhGT/VDkpDESS75XHz35pMC6HX0U7HJyKOdxfWkVvwKEBgdcWMMlNbXZN/Lb3lRNUJ6PeR6UzGAuJDliPvAtBBB9gfFHFcRCzy/WAXWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=frjSyKi5+BO6Mg8vGl+5BUPe2fuO/YE9pSPdElkPVl4=; b=pFD/YFzG81CqDHABT4BsBTdLMbFI+UBlW7n8PWexdMKkJV7u3HawXoiqDG5XuC4l8O5TAKB21Osz3TSWyAl6qVLjynctfOqAQD2t12K8DnhojdZYS1T2amHbzTpaxPCiF9l00flfNRqpuSVAnXC5UaRegRvNwu9doVvhasUhqAU= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM6PR04MB4888.eurprd04.prod.outlook.com (2603:10a6:20b:12::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.17; Wed, 11 Aug 2021 08:44:29 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384%6]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 08:44:29 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de Cc: hverkuil-cisco@xs4all.nl, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 08/14] media: imx: imx8q: add v4l2 m2m vpu encoder stateful driver Date: Wed, 11 Aug 2021 16:43:04 +0800 Message-Id: <32aea4740a4e15fbd52be433d5ca9b35e7c1913f.1628671163.git.ming.qian@nxp.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-ClientProxiedBy: SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.4 via Frontend Transport; Wed, 11 Aug 2021 08:44:26 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 61df4dfd-a241-4de0-2d4c-08d95ca43bb2 X-MS-TrafficTypeDiagnostic: AM6PR04MB4888: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:272; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SAODvwywVZLgjFLvK/LJWURZ3kXHeIEWBniX9om+ISS0gVXSLCx/n96mX7LLXunqClxE4ksKRNw5Nh27fbWGqVl5mDLlcB2ruRzq/OJ92vKcGk7tHZj6vlc0Ilu5sG6wBJ2Pg+Qy68uvawUXIOO89l4LQ2W4av6RVVtO4JtPezMKWGxjax0SSaHi7iNh2Ym5fpORpewBfYBCsiABCKo097PIG9XTfW9wa7EogcNs7ffoCe/CmRwW6oclvz2aoNOl/PxftQruN0nTK54LoT2facSp1l9Rv19NtqBW5hVDsCkxZ/AT5W8OnX5RjZhz/UHu/Vgd9INCFpgaeY46e/p9leNnMaEE/1+A3S79mp1zE0o0YBf3JWcDmGGucS4bPPonajVhAXts3s35PDOIBgLghbTVmX5xJqTqT6uRv1BA2+wyq8amtjExAJpCiCu1wbavV64qJyy5MugPm7sJLpd5S/MnyQ2ZWMs8BoDZ9x8fRNYlh4SooQt/ioQjaqsGxpBf+4J+xzQwbhLqz7vZkT9vCchEBRkX4R+/OW9SMDEfJJ9rgMQcnrCWe11b7iZ2nXOOmvJJb9MUYc3AFrelnVcfVnHDbTcHtMxbvcP3dew9SmSc8mLa357ma3cmHOHYccf15oTDkJIgWcB73reLcC1RT/lI6scx5cqv1TsTS6qmD1yAhvWBWClIk6v1dO/OPtHXhh/6VrpZbYyv+vhSmKyKY9JsbAnnSh0xJeukOqSCiostg7g4Dtsg0adVv466hDrd X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(376002)(366004)(396003)(136003)(39860400002)(7416002)(86362001)(8676002)(6486002)(36756003)(44832011)(2616005)(956004)(4326008)(7696005)(2906002)(38100700002)(66476007)(478600001)(26005)(52116002)(66556008)(66946007)(38350700002)(186003)(8936002)(316002)(6666004)(5660300002)(83380400001)(30864003)(461764006)(32563001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: XUPgA3kdO/Fo9oHoMKcX93uAPr6E+MglQbc9Js2sUp30zi6aj43S2lsiWy9PdC1oGpsUriWklJfRuyyVwBk0IJ/2t4ps3HhtndoRyotUVv6tHWSN8tEtIe/Cf5hu/liAnzpwjlHDvCu0TooR4gZ+wjkstjx0MYeMrOw5751T0o3Ec+88de4Z4H7WG1fNZnVcuXVuoXlPnWk1Ybr7WOeWffEl9I9tsEPkCgAhP7SdEvQOw+A3jP9RsegJBv2tQEiyJtMqnGrOc2szhVfvohBS3yiWD4hP29IHh+o32mwXbFVnZrAkCwReeT7A7YC0Ydesoj1XpsSu+cq/Madn0gg5HIHYnBASMLPvu1ksoVvX+1rP++bHGXuZnuhIcThhc5i6dyclnjfYPv1TDJISDJF97S1kwJHuAW2Z6mZvbfItGywJXlHWpTGCv7z+O3avYLow5eGkihuS7CYf5JbKF0+tOlvgwZHeJJ/rI6BBaTSsImA2LCx12EF1Kk6s6MJkO+0ChoCwTUoThCQK4hrOzaBCPtXic13drRcAVoivsA1bsZj8vwxQrZZU7D2YInWOBJk7Xr3TExRaiRXpHYCNfKlUzbz90rPwwX0BGFKU+V+L6ziJYRTotV0TDSK3/HwXWpUOMQMJBWEZ3FW6zlbejV61hA5eOH1SBjeLlZYCywhXWMdvfRXxMwqaVqg2oegn+PHwF2Vfe9wq6Vd/QyPvK1eeKRma1MksOvbjmZLn8O9cHp0clnfOHj1giaOH9fXK2fsw+Ju9ROFnuO1JySUQAx/jLEh3FLclPUceW76LU+hu9Rba1tkwY6Vbf6jzlyf4UW5PJwhAUv+dlGGib7UjrIlIpxL7q6xsbTmux/yIoYLIcwH7PKDCG4VsEDsG6/bbWGJFRWTfbMNTI7yQ4EpsIl7NW1M1WHDE/G9ho6lmt8Dcqx2xV95APnIXzEScNxbjR/gxa3NKcWpHqAY3+z/XqAFJZFHz1DagByH8dk2W1fRvLa9DihgkRUF6df+5syDC0tzqLypIemHp+7n2KESxBnOJDocNox1wTZ7pH0Gdgg9cG5vtA2vAG8diQviAThrh8jN+LCKVE6uOlrYFLRWs0FPgTDePi+8XFPBOmpu43MzkQJRU3hmXBf2quT+hs4a1P7SKwLxF3rSAsRqW6Kb9/stCRKwupThJVgn5ZauYKOYhx4sAQNdyxOS8LLEoqNbZJ95cJCa0cEq+FRwwT1auCOhXfLA6T75NL3vKddbKm20hLyFhBaE4i4F6VwWFqrU989cf5jjJ/C/XFSZmI9JdNkAFlfpsaIIMYoSSavYwAKkiAFY9q2DqIHJ5YAwVy5PiaDT0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 61df4dfd-a241-4de0-2d4c-08d95ca43bb2 X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 08:44:29.7509 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tHYall3S25nH05z9SzC9s793tyPoidGbvO/ywjzJa7VzueQL8kjUyiBY+k03sH37fIIrJZr+8VL2KmDRFlrZDQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB4888 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org This consists of video encoder implementation plus encoder controls. Signed-off-by: Ming Qian Signed-off-by: Shijie Qin Signed-off-by: Zhou Peng --- drivers/media/platform/imx/vpu-8q/venc.c | 1383 ++++++++++++++++++++++ 1 file changed, 1383 insertions(+) create mode 100644 drivers/media/platform/imx/vpu-8q/venc.c diff --git a/drivers/media/platform/imx/vpu-8q/venc.c b/drivers/media/platform/imx/vpu-8q/venc.c new file mode 100644 index 000000000000..d210daed38fa --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/venc.c @@ -0,0 +1,1383 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020-2021 NXP + */ + +#define TAG "ENC" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "vpu.h" +#include "vpu_defs.h" +#include "vpu_core.h" +#include "vpu_helpers.h" +#include "vpu_v4l2.h" +#include "vpu_cmds.h" +#include "vpu_rpc.h" +#include "vpu_log.h" + +#define VENC_OUTPUT_ENABLE (1 << 0) +#define VENC_CAPTURE_ENABLE (1 << 1) +#define VENC_ENABLE_MASK (VENC_OUTPUT_ENABLE | VENC_CAPTURE_ENABLE) +#define VENC_MAX_BUF_CNT 8 + +struct venc_t { + struct vpu_encode_params params; + u32 request_key_frame; + u32 input_ready; + u32 cpb_size; + bool bitrate_change; + + struct vpu_buffer enc[VENC_MAX_BUF_CNT]; + struct vpu_buffer ref[VENC_MAX_BUF_CNT]; + struct vpu_buffer act[VENC_MAX_BUF_CNT]; + struct list_head frames; + u32 frame_count; + u32 encode_count; + u32 ready_count; + u32 enable; + u32 stopped; + + u32 skipped_count; + u32 skipped_bytes; + + wait_queue_head_t wq; +}; + +struct venc_frame_t { + struct list_head list; + struct vpu_enc_pic_info info; + u32 bytesused; + s64 timestamp; +}; + +static const struct vpu_format venc_formats[] = { + { + .pixfmt = V4L2_PIX_FMT_NV12, + .num_planes = 2, + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, + }, + { + .pixfmt = V4L2_PIX_FMT_H264, + .num_planes = 1, + .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, + }, + {0, 0, 0, 0}, +}; + +static int venc_querycap(struct file *file, void *fh, struct v4l2_capability *cap) +{ + strscpy(cap->driver, "imx vpu encoder", sizeof(cap->driver)); + strscpy(cap->card, "imx vpu encoder", sizeof(cap->card)); + strscpy(cap->bus_info, "platform: imx8q-vpu", sizeof(cap->bus_info)); + + return 0; +} + +static int venc_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f) +{ + struct vpu_inst *inst = to_inst(file); + const struct vpu_format *fmt; + + memset(f->reserved, 0, sizeof(f->reserved)); + fmt = vpu_helper_enum_format(inst, f->type, f->index); + if (!fmt) + return -EINVAL; + + f->pixelformat = fmt->pixfmt; + f->flags = fmt->flags; + + return 0; +} + +static int venc_enum_framesizes(struct file *file, void *fh, struct v4l2_frmsizeenum *fsize) +{ + struct vpu_inst *inst = to_inst(file); + + if (!fsize || fsize->index) + return -EINVAL; + + if (!vpu_helper_find_format(inst, 0, fsize->pixel_format)) + return -EINVAL; + + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; + fsize->stepwise.max_width = inst->core->res->max_width; + fsize->stepwise.max_height = inst->core->res->max_height; + fsize->stepwise.min_width = inst->core->res->min_width; + fsize->stepwise.min_height = inst->core->res->min_height; + fsize->stepwise.step_width = inst->core->res->step_width; + fsize->stepwise.step_height = inst->core->res->step_height; + + return 0; +} + +static int venc_enum_frameintervals(struct file *file, void *fh, struct v4l2_frmivalenum *fival) +{ + struct vpu_inst *inst = to_inst(file); + + if (!fival || fival->index) + return -EINVAL; + + if (!vpu_helper_find_format(inst, 0, fival->pixel_format)) + return -EINVAL; + + if (!fival->width || !fival->height) + return -EINVAL; + + if (fival->width < inst->core->res->min_width || + fival->width > inst->core->res->max_width || + fival->height < inst->core->res->min_height || + fival->height > inst->core->res->max_height) + return -EINVAL; + + fival->type = V4L2_FRMIVAL_TYPE_CONTINUOUS; + fival->stepwise.min.numerator = 1; + fival->stepwise.min.denominator = USHRT_MAX; + fival->stepwise.max.numerator = USHRT_MAX; + fival->stepwise.max.denominator = 1; + fival->stepwise.step.numerator = 1; + fival->stepwise.step.denominator = 1; + + return 0; +} + +static int venc_g_fmt(struct file *file, void *fh, struct v4l2_format *f) +{ + struct vpu_inst *inst = to_inst(file); + struct venc_t *venc = inst->priv; + struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; + struct vpu_format *cur_fmt; + int i; + + cur_fmt = vpu_get_format(inst, f->type); + + pixmp->pixelformat = cur_fmt->pixfmt; + pixmp->num_planes = cur_fmt->num_planes; + pixmp->width = cur_fmt->width; + pixmp->height = cur_fmt->height; + pixmp->field = cur_fmt->field; + pixmp->flags = cur_fmt->flags; + for (i = 0; i < pixmp->num_planes; i++) { + pixmp->plane_fmt[i].bytesperline = cur_fmt->bytesperline[i]; + pixmp->plane_fmt[i].sizeimage = cur_fmt->sizeimage[i]; + } + + f->fmt.pix_mp.colorspace = venc->params.color.primaries; + f->fmt.pix_mp.xfer_func = venc->params.color.transfer; + f->fmt.pix_mp.ycbcr_enc = venc->params.color.matrix; + f->fmt.pix_mp.quantization = venc->params.color.full_range; + + return 0; +} + +static int venc_try_fmt(struct file *file, void *fh, struct v4l2_format *f) +{ + struct vpu_inst *inst = to_inst(file); + + vpu_try_fmt_common(inst, f); + + return 0; +} + +static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f) +{ + struct vpu_inst *inst = to_inst(file); + const struct vpu_format *fmt; + struct vpu_format *cur_fmt; + struct vb2_queue *q; + struct venc_t *venc = inst->priv; + struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; + int i; + + q = v4l2_m2m_get_vq(inst->m2m_ctx, f->type); + if (!q) + return -EINVAL; + if (vb2_is_streaming(q)) + return -EBUSY; + + fmt = vpu_try_fmt_common(inst, f); + if (!fmt) + return -EINVAL; + + cur_fmt = vpu_get_format(inst, f->type); + + cur_fmt->pixfmt = fmt->pixfmt; + cur_fmt->num_planes = fmt->num_planes; + cur_fmt->type = fmt->type; + cur_fmt->flags = fmt->flags; + cur_fmt->width = pix_mp->width; + cur_fmt->height = pix_mp->height; + for (i = 0; i < fmt->num_planes; i++) { + cur_fmt->sizeimage[i] = pix_mp->plane_fmt[i].sizeimage; + cur_fmt->bytesperline[i] = pix_mp->plane_fmt[i].bytesperline; + } + + if (pix_mp->field != V4L2_FIELD_ANY) + cur_fmt->field = pix_mp->field; + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + venc->params.input_format = cur_fmt->pixfmt; + venc->params.src_stride = cur_fmt->bytesperline[0]; + venc->params.src_width = cur_fmt->width; + venc->params.src_height = cur_fmt->height; + venc->params.crop.left = 0; + venc->params.crop.top = 0; + venc->params.crop.width = cur_fmt->width; + venc->params.crop.height = cur_fmt->height; + } else { + venc->params.codec_format = cur_fmt->pixfmt; + venc->params.out_width = cur_fmt->width; + venc->params.out_height = cur_fmt->height; + } + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + if (!vpu_color_check_primaries(pix_mp->colorspace)) { + venc->params.color.primaries = pix_mp->colorspace; + vpu_color_get_default(venc->params.color.primaries, + &venc->params.color.transfer, + &venc->params.color.matrix, + &venc->params.color.full_range); + } + if (!vpu_color_check_transfers(pix_mp->xfer_func)) + venc->params.color.transfer = pix_mp->xfer_func; + if (!vpu_color_check_matrix(pix_mp->ycbcr_enc)) + venc->params.color.matrix = pix_mp->ycbcr_enc; + if (!vpu_color_check_full_range(pix_mp->quantization)) + venc->params.color.full_range = pix_mp->quantization; + } + + pix_mp->colorspace = venc->params.color.primaries; + pix_mp->xfer_func = venc->params.color.transfer; + pix_mp->ycbcr_enc = venc->params.color.matrix; + pix_mp->quantization = venc->params.color.full_range; + + return 0; +} + +static int venc_g_parm(struct file *file, void *fh, struct v4l2_streamparm *parm) +{ + struct vpu_inst *inst = to_inst(file); + struct venc_t *venc = inst->priv; + struct v4l2_fract *timeperframe = &parm->parm.capture.timeperframe; + + if (!parm) + return -EINVAL; + + if (!vpu_helper_check_type(inst, parm->type)) + return -EINVAL; + + parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME; + parm->parm.capture.readbuffers = 0; + timeperframe->numerator = venc->params.frame_rate_num; + timeperframe->denominator = venc->params.frame_rate_den; + + return 0; +} + +static int venc_s_parm(struct file *file, void *fh, struct v4l2_streamparm *parm) +{ + struct vpu_inst *inst = to_inst(file); + struct venc_t *venc = inst->priv; + struct v4l2_fract *timeperframe = &parm->parm.capture.timeperframe; + + if (!parm) + return -EINVAL; + + if (!vpu_helper_check_type(inst, parm->type)) + return -EINVAL; + + if (!timeperframe->numerator) + timeperframe->numerator = venc->params.frame_rate_num; + if (!timeperframe->denominator) + timeperframe->denominator = venc->params.frame_rate_den; + + venc->params.frame_rate_num = timeperframe->numerator; + venc->params.frame_rate_den = timeperframe->denominator; + + vpu_helper_calc_coprime(&venc->params.frame_rate_num, + &venc->params.frame_rate_den); + + parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME; + memset(parm->parm.capture.reserved, + 0, sizeof(parm->parm.capture.reserved)); + + return 0; +} + +static int venc_g_selection(struct file *file, void *fh, struct v4l2_selection *s) +{ + struct vpu_inst *inst = to_inst(file); + struct venc_t *venc = inst->priv; + + if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + return -EINVAL; + + switch (s->target) { + case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP_BOUNDS: + s->r.left = 0; + s->r.top = 0; + s->r.width = inst->out_format.width; + s->r.height = inst->out_format.height; + break; + case V4L2_SEL_TGT_CROP: + s->r = venc->params.crop; + break; + default: + return -EINVAL; + } + + return 0; +} + +static int venc_valid_crop(struct venc_t *venc, struct vpu_core *core) +{ + struct v4l2_rect *rect = NULL; + u32 min_width; + u32 min_height; + u32 src_width; + u32 src_height; + + rect = &venc->params.crop; + min_width = core->res->min_width; + min_height = core->res->min_height; + src_width = venc->params.src_width; + src_height = venc->params.src_height; + + if (rect->width == 0 || rect->height == 0) + return -EINVAL; + if (rect->left > src_width - min_width || + rect->top > src_height - min_height) + return -EINVAL; + + rect->width = min(rect->width, src_width - rect->left); + rect->width = max_t(u32, rect->width, min_width); + + rect->height = min(rect->height, src_height - rect->top); + rect->height = max_t(u32, rect->height, min_height); + + return 0; +} + +static int venc_s_selection(struct file *file, void *fh, struct v4l2_selection *s) +{ + struct vpu_inst *inst = to_inst(file); + const struct vpu_core_resources *res = inst->core->res; + struct venc_t *venc = inst->priv; + + if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + return -EINVAL; + if (s->target != V4L2_SEL_TGT_CROP) + return -EINVAL; + + venc->params.crop.left = ALIGN(s->r.left, res->step_width); + venc->params.crop.top = ALIGN(s->r.top, res->step_height); + venc->params.crop.width = ALIGN(s->r.width, res->step_width); + venc->params.crop.height = ALIGN(s->r.height, res->step_height); + if (venc_valid_crop(venc, inst->core)) { + venc->params.crop.left = 0; + venc->params.crop.top = 0; + venc->params.crop.width = venc->params.src_width; + venc->params.crop.height = venc->params.src_height; + } + + inst->crop = venc->params.crop; + + return 0; +} + +static int venc_response_eos(struct vpu_inst *inst) +{ + struct venc_t *venc = inst->priv; + int ret; + + if (inst->state != VPU_CODEC_STATE_DRAIN) + return 0; + + if (v4l2_m2m_num_src_bufs_ready(inst->m2m_ctx)) + return 0; + + if (!venc->input_ready) + return 0; + + venc->input_ready = false; + inst_dbg(inst, LVL_FLOW, "stop\n"); + ret = vpu_session_stop(inst); + if (ret) + return ret; + inst->state = VPU_CODEC_STATE_STOP; + wake_up_all(&venc->wq); + + return 0; +} + +static int venc_request_eos(struct vpu_inst *inst) +{ + inst->state = VPU_CODEC_STATE_DRAIN; + venc_response_eos(inst); + + return 0; +} + +static int venc_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd *cmd) +{ + struct vpu_inst *inst = to_inst(file); + int ret; + + ret = v4l2_m2m_ioctl_try_encoder_cmd(file, fh, cmd); + if (ret) + return ret; + + vpu_inst_lock(inst); + if (cmd->cmd == V4L2_ENC_CMD_STOP) { + if (inst->state == VPU_CODEC_STATE_DEINIT) + vpu_notify_eos(inst); + else + venc_request_eos(inst); + } + vpu_inst_unlock(inst); + + return 0; +} + +static int venc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub) +{ + switch (sub->type) { + case V4L2_EVENT_EOS: + return v4l2_event_subscribe(fh, sub, 0, NULL); + case V4L2_EVENT_CTRL: + return v4l2_ctrl_subscribe_event(fh, sub); + default: + return -EINVAL; + } +} + +static const struct v4l2_ioctl_ops venc_ioctl_ops = { + .vidioc_querycap = venc_querycap, + .vidioc_enum_fmt_vid_cap = venc_enum_fmt, + .vidioc_enum_fmt_vid_out = venc_enum_fmt, + .vidioc_enum_framesizes = venc_enum_framesizes, + .vidioc_enum_frameintervals = venc_enum_frameintervals, + .vidioc_g_fmt_vid_cap_mplane = venc_g_fmt, + .vidioc_g_fmt_vid_out_mplane = venc_g_fmt, + .vidioc_try_fmt_vid_cap_mplane = venc_try_fmt, + .vidioc_try_fmt_vid_out_mplane = venc_try_fmt, + .vidioc_s_fmt_vid_cap_mplane = venc_s_fmt, + .vidioc_s_fmt_vid_out_mplane = venc_s_fmt, + .vidioc_g_parm = venc_g_parm, + .vidioc_s_parm = venc_s_parm, + .vidioc_g_selection = venc_g_selection, + .vidioc_s_selection = venc_s_selection, + .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd, + .vidioc_encoder_cmd = venc_encoder_cmd, + .vidioc_subscribe_event = venc_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, + .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querybuf = v4l2_m2m_ioctl_querybuf, + .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, + .vidioc_qbuf = v4l2_m2m_ioctl_qbuf, + .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, + .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, + .vidioc_streamon = v4l2_m2m_ioctl_streamon, + .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, +}; + +static int venc_op_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct vpu_inst *inst = ctrl_to_inst(ctrl); + struct venc_t *venc = inst->priv; + int ret = 0; + + vpu_inst_lock(inst); + switch (ctrl->id) { + case V4L2_CID_MPEG_VIDEO_H264_PROFILE: + venc->params.profile = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_H264_LEVEL: + venc->params.level = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_BITRATE_MODE: + venc->params.rc_mode = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_BITRATE: + if (ctrl->val != venc->params.bitrate) + venc->bitrate_change = true; + venc->params.bitrate = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_GOP_SIZE: + venc->params.gop_length = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_B_FRAMES: + venc->params.bframes = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP: + venc->params.i_frame_qp = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP: + venc->params.p_frame_qp = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP: + venc->params.b_frame_qp = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: + venc->request_key_frame = 1; + break; + case V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE: + venc->cpb_size = ctrl->val * 1024; + break; + case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE: + venc->params.sar.enable = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC: + venc->params.sar.idc = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_WIDTH: + venc->params.sar.width = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_HEIGHT: + venc->params.sar.height = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_HEADER_MODE: + break; + default: + ret = -EINVAL; + break; + } + vpu_inst_unlock(inst); + + return ret; +} + +static const struct v4l2_ctrl_ops venc_ctrl_ops = { + .s_ctrl = venc_op_s_ctrl, + .g_volatile_ctrl = vpu_helper_g_volatile_ctrl, +}; + +static int venc_ctrl_init(struct vpu_inst *inst) +{ + struct v4l2_ctrl *ctrl; + int ret; + + ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 20); + if (ret) + return ret; + + v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_PROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + ~((1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH)), + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH); + + v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LEVEL, + V4L2_MPEG_VIDEO_H264_LEVEL_5_1, + 0x0, + V4L2_MPEG_VIDEO_H264_LEVEL_4_0); + + v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_BITRATE_MODE, + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, + 0x0, + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR); + + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_BITRATE, + BITRATE_MIN, + BITRATE_MAX, + BITRATE_STEP, + BITRATE_DEFAULT); + + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_GOP_SIZE, 0, (1 << 16) - 1, 1, 30); + + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_B_FRAMES, 0, 4, 1, 0); + + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, 1, 51, 1, 26); + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, 1, 51, 1, 28); + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP, 1, 51, 1, 30); + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME, 0, 0, 0, 0); + ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MIN_BUFFERS_FOR_CAPTURE, 1, 32, 1, 2); + if (ctrl) + ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE; + ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MIN_BUFFERS_FOR_OUTPUT, 1, 32, 1, 2); + if (ctrl) + ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE; + + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE, 64, 10240, 1, 1024); + + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE, 0, 1, 1, 1); + v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC, + V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_EXTENDED, + 0x0, + V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_1x1); + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_WIDTH, + 0, USHRT_MAX, 1, 1); + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_HEIGHT, + 0, USHRT_MAX, 1, 1); + v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEADER_MODE, + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME, + ~(1 << V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME), + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME); + + ret = v4l2_ctrl_handler_setup(&inst->ctrl_handler); + if (ret) { + inst_err(inst, "setup ctrls fail, ret = %d\n", ret); + v4l2_ctrl_handler_free(&inst->ctrl_handler); + return ret; + } + + return 0; +} + +static bool venc_check_ready(struct vpu_inst *inst, unsigned int type) +{ + struct venc_t *venc = inst->priv; + + if (V4L2_TYPE_IS_OUTPUT(type)) { + if (vpu_helper_get_free_space(inst) < venc->cpb_size) + return false; + return venc->input_ready; + } + + return true; +} + +static u32 venc_get_enable_mask(u32 type) +{ + if (V4L2_TYPE_IS_OUTPUT(type)) + return VENC_OUTPUT_ENABLE; + else + return VENC_CAPTURE_ENABLE; +} + +static void venc_set_enable(struct venc_t *venc, u32 type, int enable) +{ + u32 mask = venc_get_enable_mask(type); + + if (enable) + venc->enable |= mask; + else + venc->enable &= ~mask; +} + +static u32 venc_get_enable(struct venc_t *venc, u32 type) +{ + return venc->enable & venc_get_enable_mask(type); +} + +static void venc_input_done(struct vpu_inst *inst) +{ + struct venc_t *venc = inst->priv; + + vpu_inst_lock(inst); + venc->input_ready = true; + vpu_process_output_buffer(inst); + venc_response_eos(inst); + vpu_inst_unlock(inst); +} + +/* + * It's hardware limitation, that there may be several bytes + * redundant data at the beginning of frame. + * For android platform, the redundant data may cause cts test fail + * So driver will strip them + */ +static int venc_precheck_encoded_frame(struct vpu_inst *inst, struct venc_frame_t *frame) +{ + struct venc_t *venc; + int skipped; + + if (!inst || !frame || !frame->bytesused) + return -EINVAL; + + venc = inst->priv; + skipped = vpu_helper_find_startcode(&inst->stream_buffer, + inst->cap_format.pixfmt, + frame->info.wptr - inst->stream_buffer.phys, + frame->bytesused); + if (skipped > 0) { + frame->bytesused -= skipped; + frame->info.wptr = vpu_helper_step_walk(&inst->stream_buffer, + frame->info.wptr, skipped); + venc->skipped_bytes += skipped; + venc->skipped_count++; + } + + return 0; +} + +static int venc_get_one_encoded_frame(struct vpu_inst *inst, + struct venc_frame_t *frame, + struct vb2_v4l2_buffer *vbuf) +{ + struct venc_t *venc = inst->priv; + struct vpu_vb2_buffer *vpu_buf; + + if (!vbuf) + return -EAGAIN; + + if (!venc_get_enable(inst->priv, vbuf->vb2_buf.type)) { + inst_dbg(inst, LVL_DEBUG, + "type %d is disabled, frame sequence %d\n", + vbuf->vb2_buf.type, frame->info.frame_id); + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + return 0; + } + vpu_buf = to_vpu_vb2_buffer(vbuf); + if (frame->bytesused > vbuf->vb2_buf.planes[0].length) { + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + return -ENOMEM; + } + + venc_precheck_encoded_frame(inst, frame); + + if (frame->bytesused) { + u32 rptr = frame->info.wptr; + void *dst = vb2_plane_vaddr(&vbuf->vb2_buf, 0); + + vpu_helper_copy_from_stream_buffer(&inst->stream_buffer, + &rptr, frame->bytesused, dst); + vpu_iface_update_stream_buffer(inst, rptr, 0); + } + vb2_set_plane_payload(&vbuf->vb2_buf, 0, frame->bytesused); + vbuf->sequence = frame->info.frame_id; + vbuf->vb2_buf.timestamp = frame->info.timestamp; + vbuf->flags |= frame->info.pic_type; + vpu_buf->state = VPU_BUF_STATE_IDLE; + inst_dbg(inst, LVL_TS, "[OUTPUT TS]%32lld\n", frame->info.timestamp); + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); + venc->ready_count++; + + if (vbuf->flags & V4L2_BUF_FLAG_KEYFRAME) + inst_dbg(inst, LVL_DEBUG, "[%d]Iframe\n", frame->info.frame_id); + + return 0; +} + +static int venc_get_encoded_frames(struct vpu_inst *inst) +{ + struct venc_t *venc; + struct venc_frame_t *frame; + struct venc_frame_t *tmp; + + if (!inst || !inst->priv) + return -EINVAL; + + venc = inst->priv; + list_for_each_entry_safe(frame, tmp, &venc->frames, list) { + if (venc_get_one_encoded_frame(inst, frame, + v4l2_m2m_dst_buf_remove(inst->m2m_ctx))) + break; + list_del_init(&frame->list); + vfree(frame); + } + + return 0; +} + +static int venc_frame_encoded(struct vpu_inst *inst, void *arg) +{ + struct vpu_enc_pic_info *info = arg; + struct venc_frame_t *frame; + struct venc_t *venc; + int ret = 0; + + if (!inst || !info) + return -EINVAL; + venc = inst->priv; + frame = vzalloc(sizeof(*frame)); + if (!frame) + return -ENOMEM; + + memcpy(&frame->info, info, sizeof(frame->info)); + frame->bytesused = info->frame_size; + + vpu_inst_lock(inst); + list_add_tail(&frame->list, &venc->frames); + venc->encode_count++; + venc_get_encoded_frames(inst); + vpu_inst_unlock(inst); + + return ret; +} + +static void venc_buf_done(struct vpu_inst *inst, struct vpu_frame_info *frame) +{ + struct vb2_v4l2_buffer *vbuf; + struct vpu_vb2_buffer *vpu_buf; + + if (!inst || !frame) + return; + + inst_dbg(inst, LVL_DEBUG, "buf done : type = %d, sequence = %d\n", + frame->type, frame->sequence); + + vpu_inst_lock(inst); + if (!venc_get_enable(inst->priv, frame->type)) { + inst_dbg(inst, LVL_DEBUG, + "type %d is disabled, frame sequence %d\n", + frame->type, frame->sequence); + goto exit; + } + vbuf = vpu_find_buf_by_sequence(inst, frame->type, frame->sequence); + if (!vbuf) { + inst_err(inst, "can't find buf: type %d, sequence %d\n", + frame->type, frame->sequence); + goto exit; + } + + vpu_buf = to_vpu_vb2_buffer(vbuf); + vpu_buf->state = VPU_BUF_STATE_IDLE; + if (V4L2_TYPE_IS_OUTPUT(frame->type)) + v4l2_m2m_src_buf_remove_by_buf(inst->m2m_ctx, vbuf); + else + v4l2_m2m_dst_buf_remove_by_buf(inst->m2m_ctx, vbuf); + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); +exit: + vpu_inst_unlock(inst); +} + +static int venc_append_empty_frame(struct vpu_inst *inst) +{ + struct venc_frame_t *frame; + struct venc_t *venc; + + if (!inst || !inst->priv) + return -EINVAL; + + venc = inst->priv; + frame = vzalloc(sizeof(*frame)); + if (!frame) + return -ENOMEM; + + frame->bytesused = 0; + frame->info.pic_type = V4L2_BUF_FLAG_LAST; + frame->info.frame_id = inst->sequence; + + list_add_tail(&frame->list, &venc->frames); + venc_get_encoded_frames(inst); + + return 0; +} + +static void venc_stop_done(struct vpu_inst *inst) +{ + struct venc_t *venc = inst->priv; + + inst_dbg(inst, LVL_FLOW, "append empty frame\n"); + + vpu_inst_lock(inst); + venc_append_empty_frame(inst); + venc->stopped = true; + vpu_inst_unlock(inst); + + wake_up_all(&venc->wq); +} + +static void venc_event_notify(struct vpu_inst *inst, u32 event, void *data) +{ +} + +static void venc_release(struct vpu_inst *inst) +{ +} + +static void venc_cleanup(struct vpu_inst *inst) +{ + struct venc_t *venc; + + if (!inst) + return; + + venc = inst->priv; + if (venc) + vfree(venc); + inst->priv = NULL; + vfree(inst); +} + +static int venc_start_session(struct vpu_inst *inst, u32 type) +{ + struct venc_t *venc = inst->priv; + int stream_buffer_size; + int ret; + + venc_set_enable(venc, type, 1); + if ((venc->enable & VENC_ENABLE_MASK) != VENC_ENABLE_MASK) + return 0; + + vpu_iface_init_instance(inst); + stream_buffer_size = vpu_iface_get_stream_buffer_size(inst->core); + if (stream_buffer_size > 0) { + inst->stream_buffer.length = max_t(u32, stream_buffer_size, venc->cpb_size * 3); + ret = vpu_alloc_dma(inst->core, &inst->stream_buffer); + if (ret) + goto error; + + inst->use_stream_buffer = true; + vpu_iface_config_stream_buffer(inst, &inst->stream_buffer); + } + + ret = vpu_iface_set_encode_params(inst, &venc->params, 0); + if (ret) + goto error; + ret = vpu_session_configure_codec(inst); + if (ret) + goto error; + + inst->state = VPU_CODEC_STATE_CONFIGURED; + /*vpu_iface_config_memory_resource*/ + + /*config enc expert mode parameter*/ + ret = vpu_iface_set_encode_params(inst, &venc->params, 1); + if (ret) + goto error; + + ret = vpu_session_start(inst); + if (ret) + goto error; + inst->state = VPU_CODEC_STATE_STARTED; + + venc->bitrate_change = false; + venc->input_ready = true; + venc->frame_count = 0; + venc->encode_count = 0; + venc->ready_count = 0; + venc->stopped = false; + vpu_process_output_buffer(inst); + if (venc->frame_count == 0) + inst_err(inst, "there is no input when starting\n"); + + return 0; +error: + venc_set_enable(venc, type, 0); + inst->state = VPU_CODEC_STATE_DEINIT; + + vpu_free_dma(&inst->stream_buffer); + return ret; +} + +static void venc_cleanup_mem_resource(struct vpu_inst *inst) +{ + struct venc_t *venc; + u32 i; + + WARN_ON(!inst || !inst->priv || !inst->core); + + venc = inst->priv; + + for (i = 0; i < ARRAY_SIZE(venc->enc); i++) + vpu_free_dma(&venc->enc[i]); + for (i = 0; i < ARRAY_SIZE(venc->ref); i++) + vpu_free_dma(&venc->ref[i]); + for (i = 0; i < ARRAY_SIZE(venc->act); i++) + vpu_free_dma(&venc->act[i]); +} + +static void venc_request_mem_resource(struct vpu_inst *inst, + u32 enc_frame_size, + u32 enc_frame_num, + u32 ref_frame_size, + u32 ref_frame_num, + u32 act_frame_size, + u32 act_frame_num) +{ + struct venc_t *venc; + u32 i; + int ret; + + WARN_ON(!inst || !inst->priv || !inst->core); + + venc = inst->priv; + + if (enc_frame_num > ARRAY_SIZE(venc->enc)) { + inst_err(inst, "enc num(%d) is out of range\n", enc_frame_num); + return; + } + if (ref_frame_num > ARRAY_SIZE(venc->ref)) { + inst_err(inst, "ref num(%d) is out of range\n", ref_frame_num); + return; + } + if (act_frame_num > ARRAY_SIZE(venc->act)) { + inst_err(inst, "act num(%d) is out of range\n", act_frame_num); + return; + } + + for (i = 0; i < enc_frame_num; i++) { + venc->enc[i].length = enc_frame_size; + ret = vpu_alloc_dma(inst->core, &venc->enc[i]); + if (ret) { + venc_cleanup_mem_resource(inst); + return; + } + } + for (i = 0; i < ref_frame_num; i++) { + venc->ref[i].length = ref_frame_size; + ret = vpu_alloc_dma(inst->core, &venc->ref[i]); + if (ret) { + venc_cleanup_mem_resource(inst); + return; + } + } + if (act_frame_num != 1 || act_frame_size > inst->act.length) { + venc_cleanup_mem_resource(inst); + return; + } + venc->act[0].length = act_frame_size; + venc->act[0].phys = inst->act.phys; + venc->act[0].virt = inst->act.virt; + + for (i = 0; i < enc_frame_num; i++) + vpu_iface_config_memory_resource(inst, MEM_RES_ENC, i, &venc->enc[i]); + for (i = 0; i < ref_frame_num; i++) + vpu_iface_config_memory_resource(inst, MEM_RES_REF, i, &venc->ref[i]); + for (i = 0; i < act_frame_num; i++) + vpu_iface_config_memory_resource(inst, MEM_RES_ACT, i, &venc->act[i]); +} + +static void venc_cleanup_frames(struct venc_t *venc) +{ + struct venc_frame_t *frame; + struct venc_frame_t *tmp; + + if (!list_empty(&venc->frames)) + vpu_dbg(LVL_DEBUG, "Warning some encoded frames are dropped\n"); + + list_for_each_entry_safe(frame, tmp, &venc->frames, list) { + list_del_init(&frame->list); + vfree(frame); + } +} + +static int venc_stop_session(struct vpu_inst *inst, u32 type) +{ + struct venc_t *venc = inst->priv; + + venc_set_enable(venc, type, 0); + if (venc->enable & VENC_ENABLE_MASK) + return 0; + + if (inst->state == VPU_CODEC_STATE_DEINIT) + return 0; + + if (inst->state != VPU_CODEC_STATE_STOP) + venc_request_eos(inst); + + call_vop(inst, wait_prepare); + if (!wait_event_timeout(venc->wq, venc->stopped, VPU_TIMEOUT)) { + set_bit(inst->id, &inst->core->hang_mask); + vpu_session_debug(inst); + } + call_vop(inst, wait_finish); + + inst->state = VPU_CODEC_STATE_DEINIT; + venc_cleanup_frames(inst->priv); + vpu_free_dma(&inst->stream_buffer); + venc_cleanup_mem_resource(inst); + + return 0; +} + +static int venc_process_output(struct vpu_inst *inst, struct vb2_buffer *vb) +{ + struct venc_t *venc = inst->priv; + struct vb2_v4l2_buffer *vbuf; + struct vpu_vb2_buffer *vpu_buf = NULL; + u32 flags; + + if (inst->state == VPU_CODEC_STATE_DEINIT) + return -EINVAL; + + vbuf = to_vb2_v4l2_buffer(vb); + vpu_buf = to_vpu_vb2_buffer(vbuf); + if (inst->state == VPU_CODEC_STATE_STARTED) + inst->state = VPU_CODEC_STATE_ACTIVE; + + flags = vbuf->flags; + if (venc->request_key_frame) { + vbuf->flags |= V4L2_BUF_FLAG_KEYFRAME; + venc->request_key_frame = 0; + } + if (venc->bitrate_change) { + vpu_session_update_parameters(inst, &venc->params); + venc->bitrate_change = false; + } + inst_dbg(inst, LVL_TS, "[INPUT TS]%32lld\n", vb->timestamp); + vpu_iface_input_frame(inst, vb); + vbuf->flags = flags; + venc->input_ready = false; + venc->frame_count++; + vpu_buf->state = VPU_BUF_STATE_INUSE; + + return 0; +} + +static int venc_process_capture(struct vpu_inst *inst, struct vb2_buffer *vb) +{ + struct venc_t *venc; + struct venc_frame_t *frame = NULL; + struct vb2_v4l2_buffer *vbuf; + int ret; + + venc = inst->priv; + if (list_empty(&venc->frames)) + return -EINVAL; + + frame = list_first_entry(&venc->frames, struct venc_frame_t, list); + vbuf = to_vb2_v4l2_buffer(vb); + v4l2_m2m_dst_buf_remove_by_buf(inst->m2m_ctx, vbuf); + ret = venc_get_one_encoded_frame(inst, frame, vbuf); + if (ret) + return ret; + + list_del_init(&frame->list); + vfree(frame); + return 0; +} + +static int venc_get_debug_info(struct vpu_inst *inst, char *str, u32 size, u32 i) +{ + struct venc_t *venc = inst->priv; + int num = -1; + + switch (i) { + case 0: + num = scnprintf(str, size, "profile = %d\n", venc->params.profile); + break; + case 1: + num = scnprintf(str, size, "level = %d\n", venc->params.level); + break; + case 2: + num = scnprintf(str, size, "fps = %d/%d\n", + venc->params.frame_rate_num, + venc->params.frame_rate_den); + break; + case 3: + num = scnprintf(str, size, "%d x %d -> %d x %d\n", + venc->params.src_width, + venc->params.src_height, + venc->params.out_width, + venc->params.out_height); + break; + case 4: + num = scnprintf(str, size, "(%d, %d) %d x %d\n", + venc->params.crop.left, + venc->params.crop.top, + venc->params.crop.width, + venc->params.crop.height); + break; + case 5: + num = scnprintf(str, size, + "enable = 0x%x, input = %d, encode = %d, ready = %d, stopped = %d\n", + venc->enable, + venc->frame_count, venc->encode_count, + venc->ready_count, + venc->stopped); + break; + case 6: + num = scnprintf(str, size, "gop = %d\n", venc->params.gop_length); + break; + case 7: + num = scnprintf(str, size, "bframes = %d\n", venc->params.bframes); + break; + case 8: + num = scnprintf(str, size, "rc: mode = %d, bitrate = %d, qp = %d\n", + venc->params.rc_mode, + venc->params.bitrate, + venc->params.i_frame_qp); + break; + case 9: + num = scnprintf(str, size, "sar: enable = %d, idc = %d, %d x %d\n", + venc->params.sar.enable, + venc->params.sar.idc, + venc->params.sar.width, + venc->params.sar.height); + + break; + case 10: + num = scnprintf(str, size, + "colorspace: primaries = %d, transfer = %d, matrix = %d, full_range = %d\n", + venc->params.color.primaries, + venc->params.color.transfer, + venc->params.color.matrix, + venc->params.color.full_range); + break; + case 11: + num = scnprintf(str, size, "skipped: count = %d, bytes = %d\n", + venc->skipped_count, venc->skipped_bytes); + break; + default: + break; + } + + return num; +} + +static struct vpu_inst_ops venc_inst_ops = { + .ctrl_init = venc_ctrl_init, + .check_ready = venc_check_ready, + .input_done = venc_input_done, + .get_one_frame = venc_frame_encoded, + .buf_done = venc_buf_done, + .stop_done = venc_stop_done, + .event_notify = venc_event_notify, + .release = venc_release, + .cleanup = venc_cleanup, + .start = venc_start_session, + .mem_request = venc_request_mem_resource, + .stop = venc_stop_session, + .process_output = venc_process_output, + .process_capture = venc_process_capture, + .get_debug_info = venc_get_debug_info, + .wait_prepare = vpu_inst_unlock, + .wait_finish = vpu_inst_lock, +}; + +static void venc_init(struct file *file) +{ + struct vpu_inst *inst = to_inst(file); + struct venc_t *venc; + struct v4l2_format f; + struct v4l2_streamparm parm; + + venc = inst->priv; + venc->params.qp_min = 1; + venc->params.qp_max = 51; + venc->params.qp_min_i = 1; + venc->params.qp_max_i = 51; + venc->params.bitrate_max = BITRATE_MAX; + venc->params.bitrate_min = BITRATE_MIN; + + memset(&f, 0, sizeof(f)); + f.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + f.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_NV12; + f.fmt.pix_mp.width = 1280; + f.fmt.pix_mp.height = 720; + f.fmt.pix_mp.field = V4L2_FIELD_NONE; + f.fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709; + venc_s_fmt(file, &inst->fh, &f); + + memset(&f, 0, sizeof(f)); + f.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + f.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264; + f.fmt.pix_mp.width = 1280; + f.fmt.pix_mp.height = 720; + f.fmt.pix_mp.field = V4L2_FIELD_NONE; + venc_s_fmt(file, &inst->fh, &f); + + memset(&parm, 0, sizeof(parm)); + parm.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + parm.parm.capture.timeperframe.numerator = 1; + parm.parm.capture.timeperframe.denominator = 30; + venc_s_parm(file, &inst->fh, &parm); +} + +int venc_open(struct file *file) +{ + struct vpu_inst *inst; + struct venc_t *venc; + int ret; + + inst = vzalloc(sizeof(*inst)); + if (!inst) + return -ENOMEM; + + venc = vzalloc(sizeof(*venc)); + if (!venc) { + vfree(inst); + return -ENOMEM; + } + + inst->ops = &venc_inst_ops; + inst->formats = venc_formats; + inst->type = VPU_CORE_TYPE_ENC; + inst->priv = venc; + INIT_LIST_HEAD(&venc->frames); + init_waitqueue_head(&venc->wq); + + ret = vpu_v4l2_open(file, inst); + if (ret) { + vfree(venc); + vfree(inst); + return ret; + } + + venc_init(file); + + return 0; +} + +static const struct v4l2_file_operations venc_fops = { + .owner = THIS_MODULE, + .open = venc_open, + .release = vpu_v4l2_close, + .unlocked_ioctl = video_ioctl2, + .poll = v4l2_m2m_fop_poll, + .mmap = v4l2_m2m_fop_mmap, +}; + +int venc_create_video_device(struct vpu_dev *vpu) +{ + struct video_device *venc; + int ret; + + if (!vpu) + return -EINVAL; + + if (vpu->vdev_enc) + return 0; + + venc = video_device_alloc(); + if (!venc) { + vpu_err("alloc vpu encoder video device fail\n"); + return -ENOMEM; + } + strscpy(venc->name, "imx-vpu-encoder", sizeof(venc->name)); + venc->release = video_device_release; + venc->fops = &venc_fops; + venc->ioctl_ops = &venc_ioctl_ops; + venc->vfl_dir = VFL_DIR_M2M; + venc->v4l2_dev = &vpu->v4l2_dev; + venc->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; + + ret = video_register_device(venc, VFL_TYPE_VIDEO, -1); + if (ret) { + video_device_release(venc); + return ret; + } + video_set_drvdata(venc, vpu); + vpu->vdev_enc = venc; + + return 0; +} From patchwork Wed Aug 11 08:43:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 495315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45B04C4338F for ; Wed, 11 Aug 2021 08:45:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DD4860527 for ; Wed, 11 Aug 2021 08:45:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236557AbhHKIp3 (ORCPT ); Wed, 11 Aug 2021 04:45:29 -0400 Received: from mail-eopbgr80054.outbound.protection.outlook.com ([40.107.8.54]:2016 "EHLO EUR04-VI1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236525AbhHKIpZ (ORCPT ); Wed, 11 Aug 2021 04:45:25 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eFNpHLuP5/ixiRit2kdI0t4dK+3KyWzY8xRuYdgLfInvkR1YRrKxEt86UjaNai22eEuUHGib56k45qX4Uo2Pys6Io9lta1+OzY26qLWoJw7GE9XQ20kmCdKvMVYBlKieDV1DyPOe6HJ27qIZOScREWUUOoJn4a+AukviciR6/iYPfL783ymyxi82zNseZ/dr3iylefHsgJyhjmOFRFeswuMYqcvFp+bC7JKMoe8zZU1c/Qccd59j7KXNu1aAox3RgAcRA8I0337KlHei+ga+6yn5Tx6YCsVxIh7WfK8yktiqeLArnc8hGW71M4A5cE3PU1nz3zO2PYrEOkNsLRXxRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mbphJhXqfRnVFaGmhHH3bT4DwqdDfAJIgISNcjmsPp4=; b=Cfd6n8KY3aTL3EzbAnFarlzyUzK4VRPEb1r3Ze2wfop5HZVoTPZUtIbsiQ8as185/cU++uQdc7djbjnGCqT+BnbHa4xzuz/RXGXYTALBQ+XVSJjKh5qnJx4mKJywmUCJgkzmOGACiSF7TKoeSfsAkAiut6h1y4W3YCj1ocZMFeoisEWLp6HdZwrXQHEiK7F6zXPiRWN0GC1rIjWSdsDTlMbYUHh8wxKr87Y1O4c5JT0UhlXH0eHmG3vgivMPNTjaw+5pGDFfwWNgcDswHCfJGHM1XqcBDhNbImwI9uJrN6n4BzkBlI7/QlOcO1vhfu+R2mHjrffUp7/hMGnrFtaz6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mbphJhXqfRnVFaGmhHH3bT4DwqdDfAJIgISNcjmsPp4=; b=bx+vcbl8m5ImoxoycftjqF1GR2W2tuf1N24tCVbT1QYjfO2eBzl+LIEN+VBmpSKaJfvm6uyFjvDYnth+kQaw1vE8kMnq2wpA+BXojgDwiRCLtOzDugOsDuXlCxZK4vK3nczxczkGM3f5BFG7gkEa2cV57HMRWjzizp6qtXeqJcQ= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM6PR04MB4888.eurprd04.prod.outlook.com (2603:10a6:20b:12::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.17; Wed, 11 Aug 2021 08:44:42 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384%6]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 08:44:42 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de Cc: hverkuil-cisco@xs4all.nl, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 11/14] media: imx: imx8q: implement malone decoder rpc interface Date: Wed, 11 Aug 2021 16:43:07 +0800 Message-Id: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-ClientProxiedBy: SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.4 via Frontend Transport; Wed, 11 Aug 2021 08:44:38 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 30c3b581-be8b-4c9c-796f-08d95ca44310 X-MS-TrafficTypeDiagnostic: AM6PR04MB4888: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:184; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RzMRYKBT2Cxzbiv/g9vgcgfjiI/9oe7cY5KVdyfVaASgaq2i9ua/Q0jwBgWDfo1jU0MDa2LELM2XoJ/CCD4Lj/fAqyk/8E8+eeRRp6i9m3bL1aP52YdJxizKaJl/bETjJYSX8Jm7mGg23vsnw8DYJZfFMK9fAXFLINErLA8qB3VTbgW5SOPTP4GhAjIkc7jK40ix0Pw7EPR8fHSxM0LAwHf3R+yy2u2nIn+Mw61mED5osLqmGU4pYuDwngXfHUzwpX23UKfh5MoCEbQOwF8ie9Rp9BnmZfe7Nf/Vm/N/ZVY9lXGVM9c/VAL4u6idEec/onhj+SCvaUt0hOAmTNRZ4l6N6eZuhXCiFCN6pwv00yPfkwSJHWnXLPTDnY3sid+gqGrV8uTIKidPXmvqG7ER+8qsRp10Hp16DoTSiVhKIUZzd/vqZRiWqOyx8dpGyUascTAFSiXbrH7ra8ilcT9d+HWC5psFvNR+eds3jP7qVMTLURfaEJRvT6xRpvU4twI3YM8xkbQXUmu+wbRD5PyVkdZIsiHP3IuQZaPM1N4Q62nARqaBhjyAHWrrYohfpYq4oZojnxzB4uOkoHoBj5a5WaxnVfcwLxk3l6WUCFmZ2J9BBgNHaPU1RVCssjXB/YS1k9sZXJMn2fGoiI4CtmD4gMXINuezGJS5H++aCDBXFkRCQQBdNb29HAbaa0RWY0ntG3a4wRw7P7IXXOE+MnVAUBvt55rcM5cp9mVSGRtziYU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(376002)(366004)(396003)(136003)(39860400002)(7416002)(86362001)(8676002)(6486002)(36756003)(44832011)(2616005)(956004)(4326008)(7696005)(2906002)(38100700002)(66476007)(478600001)(26005)(52116002)(66556008)(66946007)(38350700002)(186003)(8936002)(316002)(6666004)(5660300002)(83380400001)(30864003)(32563001)(579004)(559001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: uFWdjBvCphlGYZCxfw7KxBOoZgQ373JAfwCnBQF0wCkt/XK/ZDzx6UqC4ELbhWdhh/3ffPgQqqZTZj2UMtNCDUJJU8McmUiVmtv6mx4OsgFUQMYh+fLCLr3V7xswlGzmxgVKu8htsMS/q2it8gxX9TIURs/EFD3VMP6rdMJyrTCW9T7RgpzFxp3T3iLXDwAaseMTXAV8rubx0UPQoLAcQcPfBxLOlrgoywHMHNfYMx7nTSZVAIvzoUxbFwvzmVNJMJeIufuuCKU6/Oo8gOuoso8Rtleuryq96TnYD4+Tu3ge9ZbZJ8pnZj6X5IddhvGjG5RKB5fgDm+5SoRAgQ8eC1LMTMXYnYa0EVimP2XrkBvOEkiSTTVhGqim/puSrxOLnaqtwz5nwHsR8BG1s5fKYf8JP/NpPswiy2YZvKtVtTVey7Tq4NxQsaU6fC6i1zAwhEXTAKM1wfMERL1aF4JU+hyYYWRcAjBuSNG3XMe2xLpr0tsCk8IIeyJFm3SRsX+rtmHuVLqbhUohruFvQ2eYHYUAY/LKQiBoXEJdZ+Td2eNVUP4Is9IR4z8CTXqhk72ywg4fVY6107rpGRFT6VFT0Dve6G84tN0kI/CBeUDoYZpgIz3c+woL/JISej0PSgxftNAWL6YkGT3bpgkM84EaAMoLBlAAOGM/I/0oeSYmZgnjsv7jRbPy5RKgNFjMu6inb8Umhv6EACsCloM9RvngYnPQskhmtoQwZ8a/29XegA2xjrUY1kj7zX2xHT3jl6f8biNWNskEEoVvQ7m2lT44FGY9B7dmjzBhGQ7cCm6NCOjGXLKM1AyB+SlxKFRyB0sXdyv4+U6AGseXFVDBifsOw6jK5Bxh3U/cniIb0UecuR+YNnSDwCCeIHRWbleQA9GdiJdrSp5ijpNgOo7hP+8b8rJC4CYR9oRQU0us4d3nYLODFV+5ITEC6x+DgSq4ul6fnqo+s/C/rVpXva3xvULlEGU1/+b3yGkzj2JI77pdMBUSJoaD0pCqh67n73XpW+JxFSH7FDaLxhZ8zyrHTKgmI00fSWKcDY5yhpFg7ajVaiwv9gbfLFpIaVxvCSQ0ovMvoKpy9cJ5d9lHlwD1udSYZPKVWQVEhuzTJ0yOELsRUXYK0dk+F5+DGEr65Qn1QIFfGlCaS49L99PShAlzHo8NWi3E/8TGDmmaPFsz4GL5Sw96rkGQ6eqpK1t82QsVmBouP6vGDPb/nngYVWoce9+X7J5S08oQnUSmmXurcOi2cEzQkLRihfN4BYmijiJFsW0AC0wSjNZUUJVBv6PsfM4QttfjZhgA0GZZ+ka9Mdn2fGnQSjq7hUvGcib15ndGDqk0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 30c3b581-be8b-4c9c-796f-08d95ca44310 X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 08:44:42.1799 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pvXv/dstyx+poZ3HkhiQfAAT7/3MqoYr74JbDs4HQHVtciNp2FCrHgJY7M1d3TCEZpUjf1EoLO4LDBcIwUW5hw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB4888 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org This part implements the malone decoder rpc interface. Signed-off-by: Ming Qian Signed-off-by: Shijie Qin Signed-off-by: Zhou Peng --- .../media/platform/imx/vpu-8q/vpu_malone.c | 1687 +++++++++++++++++ .../media/platform/imx/vpu-8q/vpu_malone.h | 42 + 2 files changed, 1729 insertions(+) create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_malone.c create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_malone.h diff --git a/drivers/media/platform/imx/vpu-8q/vpu_malone.c b/drivers/media/platform/imx/vpu-8q/vpu_malone.c new file mode 100644 index 000000000000..51f78fd3247d --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_malone.c @@ -0,0 +1,1687 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020-2021 NXP + */ + +#define TAG "MALONE" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "vpu.h" +#include "vpu_rpc.h" +#include "vpu_defs.h" +#include "vpu_helpers.h" +#include "vpu_cmds.h" +#include "vpu_log.h" +#include "vpu_imx8q.h" +#include "vpu_malone.h" + +#define CMD_SIZE 25600 +#define MSG_SIZE 25600 +#define CODEC_SIZE 0x1000 +#define JPEG_SIZE 0x1000 +#define SEQ_SIZE 0x1000 +#define GOP_SIZE 0x1000 +#define PIC_SIZE 0x1000 +#define QMETER_SIZE 0x1000 +#define DBGLOG_SIZE 0x10000 +#define DEBUG_SIZE 0x80000 +#define ENG_SIZE 0x1000 +#define MALONE_SKIPPED_FRAME_ID 0x555 + +#define MALONE_ALIGN_MBI 0x800 +#define MALONE_DCP_CHUNK_BIT 16 +#define MALONE_DCP_SIZE_MAX 0x3000000 +#define MALONE_DCP_SIZE_MIN 0x100000 +#define MALONE_DCP_FIXED_MB_ALLOC 250 + +#define CONFIG_SET(val, cfg, pos, mask) \ + (*(cfg) |= (((val) << (pos)) & (mask))) +//x means source data , y means destination data +#define STREAM_CONFIG_FORMAT_SET(x, y) CONFIG_SET(x, y, 0, 0x0000000F) +#define STREAM_CONFIG_STRBUFIDX_SET(x, y) CONFIG_SET(x, y, 8, 0x00000300) +#define STREAM_CONFIG_NOSEQ_SET(x, y) CONFIG_SET(x, y, 10, 0x00000400) +#define STREAM_CONFIG_DEBLOCK_SET(x, y) CONFIG_SET(x, y, 11, 0x00000800) +#define STREAM_CONFIG_DERING_SET(x, y) CONFIG_SET(x, y, 12, 0x00001000) +#define STREAM_CONFIG_IBWAIT_SET(x, y) CONFIG_SET(x, y, 13, 0x00002000) +#define STREAM_CONFIG_FBC_SET(x, y) CONFIG_SET(x, y, 14, 0x00004000) +#define STREAM_CONFIG_PLAY_MODE_SET(x, y) CONFIG_SET(x, y, 16, 0x00030000) +#define STREAM_CONFIG_ENABLE_DCP_SET(x, y) CONFIG_SET(x, y, 20, 0x00100000) +#define STREAM_CONFIG_NUM_STR_BUF_SET(x, y) CONFIG_SET(x, y, 21, 0x00600000) +#define STREAM_CONFIG_MALONE_USAGE_SET(x, y) CONFIG_SET(x, y, 23, 0x01800000) +#define STREAM_CONFIG_MULTI_VID_SET(x, y) CONFIG_SET(x, y, 25, 0x02000000) +#define STREAM_CONFIG_OBFUSC_EN_SET(x, y) CONFIG_SET(x, y, 26, 0x04000000) +#define STREAM_CONFIG_RC4_EN_SET(x, y) CONFIG_SET(x, y, 27, 0x08000000) +#define STREAM_CONFIG_MCX_SET(x, y) CONFIG_SET(x, y, 28, 0x10000000) +#define STREAM_CONFIG_PES_SET(x, y) CONFIG_SET(x, y, 29, 0x20000000) +#define STREAM_CONFIG_NUM_DBE_SET(x, y) CONFIG_SET(x, y, 30, 0x40000000) +#define STREAM_CONFIG_FS_CTRL_MODE_SET(x, y) CONFIG_SET(x, y, 31, 0x80000000) + +enum vpu_malone_stream_input_mode { + INVALID_MODE = 0, + FRAME_LVL, + NON_FRAME_LVL +}; + +enum vpu_malone_format { + MALONE_FMT_NULL = 0x0, + MALONE_FMT_AVC = 0x1, + MALONE_FMT_MP2 = 0x2, + MALONE_FMT_VC1 = 0x3, + MALONE_FMT_AVS = 0x4, + MALONE_FMT_ASP = 0x5, + MALONE_FMT_JPG = 0x6, + MALONE_FMT_RV = 0x7, + MALONE_FMT_VP6 = 0x8, + MALONE_FMT_SPK = 0x9, + MALONE_FMT_VP8 = 0xA, + MALONE_FMT_HEVC = 0xB, + MALONE_FMT_LAST = MALONE_FMT_HEVC +}; + +enum { + VID_API_CMD_NULL = 0x00, + VID_API_CMD_PARSE_NEXT_SEQ = 0x01, + VID_API_CMD_PARSE_NEXT_I = 0x02, + VID_API_CMD_PARSE_NEXT_IP = 0x03, + VID_API_CMD_PARSE_NEXT_ANY = 0x04, + VID_API_CMD_DEC_PIC = 0x05, + VID_API_CMD_UPDATE_ES_WR_PTR = 0x06, + VID_API_CMD_UPDATE_ES_RD_PTR = 0x07, + VID_API_CMD_UPDATE_UDATA = 0x08, + VID_API_CMD_GET_FSINFO = 0x09, + VID_API_CMD_SKIP_PIC = 0x0a, + VID_API_CMD_DEC_CHUNK = 0x0b, + VID_API_CMD_START = 0x10, + VID_API_CMD_STOP = 0x11, + VID_API_CMD_ABORT = 0x12, + VID_API_CMD_RST_BUF = 0x13, + VID_API_CMD_FS_RELEASE = 0x15, + VID_API_CMD_MEM_REGION_ATTACH = 0x16, + VID_API_CMD_MEM_REGION_DETACH = 0x17, + VID_API_CMD_MVC_VIEW_SELECT = 0x18, + VID_API_CMD_FS_ALLOC = 0x19, + VID_API_CMD_DBG_GET_STATUS = 0x1C, + VID_API_CMD_DBG_START_LOG = 0x1D, + VID_API_CMD_DBG_STOP_LOG = 0x1E, + VID_API_CMD_DBG_DUMP_LOG = 0x1F, + VID_API_CMD_YUV_READY = 0x20, + VID_API_CMD_TS = 0x21, + + VID_API_CMD_FIRM_RESET = 0x40, + + VID_API_CMD_SNAPSHOT = 0xAA, + VID_API_CMD_ROLL_SNAPSHOT = 0xAB, + VID_API_CMD_LOCK_SCHEDULER = 0xAC, + VID_API_CMD_UNLOCK_SCHEDULER = 0xAD, + VID_API_CMD_CQ_FIFO_DUMP = 0xAE, + VID_API_CMD_DBG_FIFO_DUMP = 0xAF, + VID_API_CMD_SVC_ILP = 0xBB, + VID_API_CMD_FW_STATUS = 0xF0, + VID_API_CMD_INVALID = 0xFF +}; + +enum { + VID_API_EVENT_NULL = 0x00, + VID_API_EVENT_RESET_DONE = 0x01, + VID_API_EVENT_SEQ_HDR_FOUND = 0x02, + VID_API_EVENT_PIC_HDR_FOUND = 0x03, + VID_API_EVENT_PIC_DECODED = 0x04, + VID_API_EVENT_FIFO_LOW = 0x05, + VID_API_EVENT_FIFO_HIGH = 0x06, + VID_API_EVENT_FIFO_EMPTY = 0x07, + VID_API_EVENT_FIFO_FULL = 0x08, + VID_API_EVENT_BS_ERROR = 0x09, + VID_API_EVENT_UDATA_FIFO_UPTD = 0x0A, + VID_API_EVENT_RES_CHANGE = 0x0B, + VID_API_EVENT_FIFO_OVF = 0x0C, + VID_API_EVENT_CHUNK_DECODED = 0x0D, + VID_API_EVENT_REQ_FRAME_BUFF = 0x10, + VID_API_EVENT_FRAME_BUFF_RDY = 0x11, + VID_API_EVENT_REL_FRAME_BUFF = 0x12, + VID_API_EVENT_STR_BUF_RST = 0x13, + VID_API_EVENT_RET_PING = 0x14, + VID_API_EVENT_QMETER = 0x15, + VID_API_EVENT_STR_FMT_CHANGE = 0x16, + VID_API_EVENT_FIRMWARE_XCPT = 0x17, + VID_API_EVENT_START_DONE = 0x18, + VID_API_EVENT_STOPPED = 0x19, + VID_API_EVENT_ABORT_DONE = 0x1A, + VID_API_EVENT_FINISHED = 0x1B, + VID_API_EVENT_DBG_STAT_UPDATE = 0x1C, + VID_API_EVENT_DBG_LOG_STARTED = 0x1D, + VID_API_EVENT_DBG_LOG_STOPPED = 0x1E, + VID_API_EVENT_DBG_LOG_UPDATED = 0x1F, + VID_API_EVENT_DBG_MSG_DEC = 0x20, + VID_API_EVENT_DEC_SC_ERR = 0x21, + VID_API_EVENT_CQ_FIFO_DUMP = 0x22, + VID_API_EVENT_DBG_FIFO_DUMP = 0x23, + VID_API_EVENT_DEC_CHECK_RES = 0x24, + VID_API_EVENT_DEC_CFG_INFO = 0x25, + VID_API_EVENT_UNSUPPORTED_STREAM = 0x26, + VID_API_EVENT_STR_SUSPENDED = 0x30, + VID_API_EVENT_SNAPSHOT_DONE = 0x40, + VID_API_EVENT_FW_STATUS = 0xF0, + VID_API_EVENT_INVALID = 0xFF +}; + +struct vpu_malone_buffer_desc { + struct vpu_rpc_buffer_desc buffer; + u32 low; + u32 high; +}; + +struct vpu_malone_str_buffer { + u32 wptr; + u32 rptr; + u32 start; + u32 end; + u32 lwm; +}; + +struct vpu_malone_picth_info { + u32 frame_pitch; +}; + +struct vpu_malone_table_desc { + u32 array_base; + u32 size; +}; + +struct vpu_malone_dbglog_desc { + u32 addr; + u32 size; + u32 level; + u32 reserved; +}; + +struct vpu_malone_frame_buffer { + u32 addr; + u32 size; +}; + +struct vpu_malone_udata { + u32 base; + u32 total_size; + u32 slot_size; +}; + +struct vpu_malone_buffer_info { + u32 stream_input_mode; + u32 stream_pic_input_count; + u32 stream_pic_parsed_count; + u32 stream_buffer_threshold; + u32 stream_pic_end_flag; +}; + +struct vpu_malone_encrypt_info { + u32 rec4key[8]; + u32 obfusc; +}; + +struct malone_iface { + u32 exec_base_addr; + u32 exec_area_size; + struct vpu_malone_buffer_desc cmd_buffer_desc; + struct vpu_malone_buffer_desc msg_buffer_desc; + u32 cmd_int_enable[VID_API_NUM_STREAMS]; + struct vpu_malone_picth_info stream_pitch_info[VID_API_NUM_STREAMS]; + u32 stream_config[VID_API_NUM_STREAMS]; + struct vpu_malone_table_desc codec_param_tab_desc; + struct vpu_malone_table_desc jpeg_param_tab_desc; + u32 stream_buffer_desc[VID_API_NUM_STREAMS][VID_API_MAX_BUF_PER_STR]; + struct vpu_malone_table_desc seq_info_tab_desc; + struct vpu_malone_table_desc pic_info_tab_desc; + struct vpu_malone_table_desc gop_info_tab_desc; + struct vpu_malone_table_desc qmeter_info_tab_desc; + u32 stream_error[VID_API_NUM_STREAMS]; + u32 fw_version; + u32 fw_offset; + u32 max_streams; + struct vpu_malone_dbglog_desc dbglog_desc; + struct vpu_rpc_buffer_desc api_cmd_buffer_desc[VID_API_NUM_STREAMS]; + struct vpu_malone_udata udata_buffer[VID_API_NUM_STREAMS]; + struct vpu_malone_buffer_desc debug_buffer_desc; + struct vpu_malone_buffer_desc eng_access_buff_desc[VID_API_NUM_STREAMS]; + u32 encrypt_info[VID_API_NUM_STREAMS]; + struct vpu_rpc_system_config system_cfg; + u32 api_version; + struct vpu_malone_buffer_info stream_buff_info[VID_API_NUM_STREAMS]; +}; + +struct malone_jpg_params { + u32 rotation_angle; + u32 horiz_scale_factor; + u32 vert_scale_factor; + u32 rotation_mode; + u32 rgb_mode; + u32 chunk_mode; /* 0 ~ 1 */ + u32 last_chunk; /* 0 ~ 1 */ + u32 chunk_rows; /* 0 ~ 255 */ + u32 num_bytes; + u32 jpg_crop_x; + u32 jpg_crop_y; + u32 jpg_crop_width; + u32 jpg_crop_height; + u32 jpg_mjpeg_mode; + u32 jpg_mjpeg_interlaced; +}; + +struct malone_codec_params { + u32 disp_imm; + u32 fourcc; + u32 codec_version; + u32 frame_rate; + u32 dbglog_enable; + u32 bsdma_lwm; + u32 bbd_coring; + u32 bbd_s_thr_row; + u32 bbd_p_thr_row; + u32 bbd_s_thr_logo_row; + u32 bbd_p_thr_logo_row; + u32 bbd_s_thr_col; + u32 bbd_p_thr_col; + u32 bbd_chr_thr_row; + u32 bbd_chr_thr_col; + u32 bbd_uv_mid_level; + u32 bbd_excl_win_mb_left; + u32 bbd_excl_win_mb_right; +}; + +struct malone_padding_scode { + u32 scode_type; + u32 pixelformat; + u32 data[2]; +}; + +struct malone_fmt_mapping { + u32 pixelformat; + enum vpu_malone_format malone_format; +}; + +struct malone_scode_t { + struct vpu_inst *inst; + struct vb2_buffer *vb; + u32 wptr; + u32 need_data; +}; + +struct malone_scode_handler { + u32 pixelformat; + int (*insert_scode_seq)(struct malone_scode_t *scode); + int (*insert_scode_pic)(struct malone_scode_t *scode); +}; + +struct vpu_dec_ctrl { + struct malone_codec_params *codec_param; + struct malone_jpg_params *jpg; + void *seq_mem; + void *pic_mem; + void *gop_mem; + void *qmeter_mem; + void *dbglog_mem; + struct vpu_malone_str_buffer *str_buf[VID_API_NUM_STREAMS]; + u32 buf_addr[VID_API_NUM_STREAMS]; +}; + +u32 vpu_malone_get_data_size(void) +{ + return sizeof(struct vpu_dec_ctrl); +} + +void vpu_malone_init_rpc(struct vpu_shared_addr *shared, + struct vpu_buffer *rpc, dma_addr_t boot_addr) +{ + struct malone_iface *iface; + struct vpu_dec_ctrl *hc; + unsigned long base_phy_addr; + unsigned long phy_addr; + unsigned long offset; + unsigned int i; + + WARN_ON(!shared || !shared->priv); + WARN_ON(!rpc || !rpc->phys || !rpc->length || rpc->phys < boot_addr); + + iface = rpc->virt; + base_phy_addr = rpc->phys - boot_addr; + hc = shared->priv; + + shared->iface = iface; + shared->boot_addr = boot_addr; + + iface->exec_base_addr = base_phy_addr; + iface->exec_area_size = rpc->length; + + offset = sizeof(struct malone_iface); + phy_addr = base_phy_addr + offset; + + shared->cmd_desc = &iface->cmd_buffer_desc.buffer; + shared->cmd_mem_vir = rpc->virt + offset; + iface->cmd_buffer_desc.buffer.start = + iface->cmd_buffer_desc.buffer.rptr = + iface->cmd_buffer_desc.buffer.wptr = phy_addr; + iface->cmd_buffer_desc.buffer.end = iface->cmd_buffer_desc.buffer.start + CMD_SIZE; + offset += CMD_SIZE; + phy_addr = base_phy_addr + offset; + + shared->msg_desc = &iface->msg_buffer_desc.buffer; + shared->msg_mem_vir = rpc->virt + offset; + iface->msg_buffer_desc.buffer.start = + iface->msg_buffer_desc.buffer.wptr = + iface->msg_buffer_desc.buffer.rptr = phy_addr; + iface->msg_buffer_desc.buffer.end = iface->msg_buffer_desc.buffer.start + MSG_SIZE; + offset += MSG_SIZE; + phy_addr = base_phy_addr + offset; + + iface->codec_param_tab_desc.array_base = phy_addr; + hc->codec_param = rpc->virt + offset; + offset += CODEC_SIZE; + phy_addr = base_phy_addr + offset; + + iface->jpeg_param_tab_desc.array_base = phy_addr; + hc->jpg = rpc->virt + offset; + offset += JPEG_SIZE; + phy_addr = base_phy_addr + offset; + + iface->seq_info_tab_desc.array_base = phy_addr; + hc->seq_mem = rpc->virt + offset; + offset += SEQ_SIZE; + phy_addr = base_phy_addr + offset; + + iface->pic_info_tab_desc.array_base = phy_addr; + hc->pic_mem = rpc->virt + offset; + offset += PIC_SIZE; + phy_addr = base_phy_addr + offset; + + iface->gop_info_tab_desc.array_base = phy_addr; + hc->gop_mem = rpc->virt + offset; + offset += GOP_SIZE; + phy_addr = base_phy_addr + offset; + + iface->qmeter_info_tab_desc.array_base = phy_addr; + hc->qmeter_mem = rpc->virt + offset; + offset += QMETER_SIZE; + phy_addr = base_phy_addr + offset; + + iface->dbglog_desc.addr = phy_addr; + iface->dbglog_desc.size = DBGLOG_SIZE; + hc->dbglog_mem = rpc->virt + offset; + offset += DBGLOG_SIZE; + phy_addr = base_phy_addr + offset; + + for (i = 0; i < VID_API_NUM_STREAMS; i++) { + iface->eng_access_buff_desc[i].buffer.start = + iface->eng_access_buff_desc[i].buffer.wptr = + iface->eng_access_buff_desc[i].buffer.rptr = phy_addr; + iface->eng_access_buff_desc[i].buffer.end = + iface->eng_access_buff_desc[i].buffer.start + ENG_SIZE; + offset += ENG_SIZE; + phy_addr = base_phy_addr + offset; + } + + for (i = 0; i < VID_API_NUM_STREAMS; i++) { + iface->encrypt_info[i] = phy_addr; + offset += sizeof(struct vpu_malone_encrypt_info); + phy_addr = base_phy_addr + offset; + } + + rpc->bytesused = offset; +} + +void vpu_malone_set_log_buf(struct vpu_shared_addr *shared, + struct vpu_buffer *log) +{ + struct malone_iface *iface; + + WARN_ON(!shared || !log || !log->phys); + vpu_dbg(LVL_DEBUG, "set log area <0x%llx, 0x%x> (0x%lx)\n", + log->phys, log->length, shared->boot_addr); + iface = shared->iface; + iface->debug_buffer_desc.buffer.start = + iface->debug_buffer_desc.buffer.wptr = + iface->debug_buffer_desc.buffer.rptr = log->phys - shared->boot_addr; + iface->debug_buffer_desc.buffer.end = iface->debug_buffer_desc.buffer.start + log->length; +} + +static u32 get_str_buffer_offset(u32 instance) +{ + return DEC_MFD_XREG_SLV_BASE + MFD_MCX + MFD_MCX_OFF * instance; +} + +void vpu_malone_set_system_cfg(struct vpu_shared_addr *shared, + u32 regs_base, void __iomem *regs, u32 core_id) +{ + struct malone_iface *iface; + struct vpu_rpc_system_config *config; + struct vpu_dec_ctrl *hc; + int i; + + WARN_ON(!shared || !shared->iface || !shared->core || !shared->priv); + + iface = shared->iface; + config = &iface->system_cfg; + hc = shared->priv; + + vpu_imx8q_set_system_cfg_common(config, regs_base, core_id); + for (i = 0; i < VID_API_NUM_STREAMS; i++) { + u32 offset = get_str_buffer_offset(i); + + hc->buf_addr[i] = regs_base + offset; + hc->str_buf[i] = regs + offset; + } +} + +u32 vpu_malone_get_version(struct vpu_shared_addr *shared) +{ + struct malone_iface *iface; + + WARN_ON(!shared || !shared->iface); + + iface = shared->iface; + return iface->fw_version; +} + +int vpu_malone_get_stream_buffer_size(struct vpu_shared_addr *shared) +{ + return 0xc00000; +} + +int vpu_malone_config_stream_buffer(struct vpu_shared_addr *shared, + u32 instance, + struct vpu_buffer *buf) +{ + struct malone_iface *iface; + struct vpu_dec_ctrl *hc; + struct vpu_malone_str_buffer *str_buf; + + WARN_ON(!shared || !shared->iface || !shared->core || !shared->priv); + + iface = shared->iface; + hc = shared->priv; + str_buf = hc->str_buf[instance]; + str_buf->wptr = str_buf->rptr = str_buf->start = buf->phys; + str_buf->end = buf->phys + buf->length; + str_buf->lwm = 0x1; + + iface->stream_buffer_desc[instance][0] = hc->buf_addr[instance]; + + return 0; +} + +int vpu_malone_get_stream_buffer_desc(struct vpu_shared_addr *shared, + u32 instance, struct vpu_rpc_buffer_desc *desc) +{ + struct vpu_dec_ctrl *hc; + struct vpu_malone_str_buffer *str_buf; + + WARN_ON(!shared || !shared->iface || !shared->core || !shared->priv); + + hc = shared->priv; + str_buf = hc->str_buf[instance]; + + if (desc) { + desc->wptr = str_buf->wptr; + desc->rptr = str_buf->rptr; + desc->start = str_buf->start; + desc->end = str_buf->end; + } + + return 0; +} + +static void vpu_malone_update_wptr(struct vpu_malone_str_buffer *str_buf, + u32 wptr) +{ + u32 size = str_buf->end - str_buf->start; + u32 space = (str_buf->rptr + size - str_buf->wptr) % size; + u32 step = (wptr + size - str_buf->wptr) % size; + + if (space && step > space) + vpu_err("update wptr from 0x%x to 0x%x, cross over rptr 0x%x\n", + str_buf->wptr, wptr, str_buf->rptr); + + /*update wptr after data is written*/ + mb(); + str_buf->wptr = wptr; +} + +static void vpu_malone_update_rptr(struct vpu_malone_str_buffer *str_buf, + u32 rptr) +{ + u32 size = str_buf->end - str_buf->start; + u32 space = (str_buf->wptr + size - str_buf->rptr) % size; + u32 step = (rptr + size - str_buf->rptr) % size; + + if (step > space) + vpu_err("update rptr from 0x%x to 0x%x, cross over wptr 0x%x\n", + str_buf->rptr, rptr, str_buf->wptr); + /*update rptr after data is read*/ + mb(); + str_buf->rptr = rptr; +} + +int vpu_malone_update_stream_buffer(struct vpu_shared_addr *shared, + u32 instance, u32 ptr, bool write) +{ + struct vpu_dec_ctrl *hc; + struct vpu_malone_str_buffer *str_buf; + + WARN_ON(!shared || !shared->iface || !shared->core || !shared->priv); + + hc = shared->priv; + str_buf = hc->str_buf[instance]; + + if (write) + vpu_malone_update_wptr(str_buf, ptr); + else + vpu_malone_update_rptr(str_buf, ptr); + + return 0; +} + +static struct malone_fmt_mapping fmt_mappings[] = { + {V4L2_PIX_FMT_H264, MALONE_FMT_AVC}, + {V4L2_PIX_FMT_H264_MVC, MALONE_FMT_AVC}, + {V4L2_PIX_FMT_HEVC, MALONE_FMT_HEVC}, + {V4L2_PIX_FMT_VC1_ANNEX_G, MALONE_FMT_VC1}, + {V4L2_PIX_FMT_VC1_ANNEX_L, MALONE_FMT_VC1}, + {V4L2_PIX_FMT_MPEG2, MALONE_FMT_MP2}, + {V4L2_PIX_FMT_MPEG4, MALONE_FMT_ASP}, + {V4L2_PIX_FMT_XVID, MALONE_FMT_ASP}, + {V4L2_PIX_FMT_H263, MALONE_FMT_ASP}, + {V4L2_PIX_FMT_JPEG, MALONE_FMT_JPG}, + {V4L2_PIX_FMT_VP8, MALONE_FMT_VP8}, +}; + +static enum vpu_malone_format vpu_malone_format_remap(u32 pixelformat) +{ + u32 i; + + for (i = 0; i < ARRAY_SIZE(fmt_mappings); i++) { + if (pixelformat == fmt_mappings[i].pixelformat) + return fmt_mappings[i].malone_format; + } + + return MALONE_FMT_NULL; +} + +static void vpu_malone_set_stream_cfg(struct vpu_shared_addr *shared, + u32 instance, enum vpu_malone_format malone_format) +{ + struct malone_iface *iface; + u32 *curr_str_cfg; + + iface = shared->iface; + curr_str_cfg = &iface->stream_config[instance]; + + *curr_str_cfg = 0; + STREAM_CONFIG_FORMAT_SET(malone_format, curr_str_cfg); + STREAM_CONFIG_STRBUFIDX_SET(0, curr_str_cfg); + STREAM_CONFIG_NOSEQ_SET(0, curr_str_cfg); + STREAM_CONFIG_DEBLOCK_SET(0, curr_str_cfg); + STREAM_CONFIG_DERING_SET(0, curr_str_cfg); + STREAM_CONFIG_PLAY_MODE_SET(0x3, curr_str_cfg); + STREAM_CONFIG_FS_CTRL_MODE_SET(0x1, curr_str_cfg); + STREAM_CONFIG_ENABLE_DCP_SET(1, curr_str_cfg); + STREAM_CONFIG_NUM_STR_BUF_SET(1, curr_str_cfg); + STREAM_CONFIG_MALONE_USAGE_SET(1, curr_str_cfg); + STREAM_CONFIG_MULTI_VID_SET(0, curr_str_cfg); + STREAM_CONFIG_OBFUSC_EN_SET(0, curr_str_cfg); + STREAM_CONFIG_RC4_EN_SET(0, curr_str_cfg); + STREAM_CONFIG_MCX_SET(1, curr_str_cfg); + STREAM_CONFIG_PES_SET(0, curr_str_cfg); + STREAM_CONFIG_NUM_DBE_SET(1, curr_str_cfg); +} + +static int vpu_malone_set_params(struct vpu_shared_addr *shared, + u32 instance, struct vpu_decode_params *params) +{ + struct malone_iface *iface; + struct vpu_dec_ctrl *hc; + enum vpu_malone_format malone_format; + + iface = shared->iface; + hc = shared->priv; + malone_format = vpu_malone_format_remap(params->codec_format); + iface->udata_buffer[instance].base = params->udata.base; + iface->udata_buffer[instance].slot_size = params->udata.size; + + vpu_malone_set_stream_cfg(shared, instance, malone_format); + + if (malone_format == MALONE_FMT_JPG) { + //1:JPGD_MJPEG_MODE_A; 2:JPGD_MJPEG_MODE_B + hc->jpg[instance].jpg_mjpeg_mode = 1; + //0: JPGD_MJPEG_PROGRESSIVE + hc->jpg[instance].jpg_mjpeg_interlaced = 0; + } + + hc->codec_param[instance].disp_imm = params->b_dis_reorder ? 1 : 0; + hc->codec_param[instance].dbglog_enable = 0; + iface->dbglog_desc.level = 0; + + if (params->b_non_frame) + iface->stream_buff_info[instance].stream_input_mode = NON_FRAME_LVL; + else + iface->stream_buff_info[instance].stream_input_mode = FRAME_LVL; + iface->stream_buff_info[instance].stream_buffer_threshold = 0; + iface->stream_buff_info[instance].stream_pic_input_count = 0; + + return 0; +} + +static bool vpu_malone_is_non_frame_mode(struct vpu_shared_addr *shared, + u32 instance) +{ + struct malone_iface *iface; + + iface = shared->iface; + if (iface->stream_buff_info[instance].stream_input_mode == NON_FRAME_LVL) + return true; + + return false; +} + +static int vpu_malone_update_params(struct vpu_shared_addr *shared, + u32 instance, struct vpu_decode_params *params) +{ + struct malone_iface *iface; + + iface = shared->iface; + + if (params->end_flag) + iface->stream_buff_info[instance].stream_pic_end_flag = params->end_flag; + params->end_flag = 0; + + return 0; +} + +int vpu_malone_set_decode_params(struct vpu_shared_addr *shared, + u32 instance, struct vpu_decode_params *params, u32 update) +{ + if (!params) + return -EINVAL; + + if (!update) + return vpu_malone_set_params(shared, instance, params); + else + return vpu_malone_update_params(shared, instance, params); +} + +static struct vpu_pair malone_cmds[] = { + {VPU_CMD_ID_START, VID_API_CMD_START}, + {VPU_CMD_ID_STOP, VID_API_CMD_STOP}, + {VPU_CMD_ID_ABORT, VID_API_CMD_ABORT}, + {VPU_CMD_ID_RST_BUF, VID_API_CMD_RST_BUF}, + {VPU_CMD_ID_SNAPSHOT, VID_API_CMD_SNAPSHOT}, + {VPU_CMD_ID_FIRM_RESET, VID_API_CMD_FIRM_RESET}, + {VPU_CMD_ID_FS_ALLOC, VID_API_CMD_FS_ALLOC}, + {VPU_CMD_ID_FS_RELEASE, VID_API_CMD_FS_RELEASE}, + {VPU_CMD_ID_TIMESTAMP, VID_API_CMD_TS}, + {VPU_CMD_ID_DEBUG, VID_API_CMD_FW_STATUS}, +}; + +static struct vpu_pair malone_msgs[] = { + {VPU_MSG_ID_RESET_DONE, VID_API_EVENT_RESET_DONE}, + {VPU_MSG_ID_START_DONE, VID_API_EVENT_START_DONE}, + {VPU_MSG_ID_STOP_DONE, VID_API_EVENT_STOPPED}, + {VPU_MSG_ID_ABORT_DONE, VID_API_EVENT_ABORT_DONE}, + {VPU_MSG_ID_BUF_RST, VID_API_EVENT_STR_BUF_RST}, + {VPU_MSG_ID_PIC_EOS, VID_API_EVENT_FINISHED}, + {VPU_MSG_ID_SEQ_HDR_FOUND, VID_API_EVENT_SEQ_HDR_FOUND}, + {VPU_MSG_ID_RES_CHANGE, VID_API_EVENT_RES_CHANGE}, + {VPU_MSG_ID_PIC_HDR_FOUND, VID_API_EVENT_PIC_HDR_FOUND}, + {VPU_MSG_ID_PIC_DECODED, VID_API_EVENT_PIC_DECODED}, + {VPU_MSG_ID_DEC_DONE, VID_API_EVENT_FRAME_BUFF_RDY}, + {VPU_MSG_ID_FRAME_REQ, VID_API_EVENT_REQ_FRAME_BUFF}, + {VPU_MSG_ID_FRAME_RELEASE, VID_API_EVENT_REL_FRAME_BUFF}, + {VPU_MSG_ID_FIFO_LOW, VID_API_EVENT_FIFO_LOW}, + {VPU_MSG_ID_BS_ERROR, VID_API_EVENT_BS_ERROR}, + {VPU_MSG_ID_UNSUPPORTED, VID_API_EVENT_UNSUPPORTED_STREAM}, + {VPU_MSG_ID_FIRMWARE_XCPT, VID_API_EVENT_FIRMWARE_XCPT}, +}; + +static void vpu_malone_pack_fs_alloc(struct vpu_rpc_event *pkt, + struct vpu_fs_info *fs) +{ + const u32 fs_type[] = { + [MEM_RES_FRAME] = 0, + [MEM_RES_MBI] = 1, + [MEM_RES_DCP] = 2, + }; + + pkt->hdr.num = 7; + pkt->data[0] = fs->id | (fs->tag << 24); + pkt->data[1] = fs->luma_addr; + if (fs->type == MEM_RES_FRAME) { + /* + * if luma_addr equal to chroma_addr, + * means luma(plane[0]) and chromau(plane[1]) used the + * same fd -- usage of NXP codec2. Need to manually + * offset chroma addr. + */ + if (fs->luma_addr == fs->chroma_addr) + fs->chroma_addr = fs->luma_addr + fs->luma_size; + pkt->data[2] = fs->luma_addr + fs->luma_size / 2; + pkt->data[3] = fs->chroma_addr; + pkt->data[4] = fs->chroma_addr + fs->chromau_size / 2; + pkt->data[5] = fs->bytesperline; + } else { + pkt->data[2] = fs->luma_size; + pkt->data[3] = 0; + pkt->data[4] = 0; + pkt->data[5] = 0; + } + pkt->data[6] = fs_type[fs->type]; +} + +static void vpu_malone_pack_fs_release(struct vpu_rpc_event *pkt, + struct vpu_fs_info *fs) +{ + pkt->hdr.num = 1; + pkt->data[0] = fs->id | (fs->tag << 24); +} + +static void vpu_malone_pack_timestamp(struct vpu_rpc_event *pkt, + struct vpu_ts_info *info) +{ + pkt->hdr.num = 3; + if (info->timestamp < 0) { + pkt->data[0] = (u32)-1; + pkt->data[1] = 0; + } else { + pkt->data[0] = info->timestamp / NSEC_PER_SEC; + pkt->data[1] = info->timestamp % NSEC_PER_SEC; + } + pkt->data[2] = info->size; +} + +int vpu_malone_pack_cmd(struct vpu_rpc_event *pkt, u32 index, u32 id, void *data) +{ + int ret; + + WARN_ON(!pkt); + + ret = vpu_find_dst_by_src(malone_cmds, ARRAY_SIZE(malone_cmds), id); + if (ret < 0) + return ret; + + pkt->hdr.id = ret; + pkt->hdr.num = 0; + pkt->hdr.index = index; + + switch (id) { + case VPU_CMD_ID_FS_ALLOC: + vpu_malone_pack_fs_alloc(pkt, data); + break; + case VPU_CMD_ID_FS_RELEASE: + vpu_malone_pack_fs_release(pkt, data); + break; + case VPU_CMD_ID_TIMESTAMP: + vpu_malone_pack_timestamp(pkt, data); + break; + } + + pkt->hdr.index = index; + return 0; +} + +int vpu_malone_convert_msg_id(u32 id) +{ + return vpu_find_src_by_dst(malone_msgs, ARRAY_SIZE(malone_msgs), id); +} + +static void vpu_malone_fill_planes(struct vpu_dec_codec_info *info) +{ + u32 interlaced = info->progressive ? 0 : 1; + + info->sizeimage[0] = vpu_helper_get_plane_size(info->pixfmt, + info->decoded_width, info->decoded_height, + 0, info->stride, interlaced, + &info->bytesperline[0]); + info->sizeimage[1] = vpu_helper_get_plane_size(info->pixfmt, + info->decoded_width, info->decoded_height, + 1, info->stride, interlaced, + &info->bytesperline[1]); +} + +void vpu_malone_init_seq_hdr(struct vpu_dec_codec_info *info) +{ + u32 chunks = info->num_dfe_area >> MALONE_DCP_CHUNK_BIT; + + vpu_malone_fill_planes(info); + + info->mbi_size = (info->sizeimage[0] + info->sizeimage[1]) >> 2; + info->mbi_size = ALIGN(info->mbi_size, MALONE_ALIGN_MBI); + + info->dcp_size = MALONE_DCP_SIZE_MAX; + if (chunks) { + u32 mb_num; + u32 mb_w; + u32 mb_h; + + mb_w = DIV_ROUND_UP(info->decoded_width, 16); + mb_h = DIV_ROUND_UP(info->decoded_height, 16); + mb_num = mb_w * mb_h; + info->dcp_size = mb_num * MALONE_DCP_FIXED_MB_ALLOC * chunks; + info->dcp_size = clamp_t(u32, info->dcp_size, + MALONE_DCP_SIZE_MIN, MALONE_DCP_SIZE_MAX); + } +} + +static void vpu_malone_unpack_seq_hdr(struct vpu_rpc_event *pkt, + struct vpu_dec_codec_info *info) +{ + info->num_ref_frms = pkt->data[0]; + info->num_dpb_frms = pkt->data[1]; + info->num_dfe_area = pkt->data[2]; + info->progressive = pkt->data[3]; + info->width = pkt->data[5]; + info->height = pkt->data[4]; + info->decoded_width = pkt->data[12]; + info->decoded_height = pkt->data[11]; + info->frame_rate = pkt->data[8]; + info->dsp_asp_ratio = pkt->data[9]; + info->level_idc = pkt->data[10]; + info->bit_depth_luma = pkt->data[13]; + info->bit_depth_chroma = pkt->data[14]; + info->chroma_fmt = pkt->data[15]; + info->color_primaries = vpu_color_cvrt_primaries_i2v(pkt->data[16]); + info->transfer_chars = vpu_color_cvrt_transfers_i2v(pkt->data[17]); + info->matrix_coeffs = vpu_color_cvrt_matrix_i2v(pkt->data[18]); + info->full_range = vpu_color_cvrt_full_range_i2v(pkt->data[19]); + info->vui_present = pkt->data[20]; + info->mvc_num_views = pkt->data[21]; + info->offset_x = pkt->data[23]; + info->offset_y = pkt->data[25]; + info->tag = pkt->data[27]; + if (info->bit_depth_luma > 8) + info->pixfmt = V4L2_PIX_FMT_NV12_10BE_8L128; + else + info->pixfmt = V4L2_PIX_FMT_NV12_8L128; + vpu_malone_init_seq_hdr(info); +} + +static void vpu_malone_unpack_pic_info(struct vpu_rpc_event *pkt, + struct vpu_dec_pic_info *info) +{ + info->id = pkt->data[7]; + info->luma = pkt->data[0]; + info->start = pkt->data[10]; + info->end = pkt->data[12]; + info->pic_size = pkt->data[11]; + info->stride = pkt->data[5]; + info->consumed_count = pkt->data[13]; + if (info->id == MALONE_SKIPPED_FRAME_ID) + info->skipped = 1; + else + info->skipped = 0; +} + +static void vpu_malone_unpack_req_frame(struct vpu_rpc_event *pkt, + struct vpu_fs_info *info) +{ + info->type = pkt->data[1]; +} + +static void vpu_malone_unpack_rel_frame(struct vpu_rpc_event *pkt, + struct vpu_fs_info *info) +{ + info->id = pkt->data[0]; + info->type = pkt->data[1]; + info->not_displayed = pkt->data[2]; +} + +static void vpu_malone_unpack_buff_rdy(struct vpu_rpc_event *pkt, + struct vpu_dec_pic_info *info) +{ + info->id = pkt->data[0]; + info->luma = pkt->data[1]; + info->stride = pkt->data[3]; + if (info->id == MALONE_SKIPPED_FRAME_ID) + info->skipped = 1; + else + info->skipped = 0; + info->timestamp = MAKE_TIMESTAMP(pkt->data[9], pkt->data[10]); +} + +int vpu_malone_unpack_msg_data(struct vpu_rpc_event *pkt, void *data) +{ + if (!pkt || !data) + return -EINVAL; + + switch (pkt->hdr.id) { + case VID_API_EVENT_SEQ_HDR_FOUND: + vpu_malone_unpack_seq_hdr(pkt, data); + break; + case VID_API_EVENT_PIC_DECODED: + vpu_malone_unpack_pic_info(pkt, data); + break; + case VID_API_EVENT_REQ_FRAME_BUFF: + vpu_malone_unpack_req_frame(pkt, data); + break; + case VID_API_EVENT_REL_FRAME_BUFF: + vpu_malone_unpack_rel_frame(pkt, data); + break; + case VID_API_EVENT_FRAME_BUFF_RDY: + vpu_malone_unpack_buff_rdy(pkt, data); + break; + } + + return 0; +} + +static const struct malone_padding_scode padding_scodes[] = { + {SCODE_PADDING_EOS, V4L2_PIX_FMT_H264, {0x0B010000, 0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_H264_MVC, {0x0B010000, 0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_HEVC, {0x4A010000, 0x20}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_VC1_ANNEX_G, {0x0a010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_VC1_ANNEX_L, {0x0a010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_MPEG2, {0xCC010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_MPEG4, {0xb1010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_XVID, {0xb1010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_H263, {0xb1010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_VP8, {0x34010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_JPEG, {0xefff0000, 0x0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_H264, {0x0B010000, 0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_H264_MVC, {0x0B010000, 0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_HEVC, {0x4A010000, 0x20}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_VC1_ANNEX_G, {0x0a010000, 0x0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_VC1_ANNEX_L, {0x0a010000, 0x0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_MPEG2, {0xb7010000, 0x0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_MPEG4, {0xb1010000, 0x0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_XVID, {0xb1010000, 0x0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_H263, {0xb1010000, 0x0}}, + {SCODE_PADDING_ABORT, V4L2_PIX_FMT_VP8, {0x34010000, 0x0}}, + {SCODE_PADDING_EOS, V4L2_PIX_FMT_JPEG, {0x0, 0x0}}, + {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264, {0x15010000, 0x0}}, + {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264_MVC, {0x15010000, 0x0}}, +}; +static const struct malone_padding_scode padding_scode_dft = {0x0, 0x0}; + +static const struct malone_padding_scode *get_padding_scode(u32 type, u32 fmt) +{ + const struct malone_padding_scode *s; + int i; + + for (i = 0; i < ARRAY_SIZE(padding_scodes); i++) { + s = &padding_scodes[i]; + + if (s->scode_type == type && s->pixelformat == fmt) + return s; + } + + if (type != SCODE_PADDING_BUFFLUSH) + return &padding_scode_dft; + + return NULL; +} + +static int vpu_malone_add_padding_scode(struct vpu_buffer *stream_buffer, + struct vpu_malone_str_buffer *str_buf, + u32 pixelformat, u32 scode_type) +{ + u32 wptr; + u32 size; + u32 total_size = 0; + const struct malone_padding_scode *ps; + const u32 padding_size = 4096; + int ret; + + ps = get_padding_scode(scode_type, pixelformat); + if (!ps) + return -EINVAL; + + wptr = str_buf->wptr; + size = ALIGN(wptr, 4) - wptr; + if (size) + vpu_helper_memset_stream_buffer(stream_buffer, &wptr, 0, size); + total_size += size; + + size = sizeof(ps->data); + ret = vpu_helper_copy_to_stream_buffer(stream_buffer, &wptr, size, (void *)ps->data); + if (ret < size) + return -EINVAL; + total_size += size; + + size = padding_size - sizeof(ps->data); + vpu_helper_memset_stream_buffer(stream_buffer, &wptr, 0, size); + total_size += size; + + vpu_malone_update_wptr(str_buf, wptr); + return total_size; +} + +int vpu_malone_add_scode(struct vpu_shared_addr *shared, + u32 instance, + struct vpu_buffer *stream_buffer, + u32 pixelformat, + u32 scode_type) +{ + struct vpu_dec_ctrl *hc; + struct vpu_malone_str_buffer *str_buf; + int ret = -EINVAL; + + WARN_ON(!shared || !shared->iface || !shared->core || !shared->priv); + + hc = shared->priv; + str_buf = hc->str_buf[instance]; + + switch (scode_type) { + case SCODE_PADDING_EOS: + case SCODE_PADDING_ABORT: + case SCODE_PADDING_BUFFLUSH: + ret = vpu_malone_add_padding_scode(stream_buffer, + str_buf, pixelformat, scode_type); + break; + default: + break; + } + + return ret; +} + +#define MALONE_PAYLOAD_HEADER_SIZE 16 +#define MALONE_CODEC_VERSION_ID 0x1 +#define MALONE_CODEC_ID_VC1_SIMPLE 0x10 +#define MALONE_CODEC_ID_VC1_MAIN 0x11 +#define MALONE_CODEC_ID_ARV8 0x28 +#define MALONE_CODEC_ID_ARV9 0x29 +#define MALONE_CODEC_ID_VP6 0x36 +#define MALONE_CODEC_ID_VP8 0x36 +#define MALONE_CODEC_ID_DIVX3 0x38 +#define MALONE_CODEC_ID_SPK 0x39 + +#define MALONE_VP8_IVF_SEQ_HEADER_LEN 32 +#define MALONE_VP8_IVF_FRAME_HEADER_LEN 8 + +#define MALONE_VC1_RCV_CODEC_V1_VERSION 0x85 +#define MALONE_VC1_RCV_CODEC_V2_VERSION 0xC5 +#define MALONE_VC1_RCV_NUM_FRAMES 0xFF +#define MALONE_VC1_RCV_SEQ_EXT_DATA_SIZE 4 +#define MALONE_VC1_RCV_SEQ_HEADER_LEN 20 +#define MALONE_VC1_RCV_PIC_HEADER_LEN 4 +#define MALONE_VC1_NAL_HEADER_LEN 4 +#define MALONE_VC1_CONTAIN_NAL(data) ((data & 0x00FFFFFF) == 0x00010000) + + +static void set_payload_hdr(u8 *dst, u32 scd_type, u32 codec_id, + u32 buffer_size, u32 width, u32 height) +{ + unsigned int payload_size; + /* payload_size = buffer_size + itself_size(16) - start_code(4) */ + payload_size = buffer_size + 12; + + dst[0] = 0x00; + dst[1] = 0x00; + dst[2] = 0x01; + dst[3] = scd_type; + + /* length */ + dst[4] = ((payload_size>>16)&0xff); + dst[5] = ((payload_size>>8)&0xff); + dst[6] = 0x4e; + dst[7] = ((payload_size>>0)&0xff); + + /* Codec ID and Version */ + dst[8] = codec_id; + dst[9] = MALONE_CODEC_VERSION_ID; + + /* width */ + dst[10] = ((width>>8)&0xff); + dst[11] = ((width>>0)&0xff); + dst[12] = 0x58; + + /* height */ + dst[13] = ((height>>8)&0xff); + dst[14] = ((height>>0)&0xff); + dst[15] = 0x50; +} + +static void set_vp8_ivf_seqhdr(u8 *dst, u32 width, u32 height) +{ + /* 0-3byte signature "DKIF" */ + dst[0] = 0x44; + dst[1] = 0x4b; + dst[2] = 0x49; + dst[3] = 0x46; + /* 4-5byte version: should be 0*/ + dst[4] = 0x00; + dst[5] = 0x00; + /* 6-7 length of Header */ + dst[6] = MALONE_VP8_IVF_SEQ_HEADER_LEN; + dst[7] = MALONE_VP8_IVF_SEQ_HEADER_LEN >> 8; + /* 8-11 VP8 fourcc */ + dst[8] = 0x56; + dst[9] = 0x50; + dst[10] = 0x38; + dst[11] = 0x30; + /* 12-13 width in pixels */ + dst[12] = width; + dst[13] = width >> 8; + /* 14-15 height in pixels */ + dst[14] = height; + dst[15] = height >> 8; + /* 16-19 frame rate */ + dst[16] = 0xe8; + dst[17] = 0x03; + dst[18] = 0x00; + dst[19] = 0x00; + /* 20-23 time scale */ + dst[20] = 0x01; + dst[21] = 0x00; + dst[22] = 0x00; + dst[23] = 0x00; + /* 24-27 number frames */ + dst[24] = 0xdf; + dst[25] = 0xf9; + dst[26] = 0x09; + dst[27] = 0x00; + /* 28-31 reserved */ +} + +static void set_vp8_ivf_pichdr(u8 *dst, u32 frame_size) +{ + /* + * firmware just parse 64-bit timestamp(8 bytes). + * As not transfer timestamp to firmware, use default value(ZERO). + * No need to do anything here + */ +} + +static void set_vc1_rcv_seqhdr(u8 *dst, u8 *src, u32 width, u32 height) +{ + u32 frames = MALONE_VC1_RCV_NUM_FRAMES; + u32 ext_data_size = MALONE_VC1_RCV_SEQ_EXT_DATA_SIZE; + + /* 0-2 Number of frames, used default value 0xFF */ + dst[0] = frames; + dst[1] = frames >> 8; + dst[2] = frames >> 16; + + /* 3 RCV version, used V1 */ + dst[3] = MALONE_VC1_RCV_CODEC_V1_VERSION; + + /* 4-7 extension data size */ + dst[4] = ext_data_size; + dst[5] = ext_data_size >> 8; + dst[6] = ext_data_size >> 16; + dst[7] = ext_data_size >> 24; + /* 8-11 extension data */ + dst[8] = src[0]; + dst[9] = src[1]; + dst[10] = src[2]; + dst[11] = src[3]; + + /* height */ + dst[12] = height; + dst[13] = (height >> 8) & 0xff; + dst[14] = (height >> 16) & 0xff; + dst[15] = (height >> 24) & 0xff; + /* width */ + dst[16] = width; + dst[17] = (width >> 8) & 0xff; + dst[18] = (width >> 16) & 0xff; + dst[19] = (width >> 24) & 0xff; +} + +static void set_vc1_rcv_pichdr(u8 *dst, u32 buffer_size) +{ + dst[0] = buffer_size; + dst[1] = buffer_size >> 8; + dst[2] = buffer_size >> 16; + dst[3] = buffer_size >> 24; +} + +static void create_vc1_nal_pichdr(u8 *dst) +{ + /* need insert nal header: special ID */ + dst[0] = 0x0; + dst[1] = 0x0; + dst[2] = 0x01; + dst[3] = 0x0D; +} + +static int vpu_malone_insert_scode_seq(struct malone_scode_t *scode, u32 codec_id, u32 ext_size) +{ + u8 hdr[MALONE_PAYLOAD_HEADER_SIZE]; + int ret; + + set_payload_hdr(hdr, + SCODE_SEQUENCE, + codec_id, + ext_size, + scode->inst->out_format.width, + scode->inst->out_format.height); + ret = vpu_helper_copy_to_stream_buffer(&scode->inst->stream_buffer, + &scode->wptr, + sizeof(hdr), + hdr); + return ret; +} + +static int vpu_malone_insert_scode_pic(struct malone_scode_t *scode, u32 codec_id, u32 ext_size) +{ + u8 hdr[MALONE_PAYLOAD_HEADER_SIZE]; + int ret; + + set_payload_hdr(hdr, + SCODE_PICTURE, + codec_id, + ext_size + vb2_get_plane_payload(scode->vb, 0), + scode->inst->out_format.width, + scode->inst->out_format.height); + ret = vpu_helper_copy_to_stream_buffer(&scode->inst->stream_buffer, + &scode->wptr, + sizeof(hdr), + hdr); + return 0; +} + +static int vpu_malone_insert_scode_vc1_g_pic(struct malone_scode_t *scode) +{ + struct vb2_v4l2_buffer *vbuf; + u8 nal_hdr[MALONE_VC1_NAL_HEADER_LEN]; + u32 *data = NULL; + + vbuf = to_vb2_v4l2_buffer(scode->vb); + data = vb2_plane_vaddr(scode->vb, 0); + + if (vbuf->sequence == 0 || (vbuf->flags & V4L2_BUF_FLAG_CODECCONFIG)) + return 0; + if (MALONE_VC1_CONTAIN_NAL(*data)) + return 0; + + create_vc1_nal_pichdr(nal_hdr); + return vpu_helper_copy_to_stream_buffer(&scode->inst->stream_buffer, + &scode->wptr, + sizeof(nal_hdr), + nal_hdr); +} + +static int vpu_malone_insert_scode_vc1_l_seq(struct malone_scode_t *scode) +{ + int ret; + int size = 0; + u8 rcv_seqhdr[MALONE_VC1_RCV_SEQ_HEADER_LEN]; + + scode->need_data = 0; + + ret = vpu_malone_insert_scode_seq(scode, MALONE_CODEC_ID_VC1_SIMPLE, + sizeof(rcv_seqhdr)); + if (ret < 0) + return ret; + size = ret; + + set_vc1_rcv_seqhdr(rcv_seqhdr, + vb2_plane_vaddr(scode->vb, 0), + scode->inst->out_format.width, + scode->inst->out_format.height); + ret = vpu_helper_copy_to_stream_buffer(&scode->inst->stream_buffer, + &scode->wptr, + sizeof(rcv_seqhdr), + rcv_seqhdr); + + if (ret < 0) + return ret; + size += ret; + return size; +} + +static int vpu_malone_insert_scode_vc1_l_pic(struct malone_scode_t *scode) +{ + int ret; + int size = 0; + u8 rcv_pichdr[MALONE_VC1_RCV_PIC_HEADER_LEN]; + + ret = vpu_malone_insert_scode_pic(scode, MALONE_CODEC_ID_VC1_SIMPLE, + sizeof(rcv_pichdr)); + if (ret < 0) + return ret; + size = ret; + + set_vc1_rcv_pichdr(rcv_pichdr, vb2_get_plane_payload(scode->vb, 0)); + ret = vpu_helper_copy_to_stream_buffer(&scode->inst->stream_buffer, + &scode->wptr, + sizeof(rcv_pichdr), + rcv_pichdr); + if (ret < 0) + return ret; + size += ret; + return size; +} + +static int vpu_malone_insert_scode_vp8_seq(struct malone_scode_t *scode) +{ + int ret; + int size = 0; + u8 ivf_hdr[MALONE_VP8_IVF_SEQ_HEADER_LEN]; + + ret = vpu_malone_insert_scode_seq(scode, MALONE_CODEC_ID_VP8, sizeof(ivf_hdr)); + if (ret < 0) + return ret; + size = ret; + + set_vp8_ivf_seqhdr(ivf_hdr, + scode->inst->out_format.width, + scode->inst->out_format.height); + ret = vpu_helper_copy_to_stream_buffer(&scode->inst->stream_buffer, + &scode->wptr, + sizeof(ivf_hdr), + ivf_hdr); + if (ret < 0) + return ret; + size += ret; + + return size; +} + +static int vpu_malone_insert_scode_vp8_pic(struct malone_scode_t *scode) +{ + int ret; + int size = 0; + u8 ivf_hdr[MALONE_VP8_IVF_FRAME_HEADER_LEN] = {0}; + + ret = vpu_malone_insert_scode_pic(scode, MALONE_CODEC_ID_VP8, sizeof(ivf_hdr)); + if (ret < 0) + return ret; + size = ret; + + set_vp8_ivf_pichdr(ivf_hdr, vb2_get_plane_payload(scode->vb, 0)); + ret = vpu_helper_copy_to_stream_buffer(&scode->inst->stream_buffer, + &scode->wptr, + sizeof(ivf_hdr), + ivf_hdr); + if (ret < 0) + return ret; + size += ret; + + return size; +} + +static const struct malone_scode_handler scode_handlers[] = { + { + /* fix me, need to swap return operation after gstreamer swap */ + .pixelformat = V4L2_PIX_FMT_VC1_ANNEX_L, + .insert_scode_seq = vpu_malone_insert_scode_vc1_l_seq, + .insert_scode_pic = vpu_malone_insert_scode_vc1_l_pic, + }, + { + .pixelformat = V4L2_PIX_FMT_VC1_ANNEX_G, + .insert_scode_pic = vpu_malone_insert_scode_vc1_g_pic, + }, + { + .pixelformat = V4L2_PIX_FMT_VP8, + .insert_scode_seq = vpu_malone_insert_scode_vp8_seq, + .insert_scode_pic = vpu_malone_insert_scode_vp8_pic, + }, +}; + +const struct malone_scode_handler *get_scode_handler(u32 pixelformat) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(scode_handlers); i++) { + if (scode_handlers[i].pixelformat == pixelformat) + return &scode_handlers[i]; + } + + return NULL; +} + +static int vpu_malone_insert_scode(struct malone_scode_t *scode, u32 type) +{ + const struct malone_scode_handler *handler; + int ret = 0; + + if (!scode || !scode->inst || !scode->vb) + return 0; + + scode->need_data = 1; + handler = get_scode_handler(scode->inst->out_format.pixfmt); + if (!handler) + return 0; + + switch (type) { + case SCODE_SEQUENCE: + if (handler->insert_scode_seq) + ret = handler->insert_scode_seq(scode); + break; + case SCODE_PICTURE: + if (handler->insert_scode_pic) + ret = handler->insert_scode_pic(scode); + break; + default: + break; + } + + return ret; +} + +static int vpu_malone_input_frame_data(struct vpu_malone_str_buffer *str_buf, + struct vpu_inst *inst, struct vb2_buffer *vb, + u32 disp_imm) +{ + struct malone_scode_t scode; + struct vb2_v4l2_buffer *vbuf; + u32 wptr; + int size = 0; + int ret = 0; + + wptr = str_buf->wptr; + + /*add scode: SCODE_SEQUENCE, SCODE_PICTURE, SCODE_SLICE*/ + vbuf = to_vb2_v4l2_buffer(vb); + scode.inst = inst; + scode.vb = vb; + scode.wptr = wptr; + scode.need_data = 1; + if (vbuf->sequence == 0 || (vbuf->flags & V4L2_BUF_FLAG_CODECCONFIG)) + ret = vpu_malone_insert_scode(&scode, SCODE_SEQUENCE); + + if (ret < 0) + return -ENOMEM; + size += ret; + wptr = scode.wptr; + if (!scode.need_data) { + vpu_malone_update_wptr(str_buf, wptr); + return size; + } + + ret = vpu_malone_insert_scode(&scode, SCODE_PICTURE); + if (ret < 0) + return -ENOMEM; + size += ret; + wptr = scode.wptr; + + ret = vpu_helper_copy_to_stream_buffer(&inst->stream_buffer, + &wptr, + vb2_get_plane_payload(vb, 0), + vb2_plane_vaddr(vb, 0)); + if (ret < vb2_get_plane_payload(vb, 0)) + return -ENOMEM; + size += ret; + + vpu_malone_update_wptr(str_buf, wptr); + + if (disp_imm && !(vbuf->flags & V4L2_BUF_FLAG_CODECCONFIG)) { + ret = vpu_malone_add_scode(inst->core->iface, + inst->id, + &inst->stream_buffer, + inst->out_format.pixfmt, + SCODE_PADDING_BUFFLUSH); + if (ret < 0) + return ret; + size += ret; + } + + return size; +} + +static int vpu_malone_input_stream_data(struct vpu_malone_str_buffer *str_buf, + struct vpu_inst *inst, struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf; + u32 wptr; + int ret = 0; + + wptr = str_buf->wptr; + vbuf = to_vb2_v4l2_buffer(vb); + ret = vpu_helper_copy_to_stream_buffer(&inst->stream_buffer, + &wptr, + vb2_get_plane_payload(vb, 0), + vb2_plane_vaddr(vb, 0)); + if (ret < vb2_get_plane_payload(vb, 0)) + return -ENOMEM; + + vpu_malone_update_wptr(str_buf, wptr); + + return ret; +} + +static int vpu_malone_input_ts(struct vpu_inst *inst, s64 timestamp, u32 size) +{ + struct vpu_ts_info info; + + memset(&info, 0, sizeof(info)); + info.timestamp = timestamp; + info.size = size; + + return vpu_session_fill_timestamp(inst, &info); +} + +int vpu_malone_input_frame(struct vpu_shared_addr *shared, + struct vpu_inst *inst, struct vb2_buffer *vb) +{ + struct vpu_dec_ctrl *hc; + struct vb2_v4l2_buffer *vbuf; + struct vpu_malone_str_buffer *str_buf; + u32 disp_imm = 0; + u32 size; + int ret; + + WARN_ON(!shared || !shared->iface || !shared->core || !shared->priv); + hc = shared->priv; + str_buf = hc->str_buf[inst->id]; + disp_imm = hc->codec_param[inst->id].disp_imm; + + if (vpu_malone_is_non_frame_mode(shared, inst->id)) + ret = vpu_malone_input_stream_data(str_buf, inst, vb); + else + ret = vpu_malone_input_frame_data(str_buf, inst, vb, disp_imm); + if (ret < 0) + return ret; + size = ret; + + /* + * if buffer only contain codec data, and the timestamp is invalid, + * don't put the invalid timestamp to resync + * merge the data to next frame + */ + vbuf = to_vb2_v4l2_buffer(vb); + if ((vbuf->flags & V4L2_BUF_FLAG_CODECCONFIG) && (s64)vb->timestamp < 0) { + inst->extra_size += size; + return 0; + } + if (inst->extra_size) { + size += inst->extra_size; + inst->extra_size = 0; + } + + ret = vpu_malone_input_ts(inst, vb->timestamp, size); + if (ret) + return ret; + + return 0; +} + +static bool vpu_malone_check_ready(struct vpu_shared_addr *shared, u32 instance) +{ + struct malone_iface *iface; + struct vpu_rpc_buffer_desc *desc; + u32 size; + u32 rptr; + u32 wptr; + u32 used; + + iface = shared->iface; + desc = &iface->api_cmd_buffer_desc[instance]; + size = desc->end - desc->start; + rptr = desc->rptr; + wptr = desc->wptr; + used = (wptr + size - rptr) % size; + if (!size || used < size / 2) + return true; + + vpu_dbg(LVL_DEBUG, "[%d]api cmd used = %d/%d\n", instance, used, size); + + return false; +} + +bool vpu_malone_is_ready(struct vpu_shared_addr *shared, u32 instance) +{ + u32 cnt = 0; + + while (!vpu_malone_check_ready(shared, instance)) { + if (cnt > 30) + return false; + mdelay(1); + cnt++; + } + return true; +} + +int vpu_malone_pre_cmd(struct vpu_shared_addr *shared, u32 instance) +{ + if (!vpu_malone_is_ready(shared, instance)) { + vpu_err("[%d] is not ready\n", instance); + return -EINVAL; + } + + return 0; +} + +int vpu_malone_post_cmd(struct vpu_shared_addr *shared, u32 instance) +{ + struct malone_iface *iface; + struct vpu_rpc_buffer_desc *desc; + + iface = shared->iface; + desc = &iface->api_cmd_buffer_desc[instance]; + desc->wptr++; + if (desc->wptr == desc->end) + desc->wptr = desc->start; + + return 0; +} + +int vpu_malone_init_instance(struct vpu_shared_addr *shared, u32 instance) +{ + struct malone_iface *iface; + struct vpu_rpc_buffer_desc *desc; + + iface = shared->iface; + desc = &iface->api_cmd_buffer_desc[instance]; + desc->wptr = desc->rptr; + if (desc->wptr == desc->end) + desc->wptr = desc->start; + + return 0; +} + +u32 vpu_malone_get_max_instance_count(struct vpu_shared_addr *shared) +{ + struct malone_iface *iface = shared->iface; + + return iface->max_streams; +} diff --git a/drivers/media/platform/imx/vpu-8q/vpu_malone.h b/drivers/media/platform/imx/vpu-8q/vpu_malone.h new file mode 100644 index 000000000000..22c834a030a9 --- /dev/null +++ b/drivers/media/platform/imx/vpu-8q/vpu_malone.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2020-2021 NXP + */ + +#ifndef _IMX_VPU_MALONE_H +#define _IMX_VPU_MALONE_H + +u32 vpu_malone_get_data_size(void); +void vpu_malone_init_rpc(struct vpu_shared_addr *shared, + struct vpu_buffer *rpc, dma_addr_t boot_addr); +void vpu_malone_set_log_buf(struct vpu_shared_addr *shared, + struct vpu_buffer *log); +void vpu_malone_set_system_cfg(struct vpu_shared_addr *shared, + u32 regs_base, void __iomem *regs, u32 core_id); +u32 vpu_malone_get_version(struct vpu_shared_addr *shared); +int vpu_malone_get_stream_buffer_size(struct vpu_shared_addr *shared); +int vpu_malone_config_stream_buffer(struct vpu_shared_addr *shared, + u32 instance, struct vpu_buffer *buf); +int vpu_malone_get_stream_buffer_desc(struct vpu_shared_addr *shared, + u32 instance, struct vpu_rpc_buffer_desc *desc); +int vpu_malone_update_stream_buffer(struct vpu_shared_addr *shared, + u32 instance, u32 ptr, bool write); +int vpu_malone_set_decode_params(struct vpu_shared_addr *shared, + u32 instance, struct vpu_decode_params *params, u32 update); +int vpu_malone_pack_cmd(struct vpu_rpc_event *pkt, u32 index, u32 id, void *data); +int vpu_malone_convert_msg_id(u32 msg_id); +int vpu_malone_unpack_msg_data(struct vpu_rpc_event *pkt, void *data); +int vpu_malone_add_scode(struct vpu_shared_addr *shared, + u32 instance, + struct vpu_buffer *stream_buffer, + u32 pixelformat, + u32 scode_type); +int vpu_malone_input_frame(struct vpu_shared_addr *shared, + struct vpu_inst *inst, struct vb2_buffer *vb); +bool vpu_malone_is_ready(struct vpu_shared_addr *shared, u32 instance); +int vpu_malone_pre_cmd(struct vpu_shared_addr *shared, u32 instance); +int vpu_malone_post_cmd(struct vpu_shared_addr *shared, u32 instance); +int vpu_malone_init_instance(struct vpu_shared_addr *shared, u32 instance); +u32 vpu_malone_get_max_instance_count(struct vpu_shared_addr *shared); + +#endif From patchwork Wed Aug 11 08:43:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 495314 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49FDAC4338F for ; Wed, 11 Aug 2021 08:45:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2AC636104F for ; Wed, 11 Aug 2021 08:45:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236545AbhHKIpm (ORCPT ); Wed, 11 Aug 2021 04:45:42 -0400 Received: from mail-eopbgr80054.outbound.protection.outlook.com ([40.107.8.54]:2016 "EHLO EUR04-VI1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236493AbhHKIp3 (ORCPT ); Wed, 11 Aug 2021 04:45:29 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IFEn7k+7HwlCsMUMfZLSvhiheHxS1Bz2LDZNHcfFASJOtLZAiUb2KqCgtN+Xry9/Kms53Ja2Ibnz+h3qJ2LBM0zpTwRCibe1tdmYWVEQfYF8XRg23p2Vl5FYQj6CoH3oD3tY56C1hrctBrYp+i0Cf3cETtKkflUO1buF1jJ2V6mZViWnN6UU5TacZXEkkxij4kbqC+VNz8ctClpPApGZQ/gLIvvYyCdBsex1M+cugBlJoVDFn84QBT5MwgKma0ucIuL3er4FoqjtAVgZzF9WFqq27GuHdKN/ueXjISuiKlPjwkYR//5bzHPlm0BZNoSEmJytEPt/vRIW1ulNSG1tLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CphCoXulIFGtU2AEB4KsSkyW/W7RFji+tc3255MclSo=; b=On/2nhhRCOD5CTEi98hjVF9sBw2lKoyk953fNdn6fmNCy8kafzVJrV2L/lpwKHXP7fh6x72lwRiDYHMRaoHCpio/YumyP5Lgp/CvdZObu3gSNlB1hzSmv4Hf2PWvHL4T+7O7H/vWs5AjhcsVzlrCNzhEepocuURLpW1EbIqxuWBWSFr1JCICxExLmP6OWmwwCQrnBqdBvSzyPHmzJh80uHyiugM9nTqLhzwn5Mk0AqAdI8OXqbBr+F1JwQVIP7mKtpKV8mFnqyHZTV7xWvu9ymOpE/Y4851s7BQoLUT+58IeTGBtAhTWreU9oFiSllMQAS4yyzEM5J1WFC9FLu7+gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CphCoXulIFGtU2AEB4KsSkyW/W7RFji+tc3255MclSo=; b=K+lpAe1y84Z6lfhUhT+bvQwObkrNu8xuy8sFiERfumk6ABUObuPXRAw82UFahglRr2mfZbrGVQrAEAxTOs+4fkCTb2ePLmB4h9s6x7/XNFYHorcR4UnuZljiZQ9u0oc8t000fS4cyZkVAp6MLe8H4nbUpK6+aas7jzf+PKZLtWI= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM6PR04MB4888.eurprd04.prod.outlook.com (2603:10a6:20b:12::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.17; Wed, 11 Aug 2021 08:44:50 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384%6]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 08:44:50 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de Cc: hverkuil-cisco@xs4all.nl, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 13/14] firmware: imx: scu-pd: imx8q: add vpu mu resources Date: Wed, 11 Aug 2021 16:43:09 +0800 Message-Id: <6d0d2b8677ba40b8007aa46a02b676838f9b27d3.1628671163.git.ming.qian@nxp.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-ClientProxiedBy: SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.4 via Frontend Transport; Wed, 11 Aug 2021 08:44:46 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0a877644-22ca-47d0-6f04-08d95ca447ec X-MS-TrafficTypeDiagnostic: AM6PR04MB4888: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1201; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1BYIFyNOSExEEMFCYMngG3MKJ7H9CgDiHIYnPs81mKDuCtjZVFj7U5Yq8LqyIsIRzBzL8wHs3pZksD02knT6jzWn47ApI4V70y5FV0I9m9soTS26IEABXL/NCjEypEbUY6Ytv3+YzcuRGPC16ns/8f+X70Mu5GkZTBiOiSpQTICraiaM0GEEJbpIOn/uzyGcHKvgleOPP1TY6yU2dIqq2MxGKF4IQBg5x2k3Z9edT+GAY086r2p+LClRfngtnoNeGeVV3gqw20XEpS+0yu8sKCGsrFGwfHzTA0yMj1r12iuLghmBf2nULLTLSXd78dXLLMnNofe0qalgc9ws7EUPkm/7rY3Ka9oEQ78YP2SAJoiWSeMEIl09azk+NeAjPiRqD6qB1nbcZ8gLdD0X8ARMCkjIQGW4H2rCEQ7rGzq0ZdbFpMs9i9O9teCp0OTIMo5Ag+Dl8DNDBOFzVfslYmzw0+m4TJilbC7jgVBrsJwmyyeOldqiac0icWxSLQ0jvVdeSSctJPzEQYmSwm3cPCC5vnsHTbo+C3eV33jd4EKVJnS4zbecnI7tu5bwauNUg8cc9ErVF3nekcWgXsyR5cjbsTbs9VeDhOC3lbQzrhkY4vg0SeC9CKpHSmVsv+MnB8FdUyXWetMBh65Grzl3bJo7Bje80GkmGn5uBzu86QHPY5tuLXokCHjzrbqHuAPe050s0WzHcdw6HZA5cLXrwXRss5FugliWL6IptR0NHRk6F4g= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(376002)(366004)(396003)(136003)(39860400002)(7416002)(86362001)(8676002)(6486002)(36756003)(44832011)(2616005)(956004)(4326008)(7696005)(2906002)(38100700002)(4744005)(66476007)(478600001)(26005)(52116002)(66556008)(66946007)(38350700002)(186003)(8936002)(316002)(6666004)(5660300002)(32563001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Jl6FqAJkuOnFpHYXl5sQ9fmbirOjj9dVhjyBnjC5k1FLP6M1chBhWRusZ9Lm97UsoQOXrcl2ZNYSjl56vLuvAXgWBfu3H2Uy6bataonsNUyvhokyXg1YgtIrhZglUyblJFrppZHaAqf9moG1fYTaP5OtDukBDQW1rs17HS6+jhQYotPinZm049GGfLEg6eSkBvlnCGGyx8As4YoNzHq0eyhpFw1uuZdDIGpuVKxcOKTMbaIDAJi/BFpkUULRslElcv7vqcgWFfri5l+/im5rWUIZSFufynZPnZ3Z6sUMqEcWe8WGxfO9hA0CeYGsufufoi3kTAmiEZ9k9HSH8SpcFuqsA2Rek3srTWYLCjNXBb2dmzuEaj96MOBOkfp6APTja5p2Emxxu5x1v2tsS+ijHbWX/v3lbOSVH//mCxqfowOb/MTSz0B2MFr7DofSFhRHYb2MvyEf/MfzDELr40Aar9jUmQV1DmJ+EcpxrCQHVt8KdCIUFQUj8PKABpmCJ2gimi1uEr8vER3z4lptqcyiaTcLJMxWIpyj8eXvLl3a4UXvgbdMD5T2Mv2LoyuLTIdCh6id15Kdew+Bv23iiZ4Bo1Wa8j2v8rvhoXufUZYDSkmNQTEiFlU2Q39PVNFV6f28s4QuJ74762WJsZ2X45gkyjjPPJCLTLHeaETGbBDAWrVR1/XCEwTmPRZ+TNXqnzLHBFIIFvY/HSsdQEnBdWowwM6wFQRsSQ9bw0G1fETyTY9bAUfs8+0zY+5ZE2oxsMTqfWPjAcln3O37EHW/Ix/6RwgQSKSkWGbXyiDquv5j0XXr9IeeqYD0HLlSO9RQJ6BZPJYVhKdl7hggK3C6WIDPogzfVAYEifUUzksbbFvdv5BwK88TsUpKT8xiVO0OeGDSonmvm8Uc9M+un6vr+hv2UiL4c/IAaBVYhIQ1JRrbF9/45xkS+uMDw40yAXIGkis6HB6VgNGOtb/861Ytq6HmALCTh2EYI7hc4baefGlCuyhI1xYd2P9EbEgJuinacKw3YixHB45+WdIC1eL9SA8Y0ONJW6cUxF+MLBs1Iomd7WJDGfXtE98313BfoJhU5+wJlXKgRhGL0TC8w5CkOHG07FreyCoaf67X80qQbfnwVIaZAyzb/bEdBbRQF6q+4KMevod06uLyhaYGrEBrswyJvydEjpEFgkDJVgj26yNY+CHy2Ty5EbnILgR4P3mwfaBCPFftSlfTFW7k+dP/fJLbWDpMvn6YBJOGcEACE6Di+3OS8G+fJzNRgIp5fxhFjtQLUeZe4P2eXZNZPwRQgKcm69NZQmF8M3SpDo+ah380MNWHq/DdMB/jkF6B0P0AaGME X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a877644-22ca-47d0-6f04-08d95ca447ec X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 08:44:50.3653 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UEewjses7CDv6mIcHsjRGU4J74W1htvhbu7vShyjvsxs+kReHZkluq5ch/AEj+789IFDJT6kH66jyRetAeZ4mw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB4888 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org the vpu core depends on the mu resources. if they're missed, the vpu can't work. Signed-off-by: Ming Qian Signed-off-by: Shijie Qin Signed-off-by: Zhou Peng --- drivers/firmware/imx/scu-pd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/firmware/imx/scu-pd.c b/drivers/firmware/imx/scu-pd.c index ff6569c4a53b..af3d057e6421 100644 --- a/drivers/firmware/imx/scu-pd.c +++ b/drivers/firmware/imx/scu-pd.c @@ -155,6 +155,10 @@ static const struct imx_sc_pd_range imx8qxp_scu_pd_ranges[] = { { "vpu-pid", IMX_SC_R_VPU_PID0, 8, true, 0 }, { "vpu-dec0", IMX_SC_R_VPU_DEC_0, 1, false, 0 }, { "vpu-enc0", IMX_SC_R_VPU_ENC_0, 1, false, 0 }, + { "vpu-enc1", IMX_SC_R_VPU_ENC_1, 1, false, 0 }, + { "vpu-mu0", IMX_SC_R_VPU_MU_0, 1, false, 0 }, + { "vpu-mu1", IMX_SC_R_VPU_MU_1, 1, false, 0 }, + { "vpu-mu2", IMX_SC_R_VPU_MU_2, 1, false, 0 }, /* GPU SS */ { "gpu0-pid", IMX_SC_R_GPU_0_PID0, 4, true, 0 }, From patchwork Wed Aug 11 08:43:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 495313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5718FC4338F for ; Wed, 11 Aug 2021 08:45:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 379A961058 for ; Wed, 11 Aug 2021 08:45:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236371AbhHKIqB (ORCPT ); Wed, 11 Aug 2021 04:46:01 -0400 Received: from mail-eopbgr80054.outbound.protection.outlook.com ([40.107.8.54]:2016 "EHLO EUR04-VI1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236434AbhHKIpm (ORCPT ); Wed, 11 Aug 2021 04:45:42 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xu5OR/xdH2poyLE75ozMi2zagMZC68fuEBbcsLfiOyLrhq0to6J0/PTvVWiJ6WcqvWxPHsHgU32ySxzgMxBMF24q18x8jNRIPCgOnAcQSsA7MjSXdZe+lYA/aTc0uOULwV5OStZHzBfwtZxHkLlVO9p/NAkU0tWRCfXNY5ZQJBMbpIocDEMwc3gyfuJswdeIJlo9gQcoD3uTTaSdCAyPnPOypljvL2RSKYR5TKKrEIkpoJBQQ++2iyakMhv7jRWYrSBe2R13W1cNLFxYiTUhgqVWynWyErJNCTGOXybZ5OwMpIso6tCVhyKHHDfvd3M0ESVeDT0RYVYHhcYtn5t7Dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZIxbb9rEHNY5D28FLfrsP7F6i+sjxtTYatIyhaTEdKY=; b=WJV1ylruJgPwdDja/+c/p0ofZhbuvX4wV9Ca6DUDOzeIoDMzHaCORsOOUdLUEOQBbM1eHh4cfB5t0ZzjKwTFx08L9zZigLGYCwMavzWOpifmlEPbQELPrWi5U5gMLiBPHLCcqUt7KEdGzUY5AjI4QCQI0vTfA8IvIPTAutlC0qAugPmkhYVur3+HY2F0wvcxlpX28VltXuOZFboo8ohVQdoClG70brhyHnqfIs8zL5zEQ0PvcZKlAwoiI7yapvLq9FR9carP5Uc9UNujygqD7Z2g8PjjWP/DLMHVxKCmEo88AsHL5ZC/1jmWBVUJC0bKcGdOauQdl2O16dQukXVxCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZIxbb9rEHNY5D28FLfrsP7F6i+sjxtTYatIyhaTEdKY=; b=iq6Ieg9ZgZ7HZApN8soBmwui1kCsgkR3jNzNoF9JA3DwY7hPTyl180krYzoLKB9rr9W5gQGKkgSTwKkvfgByKY0540+71S5A2RXw8lgbVgB9dBbh+0HV6NNz3lpw/8A8V81xDGXWnb7BYfnIlqKG7cCNFZ8k7zvmiUuNiJTqv4E= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=none action=none header.from=nxp.com; Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM6PR04MB4888.eurprd04.prod.outlook.com (2603:10a6:20b:12::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.17; Wed, 11 Aug 2021 08:44:54 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::9cb5:bd97:8500:2384%6]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 08:44:54 +0000 From: Ming Qian To: mchehab@kernel.org, shawnguo@kernel.org, robh+dt@kernel.org, s.hauer@pengutronix.de Cc: hverkuil-cisco@xs4all.nl, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, aisheng.dong@nxp.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 14/14] MAINTAINERS: add NXP IMX8Q VPU CODEC V4L2 driver entry Date: Wed, 11 Aug 2021 16:43:10 +0800 Message-Id: <2d1fc6f2be99f96e34a1074485e936f6bfb4b1f5.1628671163.git.ming.qian@nxp.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: X-ClientProxiedBy: SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) To AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv11149.swis.cn-sha01.nxp.com (119.31.174.70) by SG2P153CA0022.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c7::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.4 via Frontend Transport; Wed, 11 Aug 2021 08:44:50 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2923b9f4-ba05-424f-e08e-08d95ca44a62 X-MS-TrafficTypeDiagnostic: AM6PR04MB4888: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:243; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3iODo3fVZ4SBV2VF9S3GMIPMm0yJ2OBu3CblrTFyjnVOwOxU5OkIRcngB1MJe7j+NQ4CxRT3z4ktfhnZfDBcFLuNuxlhHX1mdqaHXVr6sTK2mfGHtjpwtllFnBggkNQCSuy4ktxa/zSYiJ8M03/0Jy++rYcAf49CCCb5w18acNTIbrIkhVm2alQCkblvDcv2ftGHZ+a0cy1a5iwHieS+MvGFFx60uF8iiWDa5uXhLmqsGGHPG/N9ZmERj9uZlbzJAFhKRvVBJkLncl0JRApzwdkOoQZneHOtM+FPYeHOUNMRg2oF9zLVEAwr59ArMrLNnsaJiVrjvFznkr9ZOIDvmwdFrmTh3PwbSMlE9HTmUdeCzwB26IMwB+tWFlpIYnbuOyTiVU+W7ppAokvNn1DZbIxbNcxCwijGAqMHLUAin4Ib1iEJB0buV37J9SwNrRQZkK8UJRm4Q0Y/obeXeN5k8kkmPdTKo/RMf4I67gWsNs1npfLvPKMAa7YAx9OFwBrrQBgUeZXdQa0Kj4zQ3oUWylXdKlnpDsc24sjG+pdimj8zaRI+VI6gZp6JIHq9Sjo8Rbf946R32dleanYP0VghuVruT64HIsqH1TXZsO7cV+GbKlUh9KhGT2rphopeI2BtoptOFycQADWx8yreNKrWAykDWln64Tqit2uy2xtHcHg29+AksRwWP+avqHRIBxCVit0dJh6ck3GptxK0IZjhRHWYg93Ak/Vb5e8+i2l0zQk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(376002)(366004)(396003)(136003)(39860400002)(7416002)(86362001)(8676002)(6486002)(36756003)(44832011)(2616005)(956004)(4326008)(7696005)(2906002)(38100700002)(4744005)(66476007)(478600001)(26005)(52116002)(66556008)(66946007)(38350700002)(186003)(8936002)(316002)(6666004)(5660300002)(32563001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tIQcPhc1f7rXKoOOlUtjZI8dD4AKgu8BkcnfS8+CLzsewSHOlczTHH6EEcDMrYhLNiNPK1qYaubpmix8ZD8enbWX7lQ2LhFSZO6wnbkSR/F1FH77Y2sqP5KCJ5Tmf1JmSOntucuIPloLAfpK5g49l8e03xNsKoShuHi9CXR9N5uqJALK2tShpGdmpRuIm0RWFQZrAf31vdTNvWP2/pax1f5euzknTHDlauyGWwoF6zoicsb0N21comFFjnRN2j+xoiREmws4i2ZedmqaWTRCH2SwGvNJI/Fx561/t3bOEOk7l7NQPscXyd6DLRpMGCgsYxq4WZjKA7mKXOjApB2pfk7qeDTLSAkpP8wOkfAuN2qg4bVk6eydpQgwiYoHqbwT46CV0yNQBhEG7IrMg5KaTIfSKrG9phN2+nVffnoUfK6ohLlMXUvzpmrLWkN6cP8CSZ8cXThEC7wHbpEn0bN+Am+TXaGvY6QptUs4SYAHuEXOQqUeIiLqPgj+go3kskU5owxbzHl4qF1HrvNDBTth6KCf2hCaW0dS5nlKebKOWALjVh2UV+HHkJj1+51AMA6uD+d2wLgunjAt5IbPolFN10OD+7coLOkHlOeut164NQWYqSIYM6R3EvAZeDvILpeHtKlPeWDdMeKpZkjgqQW3IW5kq0+AZkhYfhrRoFy6j5qcms4hTusSsiiJ2QkArHA8aUUjWkaR4WHroROV7Up+nqc6JbvZuk+V0pAaHqNS9ucL5rl5RRx5NRZBa1smImImPqxpjfimvHi8LxGQYxu6V/ORANjTffG0POCOWEQSOXZnP6sBCqhE8jczJ/AHQ3ndTEvnECopxHf09PEhpIAejpceWXw0ngunrvDAQF8FJI9rV/R8FBobb+go8jvfF26BsPYCxAm0nDfVPVAlPddDZYf9KuPLeFj8Mg3oEHZq8gDiSsoWvhD17JC1wwdV6O6Indhu2M29B/JlrCNlpYyK19ck3cqSstDACkw9hKoBUUxWS7bX7KC9mPf4cK06YPOP+xeOSJ1/oKEVJFXzydSshy4bPsVguqzVOooaOHRiuQRYDspWVvH8SXubaE6h7m7pGDTyKAm0HeohPCqe6JIGbAsu+13W0kjgTTfwp1uhdzVV50s8vrJ2fGInj26O5xOwPV7QxiAonrfQH3kbf2SO6ChSfawn6gsu0r5IyXAGYIpSjOqT+Ej8Xn5Id6JtDmwM9ZV4Kh2JHWUhbA2xg+iORF5aReun7VPz3DktKVKK2BvI0swcFmPN+qgr6wIvH6XCFVz/wVtSD28WGlvHkGqBZuDq4p5H/snJlMQ/RI4/diqYmxXTqlJKk0GKA2eVNLgL X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2923b9f4-ba05-424f-e08e-08d95ca44a62 X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 08:44:54.5359 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Lqiv1VNUDM0ZmFOfkpkFkLWcguXmuR4/9gVO7Rotxf07gk14XlFL3tJoHEi+NAgQUlb+XY+hTDCr/D3oq3M4dw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB4888 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Add NXP IMX8Q VPU CODEC v4l2 driver entry Signed-off-by: Ming Qian Signed-off-by: Shijie Qin Signed-off-by: Zhou Peng --- MAINTAINERS | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 524eabe50d79..77d30a55983a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13416,6 +13416,16 @@ S: Maintained F: Documentation/devicetree/bindings/media/imx8-jpeg.yaml F: drivers/media/platform/imx-jpeg +NXP i.MX 8QXP/8QM VPU CODEC V4L2 DRIVER +M: Ming Qian +M: Shijie Qin +M: Zhou Peng +L: linux-media@vger.kernel.org +S: Maintained +F: Documentation/devicetree/bindings/media/nxp,imx8q-vpu.yaml +F: drivers/media/platform/imx/vpu-8q/ +F: include/uapi/linux/imx_vpu.h + NZXT-KRAKEN2 HARDWARE MONITORING DRIVER M: Jonas Malaco L: linux-hwmon@vger.kernel.org