From patchwork Thu Dec 9 14:55:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522496 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1149984imb; Thu, 9 Dec 2021 06:58:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJzG7JW0WWrNrjbUw3JrUx5D6aFmsB7sezpKdKf1CLPZ7M/8zRbHQIrf3u6AY9XmjgCf4xZt X-Received: by 2002:a25:7489:: with SMTP id p131mr6900224ybc.339.1639061892103; Thu, 09 Dec 2021 06:58:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639061892; cv=none; d=google.com; s=arc-20160816; b=hgeKFb61hfYa1Hv/UdQAZVjx16GlXFlO2by2YKAKJwRXQawgZHApE6kE3T7BGOWQF0 QM7vK8Toti005clsszLxWDRDG1KtwfSBRQRdhHtKRDL1691EqbAvJ6AAfRMf3trmPZFU glskpspP94ScUFxRQhDXhun41fD6y5n8yQ32ZOjTllkjB9BPIWZNR9KIt//SP0wC3kab ITQAN1wV4P6uD78zPGeeyfqcDkqKYYdqj1UFIgQlgDTzGdW5/PQd7+cB/j6l1U/fIGbE EWmCtG3YZonkkXJCGlXJUibyADKdBiUgVUgReNd3cTN68l8pmLdgckc+D4AznsyO4k1n scoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=1QYgAjp609xS1sl67fG+HMcw0ipi2+jgawDo+pv6YsU=; b=dkZyv4WH6JwR9G50aZrOL8K8hF6AiVf5zLcUuo7jPjb2nbuIeYUiiYeACFNRYJhhJ/ qk4WtVqzqTAVuwzX1GZibUoMqu428YWqSmjG446qSR9K3dB0o65hMDCy2N5tOkZgWoYC c3WKmMUNJFQ8F2TtSnz8AUpSXmaV8qaRegjmqzR5yteHdwDmHi2dKUVjTXLb2TJSyPUz OWW59o6pOyIP8xZIhZRsM54lLtLPuf+xXPkaX+S5W3JCNvsYvGHyjwNi2RmHKRt37I9G IBkjzUUEXOeULYcWvbkZPwcmSMHHXIrnm0f7omPB8mHJVxQmIS79AE5gCy7WQmL07YLU I7LQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=AqIaKPAs; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 184si8884213ybq.116.2021.12.09.06.58.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 06:58:12 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=AqIaKPAs; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43428 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvKs7-0002e1-IX for patch@linaro.org; Thu, 09 Dec 2021 09:58:11 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59226) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqg-0002Ya-1v for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:42 -0500 Received: from [2a00:1450:4864:20::42a] (port=37614 helo=mail-wr1-x42a.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqd-0001XX-2i for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:41 -0500 Received: by mail-wr1-x42a.google.com with SMTP id d9so10162945wrw.4 for ; Thu, 09 Dec 2021 06:56:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1QYgAjp609xS1sl67fG+HMcw0ipi2+jgawDo+pv6YsU=; b=AqIaKPAslWw78cQsF3zgvKdCrZ68QcTqvHm/rvit/XGRO0z/U7DirTRUrgvy1zVHSt Mn6Q7BYlr9rWxdtbfTybBHqQfLsmsVUDJC3XgOCI0jujHzJlgDKW+3CX9o00rr5sBnbn v2Ke0QD3BRzm4OKJeVNKFLWZUT8jWQCt9YTFokTv947sQsZDQ8EujSwMpa0ox36joYEA tVDpyn4TEDvh3B5jcRcby7nne1AkRLzV7GFjhsGvtVNIIeu4dtmOMsdUV1shS710cdtZ Ja4Ae6y2iIqNC2XLyEfDgD8IV8QIiGBihvJgsDLcf1ROH3/GWu4WtV/QUVCSF+3YncMm KB4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1QYgAjp609xS1sl67fG+HMcw0ipi2+jgawDo+pv6YsU=; b=F/wp6blwWwmXjry3k4LGiVxYRcEB59erx69KhLhspnOXulfqIdn7gw6ErDMwT0I2e/ J+/BzJ28KDUICDZ1I6bnXBiSWs/OHaOcNfKz0QYQaHKAs7bWeYlNfSdSgptHQjWaZjrw mliKEidHbyJDwVmGZOWYh8bfNd2UnowV3CXaxB4u9NCVg6cgKxhTttxvzT0SwJGUSH5s 3HY5WMLAoMRLiYHTUQX4WFlEKGqsoeAUGPurTk0k6jHAM0CS2iA/AFvZ9DPxEK/208v7 D778t4pwVPxT5fWHrXbjF0g/VKH3OFhMiDSMyWaEuyzcDtoeZFCd1OE2Uq1KZBXuVD0G 8OdQ== X-Gm-Message-State: AOAM532vl3dCIpNplCWz8brOcK1UmvkiK7foWy4F5OeoDjb4iilTObJ6 FEhHN4kwgWBrdCWV1d+kPdGqcw== X-Received: by 2002:a5d:58ed:: with SMTP id f13mr6976991wrd.373.1639061797086; Thu, 09 Dec 2021 06:56:37 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:36 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 1/8] vhost-user-video: Add a README.md with cheat sheet of commands Date: Thu, 9 Dec 2021 14:55:54 +0000 Message-Id: <20211209145601.331477-2-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::42a (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::42a; envelope-from=peter.griffin@linaro.org; helo=mail-wr1-x42a.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Griffin --- tools/vhost-user-video/README.md | 98 ++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 tools/vhost-user-video/README.md diff --git a/tools/vhost-user-video/README.md b/tools/vhost-user-video/README.md new file mode 100644 index 0000000000..c55e0a7b68 --- /dev/null +++ b/tools/vhost-user-video/README.md @@ -0,0 +1,98 @@ +# Overview vhost-user-video + +This vmm translates from virtio-video v3 protocol and writes +to a v4l2 mem2mem stateful decoder/encoder device [1]. v3 was +chosen as that is what the virtio-video Linux frontend driver +currently implements. + +The primary goal so far is to enable development of virtio-video +frontend driver using purely open source software. Using vicodec +v4l2 stateful decoder on the host for testing then allows a pure +virtual environment for development and testing. + +Currently the vmm only supports v4l2 stateful devices, and the +intention is it will be used with Arm SoCs that implement stateful +decode/encode devices such as Qcom Venus, RPi, MediaTek etc. + +A Qemu + vicodec setup for virtio-video should also allow for +CI systems like kernelci, lkft to test the virtio-video interface +easily. + +Currently support for VAAPI or decoding via libavcodec or similar +libraries is not implemented, but this could be added in the future. + +Some example commands are provided below on how to run the daemon +and achieve a video decode using vicodec and a link to some test +content. + +[1] https://www.kernel.org/doc/html/latest/userspace-api/media/ + v4l/dev-decoder.html + +[2] https://lwn.net/Articles/760650/ + +# Guest Linux kernel modules +CONFIG_MEDIA_SUPPORT=y +CONFIG_MEDIA_TEST_SUPPORT=y +CONFIG_V4L_TEST_DRIVERS=y +CONFIG_VIRTIO_VIDEO=y +CONFIG_GDB_SCRIPTS=y +CONFIG_DRM_VIRTIO_GPU=y + +# Host kernel modules +CONFIG_MEDIA_SUPPORT=y +CONFIG_MEDIA_TEST_SUPPORT=y +CONFIG_V4L_TEST_DRIVERS=y +CONFIG_VIDEO_VICODEC=y + +# Run vhost-user-video daemon with vicodec +# (video3 typically is the stateful video) +vhost-user-video --socket-path=/tmp/video.sock --v4l2-device=/dev/video3 + +# Qemu command for virtio-video device + +-device vhost-user-video-pci,chardev=video,id=video +-chardev socket,path=/tmp//video.sock,id=video + +# Example v4l2-ctl decode command +wget https://people.linaro.org/~peter.griffin/jelly_640_480-420P.fwht + +v4l2-ctl -d0 -x width=640,height=480 -v width=640,height=480,pixelformat=YU12 +--stream-mmap --stream-out-mmap --stream-from jelly_640_480-420P.fwht +--stream-to out-jelly-640-480.YU12 + +# Play the raw decoded video with ffplay or mplayer +ffplay -loglevel warning -v info -f rawvideo -pixel_format yuv420p + -video_size "640x480" ./out-jelly-640-480.YU12 + +mplayer -demuxer rawvideo -rawvideo + format=i420:w=640:h=480:fps=25 out-jelly-640-480.YU12 + +# Enable v4l2 debug in virtio-video frontend driver +echo 0x1f > /sys/class/video4linux/video0/dev_debug + +# Enable v4l2 debug in vicodec backend driver +echo 0x1f > /sys/class/video4linux/video3/dev_debug + +# optee-build system qemu virtio-video command +make QEMU_VIRTFS_ENABLE=y QEMU_USERNET_ENABLE=y CFG_TA_ASLR=n + QEMU_VHOSTUSER_MEM=y QEMU_VIRTVIDEO_ENABLE=y SSH_PORT_FW=y run-only + +Current status +* Tested with v4l2-ctl from v4l2-utils and vicodec stateful decoder driver +* v4l2-compliance - reports +Total: 43, Succeeded: 37, Failed: 6, Warnings: 0 + +Known Issues +* 6 v4l2-compliance failures remaining +* v4l2-ctl 0fps misleading output +* v4l2-ctl sometimes reports - 0 != +* Encoder not tested yet + +TODOs +* Test with a "real" stateful decoder & codec + (e.g. Qcom Venus or RPi). +* Test more v4l2 userspaces in the guest + +Future potential features +* Emulation using libavcodec or similar library +* Support for VAAPI, OpenMax or v4l2 stateless devices From patchwork Thu Dec 9 14:55:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522497 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1150106imb; Thu, 9 Dec 2021 06:58:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJzJoX1F1jhpw2MyqpO5n5Ifoy8099zQ0MaWGcgGEAMsplb7yRUzruKQJbVhLETkXg1bevFR X-Received: by 2002:a05:622a:48f:: with SMTP id p15mr18055469qtx.42.1639061898994; Thu, 09 Dec 2021 06:58:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639061898; cv=none; d=google.com; s=arc-20160816; b=zLB5H3ZSe/zHALtQ0zbXCnDxMtcC+0kpahCiy6MrnWI7UGl735JKb3Rllv4HgDChAH T+4udrh+90be5o2qun6ej9GtvCyRhT7Qxf42YRYn8wfgis2Ibm2qrA4iCnENVn9MN7/q IjdP1f0f3zA72iH1d+QrkN28JC5Ma1qBpYm3k2/Rp8R3HyMNZ//0wxljfFNbn7kQRQa5 e/BROj19eKThhhBheVlpJRizel2xhmK5xDbvOxfF3GUBii66RGWe52goKVlJmUdy+mGn g5NgAYDiRAm8bpH6ltNppV2if7PLgAVjQXl6I3u/WMhvJ1YWimsIlSKKwNBMd2txq4eO tWuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=g4gFnI92APB9JNXKvvn3fcRzbuIPz7UahWTPyQnD0ok=; b=m6WqnSg6tu3UEsu+CPeL529lh6iBNHo5Hlue2Al+WLzycQ3G+6UU5YeukJa91TY9wP gTwK5aywcRJfIq6Cc7ifRZB7b4y0eh/bm2rwDLMEYS270gNIpT1tB3tl0rgJdQdJ4mJN izHGmc/XsptmoM5o9FNJzkbSQhH5K7BYzHXsbS1jTtrfCILNU7TRKPvFfPYnddC1o0aF PYOxNgaLxO3ue0oDBjPFkrr7fHNCLqxyj4GYKewyf9t5fBIsnd137OCB0BSXYet8EkQs 2MiRACSANr2Q0vXpH7SilaTZZnuddS5HSgvTr/IOz4Y5CL95lR75WMGKI5t9n+kfbGX3 gYtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=diloo+98; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p14si9653911vsl.591.2021.12.09.06.58.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 06:58:18 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=diloo+98; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43368 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvKsD-0002b8-3m for patch@linaro.org; Thu, 09 Dec 2021 09:58:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqe-0002XU-T6 for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:41 -0500 Received: from [2a00:1450:4864:20::42c] (port=37616 helo=mail-wr1-x42c.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqd-0001Xb-2x for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:40 -0500 Received: by mail-wr1-x42c.google.com with SMTP id d9so10163011wrw.4 for ; Thu, 09 Dec 2021 06:56:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g4gFnI92APB9JNXKvvn3fcRzbuIPz7UahWTPyQnD0ok=; b=diloo+98qc02fYIj5z1V9B75In1JJ6oB0Ig1lhV/LhohW8lnREB3uOkk7iodJK23zS 1Iy7hmsZ8OqUFlZ8yj/T6tAyK29pt63f4RTNS9vNVU5Dx9KfU29JcguHVr5Whi+r5Uot 1RXwCeVZD3BTDucQfmWrlZYqTi0S7VoimOXokoxDCmzNJab9D3OEWaWu5+CMgEvgYJ1c 4j34X+VZ+7DTV5CPzaeHKhPl4X+WzPfir529dpwx8o+jTkWKmjxcO6ixBFBaaXnVFCLi jno6R9ZEatUm9qdpLXcqKGU+1c88Y6ehSGLkBuRJnfCYq5ldY4x2bniw7E21j6HmaIsu i1ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g4gFnI92APB9JNXKvvn3fcRzbuIPz7UahWTPyQnD0ok=; b=pgu2TH5DaXG/dG3ghcWp0s3DMSeW/xOvEUAokUabr22OUVRpZzrdxkF4B/8qmjM16p pfmtKYd2k9Sb2xaHxFfZTwgSyPtjLdYdbFYQIvABNwaHP3F9K61H/WgNKw1BtnAg3Vtk 4Dh3f+Atb1KmqLT3Gdx/B+La0fy6lXdjRe7IirXmn3bwEvv/QzEmeyoyjkO6SsOR6gkc pPCsO/Q+eYSEhHlptuthCNuvF0zElGqskeLKIFgw5E1kpWhd7OCuv/PhJDgrDGM8fDCa axo2t7FHQlrm1rBh4OnvMRrWMLjaApkukFegk/CdmCoqhL2G5axsb3M5evK7jKxyPYsV ikQw== X-Gm-Message-State: AOAM5308S7g4k6/Anv963UbW7F/gFKlJwC/iQa5jjuOcV65cqGn+5sdq c6801g4paJQkV6TWN0jbyIziDA== X-Received: by 2002:a5d:4411:: with SMTP id z17mr6946508wrq.59.1639061797834; Thu, 09 Dec 2021 06:56:37 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:37 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 2/8] MAINTAINERS: Add virtio-video section Date: Thu, 9 Dec 2021 14:55:55 +0000 Message-Id: <20211209145601.331477-3-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::42c (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=peter.griffin@linaro.org; helo=mail-wr1-x42c.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Add myself as maintainer of the virtio-video files added in this series. Signed-off-by: Peter Griffin Reviewed-by: Alex Bennée --- MAINTAINERS | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 7543eb4d59..43c53aded8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2012,6 +2012,14 @@ F: hw/virtio/vhost-user-rng-pci.c F: include/hw/virtio/vhost-user-rng.h F: tools/vhost-user-rng/* +virtio-video +M: Peter Griffin +S: Supported +F: hw/display/vhost-user-video.c +F: hw/display/vhost-user-video-pci.c +F: include/hw/virtio/vhost-user-video.h +F: tools/vhost-user-video/* + virtio-crypto M: Gonglei S: Supported From patchwork Thu Dec 9 14:55:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522504 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1166109imb; Thu, 9 Dec 2021 07:09:13 -0800 (PST) X-Google-Smtp-Source: ABdhPJxTGEVA6HJTP1O/sccZag531RlDiGV6/b3E2SnywX0TMD303eBClBEOCgZk5+SsonwzXeC8 X-Received: by 2002:a2e:a593:: with SMTP id m19mr7067438ljp.407.1639062553054; Thu, 09 Dec 2021 07:09:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639062552; cv=none; d=google.com; s=arc-20160816; b=dA8e/4XqQ8GV5jIziU9/BUbbSGMzqkEITyfwFSFIq3CoQHckcg/vFbiRxTgRZDN34m aiJK2pcvQVI6Ay3Y0AYL0QbDFJzUjig7X1x54XNzq9rMRxWYVsVA6BzLalXPul/yzBS/ +9yGGtNtiOTTNdJ0PhCoiHHf2J9sVHFjgV5NyeR3kh3tTHeXMO6nqNLaNiuwwogZlP05 nqlQ4jJFNA+cZnTxS8I6mtvbU0DUC7KBAVHuqLMwPzHOXPFeHWdrbFnfR1h0Iyn9dB7s UFRwVbSjEi0ncdNHLNyol1MjTSiTceokKlQ8QK00/i3D3fBTxvy9GfdHz0FfiWam/YA5 Z5TA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0yXXniY95IkJujK31ekJh+HWJyF51d5WiUb5hhe8MAk=; b=u/WT6pg99/WLMiD544kqp/QpAkTbbW7SJPQHnXk/3rfMxijxyGV1WRmMebOAr5v0RV /lm0UJzdGb4lKN2KuphjKD2KPm0SVGNApGZ7jCcPuYmsdVzzztox3kC/DP8TZYtuQPtQ fwxogklWZNonRSgRQyDJ9v3LbnqGFtu3j/E0keQj8qPOWth2Y6LHZIoHmYGsplI5Z27W eQC/Mw6FsgoxyA3nyO7UW9GH1DUgglttMB4+JB9zP9r8gWOCu0b31cPATGbhmDPwfXHA 5H3ia6wcvAwVBX2h6tAhJYDlt9IErtgDBn8Mk99A16TXWlepF9unzOWYTDPbFcsRQVT7 edRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=KC1+f1ET; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h25si59391uan.182.2021.12.09.07.09.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 07:09:12 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=KC1+f1ET; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40146 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvL2m-0003Rl-FF for patch@linaro.org; Thu, 09 Dec 2021 10:09:12 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59310) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqj-0002bY-1D for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:45 -0500 Received: from [2a00:1450:4864:20::42b] (port=45656 helo=mail-wr1-x42b.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqe-0001Xs-Gv for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:44 -0500 Received: by mail-wr1-x42b.google.com with SMTP id o13so10098877wrs.12 for ; Thu, 09 Dec 2021 06:56:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0yXXniY95IkJujK31ekJh+HWJyF51d5WiUb5hhe8MAk=; b=KC1+f1ET47+q9q95fZK/81a2EiGWvHbJx4dasVFpkd+mj+y1cMBR7pRprwYoU+x353 fLJcA0nU1FTLKtvXArsl0ohiiQrssqvTePNZX+eYiwonNCkzxxm+MfFA7XjyyNtv8ovI rHOFtrFVlfjLGjyj2h/i6igipsY6GK/LNde0MEgX76r1EWceNpRcZLwA0V+9vJ1xmA3I cM4MjSj/QPQI7xGeocxYP7xGmS9Az2ixa3iOtCpFaBWx/uUaKKFtv2RlPNNcFgIG8Erw 4RFqPfx0KNYTmtiHJuMsu2y9ZKrbAtqsqCp1R+fMwqzM7mV0I8UJBk+W7I9O0oKyJJ0A 2liA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0yXXniY95IkJujK31ekJh+HWJyF51d5WiUb5hhe8MAk=; b=42CWIlyVTqHR+lNdoG9dsfekxGQyxmfmIVrYZE46NH0F4CSXC0+KFdeWKiMvRd10Gu deXp/+64XlXHqZUKR50NcTy5oo8mPOCODLED9d9qIYp9H/qNTR4S/p5b/bU4Sn1rHdMF BR3cfpjMfI9MtmXpZNKy7SPWg+dRbRfM9bLyWjon4lUcwLvQIU6ww4/hto8+E/eZEIIU i6axBB7x2hW7fj1o6nsi7FDD3SgS+HUbm/46ouI57vP0rvwYFBlsK3ID1xwzqZKblFrH pUTMzGrm78BMsZPSz8zNSzolQGxoivwLttBDNaXhJyYQM533vNe13/LxoXgPoCfq+NWc obPQ== X-Gm-Message-State: AOAM532/bMCMJunVeg8i4+TWe0g8a4awC3ULsGfo3fJR1B7sNlztU6e+ LweavBL3j2U/HGwKB+3/YhaBdntS7aBAqQ== X-Received: by 2002:adf:e109:: with SMTP id t9mr6801799wrz.387.1639061798715; Thu, 09 Dec 2021 06:56:38 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:38 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 3/8] vhost-user-video: boiler plate code for vhost-user-video device Date: Thu, 9 Dec 2021 14:55:56 +0000 Message-Id: <20211209145601.331477-4-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::42b (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::42b; envelope-from=peter.griffin@linaro.org; helo=mail-wr1-x42b.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Griffin Reviewed-by: Alex Bennée --- hw/display/Kconfig | 5 + hw/display/meson.build | 3 + hw/display/vhost-user-video.c | 386 +++++++++++++++++++++++++++ include/hw/virtio/vhost-user-video.h | 41 +++ 4 files changed, 435 insertions(+) create mode 100644 hw/display/vhost-user-video.c create mode 100644 include/hw/virtio/vhost-user-video.h diff --git a/hw/display/Kconfig b/hw/display/Kconfig index a2306b67d8..186163b015 100644 --- a/hw/display/Kconfig +++ b/hw/display/Kconfig @@ -118,6 +118,11 @@ config VHOST_USER_VGA default y depends on VIRTIO_VGA && VHOST_USER_GPU +config VHOST_USER_VIDEO + bool + default y + depends on VIRTIO && VHOST_USER + config DPCD bool select AUX diff --git a/hw/display/meson.build b/hw/display/meson.build index 861c43ff98..48284528cf 100644 --- a/hw/display/meson.build +++ b/hw/display/meson.build @@ -37,6 +37,9 @@ softmmu_ss.add(when: 'CONFIG_MACFB', if_true: files('macfb.c')) softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c')) specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c')) +specific_ss.add(when: 'CONFIG_VHOST_USER_VIDEO', if_true: files('vhost-user-video.c')) +specific_ss.add(when: ['CONFIG_VHOST_USER_VIDEO', 'CONFIG_VIRTIO_PCI' ], + if_true: files('vhost-user-video-pci.c')) if config_all_devices.has_key('CONFIG_QXL') qxl_ss = ss.source_set() diff --git a/hw/display/vhost-user-video.c b/hw/display/vhost-user-video.c new file mode 100644 index 0000000000..506e350365 --- /dev/null +++ b/hw/display/vhost-user-video.c @@ -0,0 +1,386 @@ +/* + * Vhost-user VIDEO virtio device + * + * This is the boilerplate for instantiating a vhost-user device + * implementing a virtio-video device. + * + * The virtio video decoder and encoder devices are virtual devices that + * support encoding and decoding respectively. + * + * The actual back-end for this driver is the vhost-user-video daemon. + * The code here just connects up the device in QEMU and allows it to + * be instantiated. + * + * Copyright (c) 2021 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "hw/qdev-properties.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/virtio/vhost-user-video.h" +#include "qemu/error-report.h" + +/* currently there is no VIDEO enc/dec defined in Linux virtio_ids.h */ +#define VIRTIO_ID_VIDEO_ENC 30 +#define VIRTIO_ID_VIDEO_DEC 31 +#define MAX_CAPS_LEN 4096 + +static void vhost_user_video_get_config(VirtIODevice *vdev, uint8_t *config) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + struct virtio_video_config *vconfig = (struct virtio_video_config *)config; + int ret; + Error *local_err = NULL; + + memset(config, 0, sizeof(struct virtio_video_config)); + + ret = vhost_dev_get_config(&video->vhost_dev, config, + sizeof(struct virtio_video_config), &local_err); + if (ret) { + error_report("vhost-user-video: get device config space failed"); + + /*TODO vhost_dev_get_config() fails so for now lets just set it here */ + vconfig = (struct virtio_video_config *)config; + vconfig->version = 0; + vconfig->max_caps_length = MAX_CAPS_LEN; + vconfig->max_resp_length = MAX_CAPS_LEN; + return; + } +} + +static void vhost_user_video_start(VirtIODevice *vdev) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + int i; + + if (!k->set_guest_notifiers) { + error_report("binding does not support guest notifiers"); + return; + } + + ret = vhost_dev_enable_notifiers(&video->vhost_dev, vdev); + if (ret < 0) { + error_report("Error enabling host notifiers: %d", -ret); + return; + } + + ret = k->set_guest_notifiers(qbus->parent, video->vhost_dev.nvqs, true); + if (ret < 0) { + error_report("Error binding guest notifier: %d", -ret); + goto err_host_notifiers; + } + + video->vhost_dev.acked_features = vdev->guest_features; + + ret = vhost_dev_start(&video->vhost_dev, vdev); + if (ret < 0) { + error_report("Error starting vhost-user-video: %d", -ret); + goto err_guest_notifiers; + } + + /* + * guest_notifier_mask/pending not used yet, so just unmask + * everything here. virtio-pci will do the right thing by + * enabling/disabling irqfd. + */ + for (i = 0; i < video->vhost_dev.nvqs; i++) { + vhost_virtqueue_mask(&video->vhost_dev, vdev, i, false); + } + + return; + +err_guest_notifiers: + k->set_guest_notifiers(qbus->parent, video->vhost_dev.nvqs, false); +err_host_notifiers: + vhost_dev_disable_notifiers(&video->vhost_dev, vdev); +} + +static void vhost_user_video_stop(VirtIODevice *vdev) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + + if (!k->set_guest_notifiers) { + return; + } + + vhost_dev_stop(&video->vhost_dev, vdev); + + ret = k->set_guest_notifiers(qbus->parent, video->vhost_dev.nvqs, false); + if (ret < 0) { + error_report("vhost guest notifier cleanup failed: %d", ret); + return; + } + + vhost_dev_disable_notifiers(&video->vhost_dev, vdev); +} + +static void vhost_user_video_set_status(VirtIODevice *vdev, uint8_t status) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + bool should_start = status & VIRTIO_CONFIG_S_DRIVER_OK; + + if (!vdev->vm_running) { + should_start = false; + } + + if (video->vhost_dev.started == should_start) { + return; + } + + if (should_start) { + vhost_user_video_start(vdev); + } else { + vhost_user_video_stop(vdev); + } +} + +static uint64_t vhost_user_video_get_features(VirtIODevice *vdev, + uint64_t requested_features, + Error **errp) +{ + /* currently only support guest pages */ + virtio_add_feature(&requested_features, + VIRTIO_VIDEO_F_RESOURCE_GUEST_PAGES); + + return requested_features; +} + +static void vhost_user_video_handle_output(VirtIODevice *vdev, VirtQueue *vq) +{ + /* + * Not normally called; it's the daemon that handles the queue; + * however virtio's cleanup path can call this. + */ +} + +static void vhost_user_video_guest_notifier_mask(VirtIODevice *vdev, int idx, + bool mask) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + vhost_virtqueue_mask(&video->vhost_dev, vdev, idx, mask); +} + +static bool vhost_user_video_guest_notifier_pending(VirtIODevice *vdev, int idx) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + return vhost_virtqueue_pending(&video->vhost_dev, idx); +} + +/* + * Chardev connect/disconnect events + */ + +static int vhost_user_video_handle_config_change(struct vhost_dev *dev) +{ + int ret; + VHostUserVIDEO *video = VHOST_USER_VIDEO(dev->vdev); + Error *local_err = NULL; + + ret = vhost_dev_get_config(dev, (uint8_t *)&video->conf.config, + sizeof(struct virtio_video_config), &local_err); + if (ret < 0) { + error_report("get config space failed"); + return -1; + } + + return 0; +} + +const VhostDevConfigOps video_ops = { + .vhost_dev_config_notifier = vhost_user_video_handle_config_change, +}; + +static int vhost_user_video_connect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + if (video->connected) { + return 0; + } + video->connected = true; + + /* restore vhost state */ + if (virtio_device_started(vdev, vdev->status)) { + vhost_user_video_start(vdev); + } + + return 0; +} + +static void vhost_user_video_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + if (!video->connected) { + return; + } + video->connected = false; + + if (video->vhost_dev.started) { + vhost_user_video_stop(vdev); + } + + vhost_dev_cleanup(&video->vhost_dev); +} + +static void vhost_user_video_event(void *opaque, QEMUChrEvent event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_video_connect(dev) < 0) { + qemu_chr_fe_disconnect(&video->conf.chardev); + return; + } + break; + case CHR_EVENT_CLOSED: + vhost_user_video_disconnect(dev); + break; + case CHR_EVENT_BREAK: + case CHR_EVENT_MUX_IN: + case CHR_EVENT_MUX_OUT: + /* Ignore */ + break; + } +} + +static void do_vhost_user_cleanup(VirtIODevice *vdev, VHostUserVIDEO *video) +{ + vhost_user_cleanup(&video->vhost_user); + virtio_delete_queue(video->command_vq); + virtio_delete_queue(video->event_vq); + virtio_cleanup(vdev); + g_free(video->vhost_dev.vqs); + video->vhost_dev.vqs = NULL; +} + + +static void vhost_user_video_device_realize(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(dev); + int ret; + + if (!vhost_user_init(&video->vhost_user, &video->conf.chardev, errp)) { + return; + } + + /* TODO Implement VIDEO_ENC, currently only support VIDEO_DEC */ + virtio_init(vdev, "vhost-user-video", VIRTIO_ID_VIDEO_DEC, + sizeof(struct virtio_video_config)); + + /* one command queue and one event queue */ + video->vhost_dev.nvqs = 2; + video->vhost_dev.vqs = g_new0(struct vhost_virtqueue, + video->vhost_dev.nvqs); + + ret = vhost_dev_init(&video->vhost_dev, &video->vhost_user, + VHOST_BACKEND_TYPE_USER, 0, errp); + if (ret < 0) { + error_report("vhost_dev_init failed: %s", strerror(-ret)); + goto vhost_dev_init_failed; + } + + /* One command queue, for sending commands */ + video->command_vq = virtio_add_queue(vdev, 128, + vhost_user_video_handle_output); + + if (!video->command_vq) { + error_setg_errno(errp, -1, "virtio_add_queue() failed"); + goto cmd_q_fail; + } + + /* One event queue, for sending events */ + video->event_vq = virtio_add_queue(vdev, 128, + vhost_user_video_handle_output); + + if (!video->command_vq) { + error_setg_errno(errp, -1, "virtio_add_queue() failed"); + goto event_q_fail; + } + + /* + * At this point the next event we will get is a connection from + * the daemon on the control socket. + */ + + qemu_chr_fe_set_handlers(&video->conf.chardev, NULL, + NULL, vhost_user_video_event, + NULL, (void *)dev, NULL, true); + + return; + +event_q_fail: + virtio_delete_queue(video->event_vq); +cmd_q_fail: + vhost_user_cleanup(&video->vhost_user); +vhost_dev_init_failed: + virtio_cleanup(vdev); + +} + +static void vhost_user_video_device_unrealize(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(dev); + + /* This will stop vhost backend if appropriate. */ + vhost_user_video_set_status(vdev, 0); + + do_vhost_user_cleanup(vdev, video); +} + +static const VMStateDescription vhost_user_video_vmstate = { + .name = "vhost-user-video", + .unmigratable = 1, +}; + +static Property vhost_user_video_properties[] = { + DEFINE_PROP_CHR("chardev", VHostUserVIDEO, conf.chardev), + DEFINE_PROP_END_OF_LIST(), +}; + +static void vhost_user_video_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass); + + device_class_set_props(dc, vhost_user_video_properties); + dc->vmsd = &vhost_user_video_vmstate; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + vdc->realize = vhost_user_video_device_realize; + vdc->unrealize = vhost_user_video_device_unrealize; + vdc->get_features = vhost_user_video_get_features; + vdc->get_config = vhost_user_video_get_config; + vdc->set_status = vhost_user_video_set_status; + vdc->guest_notifier_mask = vhost_user_video_guest_notifier_mask; + vdc->guest_notifier_pending = vhost_user_video_guest_notifier_pending; +} + +static const TypeInfo vhost_user_video_info = { + .name = TYPE_VHOST_USER_VIDEO, + .parent = TYPE_VIRTIO_DEVICE, + .instance_size = sizeof(VHostUserVIDEO), + .class_init = vhost_user_video_class_init, +}; + +static void vhost_user_video_register_types(void) +{ + type_register_static(&vhost_user_video_info); +} + +type_init(vhost_user_video_register_types) diff --git a/include/hw/virtio/vhost-user-video.h b/include/hw/virtio/vhost-user-video.h new file mode 100644 index 0000000000..09a4374ca6 --- /dev/null +++ b/include/hw/virtio/vhost-user-video.h @@ -0,0 +1,41 @@ +/* + * vhost-user-video virtio device + * + * Copyright (c) 2021 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef _VHOST_USER_VIDEO_H_ +#define _VHOST_USER_VIDEO_H_ + +#include "standard-headers/linux/virtio_video.h" +#include "hw/virtio/virtio.h" +#include "hw/virtio/vhost.h" +#include "hw/virtio/vhost-user.h" +#include "chardev/char-fe.h" + +#define TYPE_VHOST_USER_VIDEO "vhost-user-video-device" +#define VHOST_USER_VIDEO(obj) \ + OBJECT_CHECK(VHostUserVIDEO, (obj), TYPE_VHOST_USER_VIDEO) + +typedef struct { + CharBackend chardev; + struct virtio_video_config config; +} VHostUserVIDEOConf; + +typedef struct { + /*< private >*/ + VirtIODevice parent; + VHostUserVIDEOConf conf; + struct vhost_virtqueue *vhost_vq; + struct vhost_dev vhost_dev; + VhostUserState vhost_user; + VirtQueue *command_vq; + VirtQueue *event_vq; + bool connected; + /*< public >*/ +} VHostUserVIDEO; + + +#endif /* _VHOST_USER_VIDEO_H_ */ From patchwork Thu Dec 9 14:55:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522500 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1158276imb; Thu, 9 Dec 2021 07:04:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJzjg+icRqeuZ1qRr2G9Il5q/nJ/TGKzGnEEqx9S4CszX5JyG1GgOPpem2q6ydVJGNLtwfgh X-Received: by 2002:a5d:4a85:: with SMTP id o5mr6983145wrq.109.1639062240958; Thu, 09 Dec 2021 07:04:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639062240; cv=none; d=google.com; s=arc-20160816; b=sqrBlaSlQ7dsMdP+tX8fw56WdTUOy/dcSH1vTflWtDdsWqbKvupKd8KIqZp1hK7b4+ l6qDGUHYeOQmEQ+rwo+W5A/e9+UIOex+M7y4JJ/6bVyEMF9RVBLMXDYc/mUmHT6LdNiV 1WihgzekBY5s4VHgWJko8NRXC4uFO4piSaXG0ZJhWSDGjI2g5LGSblfIrYuQH+BQUfN9 PKK7dAc6kbg/YgfdOJCxYTAISBPW+NZFfyUfOkb/Htijgvq+ci+BxS+JR+oaAaLqdYbC LfUCSru1NA5Zw6NNSLSpDaMI5W8twNbrDaspcynBaBD7QChv7NBE+Xd07MrUZohVEZEn 0b6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=irUYZBqjdMbkZI8UB9bGVH0245A61P32W1SaY6iDPao=; b=hOMpRYE3p4TrxRgPkeXHM01ODuRASN7JbPL58HWAEZxxg5xB5hHx0t8OfApUONq27t 6MHaH+BrLy0PgYqOZFZet7rxUKJpMMRQOzbk5cfR3iHTCDhuTKomrO2BSRq3UZjG49ar 9eu0ymOleL5110NbcfpZ9c3FAfWK052pJcxZjZiNt5Yc/hRMS52TGjmQ/ysnHDPETgFt byX1RNAs271b0S0FHR44SyMbDt1eT+BivwymWPZLYeXWlSPiKymwe8EHL19C7eME2vSp hnyrL6KXMQSAeiymdMBwxUJv6qpzdWGh+vZ27muhRzwC3cYh7d2aCnYlcLZNX7KA/3wA JPng== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=OT8H9g6q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l18si10519706wmq.76.2021.12.09.07.04.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 07:04:00 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=OT8H9g6q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvKxj-0003RM-Qk for patch@linaro.org; Thu, 09 Dec 2021 10:03:59 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqi-0002b7-Hv for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:45 -0500 Received: from [2a00:1450:4864:20::334] (port=54105 helo=mail-wm1-x334.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqe-0001ZB-Sz for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:43 -0500 Received: by mail-wm1-x334.google.com with SMTP id y196so4409696wmc.3 for ; Thu, 09 Dec 2021 06:56:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=irUYZBqjdMbkZI8UB9bGVH0245A61P32W1SaY6iDPao=; b=OT8H9g6qLo1xH0CT23Zqeqzkg4OdPgERVYsXmy2grMQvXchNg9QaP13l6ozMaa4l7X 1ybsRtPDMKulQmSh7tF7d2vhQGc9DOO3UQWQwEDNjLAfnjLxEqfAA2+KClMvSANMY8Pd icyhRzgJRX4yWuQi+Qvna4dVfi8mk6B2Pu3IxUf4yOLnYwUDs+rLXHSBc7bYk62OpBf3 S+WCWCa1n0BWf/UfUES/fqtbL/oWFTLNF7ASaNlkhcj3cX8UJlXtihB4u1iJE2EZuONw w6Uv1RYzb1mhdBN8rpuYmSa54tpeeSt7reVsyZP6eA7ITIIv6diaUyF2PPDsvWVGCmaf zksA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=irUYZBqjdMbkZI8UB9bGVH0245A61P32W1SaY6iDPao=; b=vlWVMQxnrpp0ilBX/XGbAclw+OqAxnz1Qtc7oeQC4Em0J/BZRtDh0xfs2TWfWIxOWz u8Y1pBg9yfdR8WWxidJwDnhxxrpJTsGEfIUETR39POu8y93/JjRoz64NuqhItc7JJb/j 3Xh2THQFTGAB7C3AkmWWveigXnkxtmW746HB2V//9sL89DnVwG3yQUIzqeLSAwzIce7i XsJB26oKzXz6lNBzgAGtHrQ7lI6pQ0eYGSGOL8ImTCBopC7ghfI8wfkQThh3XS07fRUg 1DRx35vy3o+BkRMwwX1gLYi/9b/vYTJVaPM76Hgh0fYGpGwKkTI3A28hTE7gyHCyL3Ky uXRA== X-Gm-Message-State: AOAM532MyEwG2a7FH9KelKrLOthIeTvH0K43Rq2ckiZCQk/kysZ4p3Yh w7kZUScLtB3722AuocHdkUdn2w== X-Received: by 2002:a05:600c:1ca4:: with SMTP id k36mr7839448wms.169.1639061799549; Thu, 09 Dec 2021 06:56:39 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:39 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 4/8] vhost-user-video: add meson subdir build logic Date: Thu, 9 Dec 2021 14:55:57 +0000 Message-Id: <20211209145601.331477-5-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::334 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::334; envelope-from=peter.griffin@linaro.org; helo=mail-wm1-x334.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Griffin --- tools/meson.build | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tools/meson.build b/tools/meson.build index 3e5a0abfa2..3314b5efc5 100644 --- a/tools/meson.build +++ b/tools/meson.build @@ -24,3 +24,12 @@ endif if have_virtiofsd subdir('virtiofsd') endif + +have_virtiovideo = (have_system and + have_tools and + 'CONFIG_LINUX' in config_host) + +if have_virtiovideo + subdir('vhost-user-video') +endif + From patchwork Thu Dec 9 14:55:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522499 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1153150imb; Thu, 9 Dec 2021 07:00:33 -0800 (PST) X-Google-Smtp-Source: ABdhPJyAOJWHExkgVGQj6T12aH2f/JKu3INQDrprN7bmRb0AyYRbkCkTmzEqYzkQCTY9BWwQ/kXY X-Received: by 2002:a1f:20c2:: with SMTP id g185mr8850134vkg.25.1639062030402; Thu, 09 Dec 2021 07:00:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639062030; cv=none; d=google.com; s=arc-20160816; b=YjfmNYsbTe/c9LPv6ao09ZSN5myimZwCwTWkOHeGMZcTmVUr+j6+1+NftAcfeoKR58 8e+VpZtwP4M5t/9E30aqBBUtjYN11mjRa1cIRUbf1b1qTvMkKlhxOg03TZKmqRK/l4Ap IKcnRSCeAy9/uuESFCKBirIH1E3rCsMvJ1sVQxgJnz7bAHGd/cayKJdVJghAdtzW7Pv8 /Hf4OkzOhYWDIdiZMwKvhX99v/MU5xKZbusFzjcmWT1yoYcanlikhA3kmBCeN9KcfB9v ea2sGk7RhrmqmPcjbCLde6a6Suio67vTXAep5rdE7RsZpxn41CxnekDXX5j/A/ABU/d3 nK+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=AEQ/hf2H6Lmolyi6kLAaOjgkgfooSi3nVD5VGm8eWOQ=; b=ENOdUsHWPcuepy1TgoBjy1C3xVBX5JUSCGgeIyGUZWFdD4gHWZzrpXquTTlb+ia0d1 uakGEgAhguyZFtOCUL/TAppxZUXfxuJxpvGocXy1I/4CnoxJu5egrEHYkrwkbC1m2Ehg R767pMDOe4xTrRDkpj6ftxm6E9tyrDY/4E49EW7yXpqC5fn01i/2LuVOvpTFpXfzh4c8 zbmT0hWgE9OnqQf6Rcv5G4llPuQ6uoRbWJo7N9pHghsBJk6Y9RgjQt1/kCoiN5mLIyPX sZiBi8jdJyVkxN/daxz7vvaueUEruDcjgZUUy15F/MKpkT99Iwfu+WkI0TmdhIndDXoW tGLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=JkWEuzF4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s6si9490916vkn.71.2021.12.09.07.00.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 07:00:30 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=JkWEuzF4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51692 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvKuL-0008LN-PK for patch@linaro.org; Thu, 09 Dec 2021 10:00:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59316) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqj-0002bc-Dg for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:45 -0500 Received: from [2a00:1450:4864:20::32b] (port=35653 helo=mail-wm1-x32b.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqg-0001Za-5t for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:45 -0500 Received: by mail-wm1-x32b.google.com with SMTP id 77-20020a1c0450000000b0033123de3425so6801473wme.0 for ; Thu, 09 Dec 2021 06:56:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AEQ/hf2H6Lmolyi6kLAaOjgkgfooSi3nVD5VGm8eWOQ=; b=JkWEuzF4a2Nwmp5cenhM+MziWwgnGiqSPpDsHsUjov2zj1c7SpxRZwieieoZoW6F4z /1AYMTvxOd1+KbkHCNfug0w/lhFSqRhvWUy2UzQAFRdFZ8Rgy9ThXHsmw6jro8kKf2g7 /+ZxwcQnhY85fBrsVbRM8pEbr1+Lb9Li5yzR/DIW1ncgo9OdfakbBMnxBFdOaKKKigvS 7bJ6uAYDAKTlg+bqTTCDXIZN1odxdsni+azxSM0WJTZ/5ogQWX7bfOWTtsl9OmzL1OM1 vZqxg4S/soET/bfkpo7Fp5V8hwuQIWrGmyRfZ+DpBmPmM8J5XYR9980mQTq9FoNgFRDD uUsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AEQ/hf2H6Lmolyi6kLAaOjgkgfooSi3nVD5VGm8eWOQ=; b=MsdVmW+X+U83jdN4a1si8sip+Q18k33aj7D1AmM4ADjsvRbuaGTyCovLTl3/DqYeLg 0Ko8AzdIsCAICFklFdtLViCnZoJ6urkfJoDG8w4AW5svtwBAtewatLapgqsKJNDxvcsi sGVgZDA4HVX+4wb04uZKWB1yswrqu2eLTpJDrUAPWI4RWS/7mTR10RTcXWYiVnqM2TN5 Bj6zWJecA5bG99drWP6SBzI7/fFKapIpaLPLheC3BFhx/MgzC9C1CmuEnf2nKUL00Wgt uFYoiXqvYXTIoFbqZZqcn5gCfEHpZeLuHt8E5ftab9+ZaKAvDXWBRTtXSsSrfQAu4ngz pneA== X-Gm-Message-State: AOAM532O+yBTyLmMzD81Zp8CWdUYNn5VkhwTOAMIg8UD32AEdnqVHKl7 g7IrR9zPQpO/ECBtM80e4uBhPw== X-Received: by 2002:a7b:c770:: with SMTP id x16mr8143322wmk.66.1639061800603; Thu, 09 Dec 2021 06:56:40 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:40 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 5/8] standard-headers: Add virtio_video.h Date: Thu, 9 Dec 2021 14:55:58 +0000 Message-Id: <20211209145601.331477-6-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::32b (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=peter.griffin@linaro.org; helo=mail-wm1-x32b.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Griffin --- include/standard-headers/linux/virtio_video.h | 483 ++++++++++++++++++ 1 file changed, 483 insertions(+) create mode 100644 include/standard-headers/linux/virtio_video.h diff --git a/include/standard-headers/linux/virtio_video.h b/include/standard-headers/linux/virtio_video.h new file mode 100644 index 0000000000..16b5f642a9 --- /dev/null +++ b/include/standard-headers/linux/virtio_video.h @@ -0,0 +1,483 @@ +/* SPDX-License-Identifier: BSD-3-Clause */ +/* + * Virtio Video Device + * + * This header is BSD licensed so anyone can use the definitions + * to implement compatible drivers/servers: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of IBM nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL IBM OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * Copyright (C) 2019 OpenSynergy GmbH. + */ + +#ifndef _UAPI_LINUX_VIRTIO_VIDEO_H +#define _UAPI_LINUX_VIRTIO_VIDEO_H + +#include +#include + +/* + * Feature bits + */ + +/* Guest pages can be used for video buffers. */ +#define VIRTIO_VIDEO_F_RESOURCE_GUEST_PAGES 0 +/* + * The host can process buffers even if they are non-contiguous memory such as + * scatter-gather lists. + */ +#define VIRTIO_VIDEO_F_RESOURCE_NON_CONTIG 1 +/* Objects exported by another virtio device can be used for video buffers */ +#define VIRTIO_VIDEO_F_RESOURCE_VIRTIO_OBJECT 2 + +/* + * Image formats + */ + +enum virtio_video_format { + /* Raw formats */ + VIRTIO_VIDEO_FORMAT_RAW_MIN = 1, + VIRTIO_VIDEO_FORMAT_ARGB8888 = VIRTIO_VIDEO_FORMAT_RAW_MIN, + VIRTIO_VIDEO_FORMAT_BGRA8888, + VIRTIO_VIDEO_FORMAT_NV12, /* 12 Y/CbCr 4:2:0 */ + VIRTIO_VIDEO_FORMAT_YUV420, /* 12 YUV 4:2:0 */ + VIRTIO_VIDEO_FORMAT_YVU420, /* 12 YVU 4:2:0 */ + VIRTIO_VIDEO_FORMAT_RAW_MAX = VIRTIO_VIDEO_FORMAT_YVU420, + + /* Coded formats */ + VIRTIO_VIDEO_FORMAT_CODED_MIN = 0x1000, + VIRTIO_VIDEO_FORMAT_MPEG2 = + VIRTIO_VIDEO_FORMAT_CODED_MIN, /* MPEG-2 Part 2 */ + VIRTIO_VIDEO_FORMAT_MPEG4, /* MPEG-4 Part 2 */ + VIRTIO_VIDEO_FORMAT_H264, /* H.264 */ + VIRTIO_VIDEO_FORMAT_HEVC, /* HEVC aka H.265*/ + VIRTIO_VIDEO_FORMAT_VP8, /* VP8 */ + VIRTIO_VIDEO_FORMAT_VP9, /* VP9 */ + VIRTIO_VIDEO_FORMAT_CODED_MAX = VIRTIO_VIDEO_FORMAT_VP9, +}; + +enum virtio_video_profile { + /* H.264 */ + VIRTIO_VIDEO_PROFILE_H264_MIN = 0x100, + VIRTIO_VIDEO_PROFILE_H264_BASELINE = VIRTIO_VIDEO_PROFILE_H264_MIN, + VIRTIO_VIDEO_PROFILE_H264_MAIN, + VIRTIO_VIDEO_PROFILE_H264_EXTENDED, + VIRTIO_VIDEO_PROFILE_H264_HIGH, + VIRTIO_VIDEO_PROFILE_H264_HIGH10PROFILE, + VIRTIO_VIDEO_PROFILE_H264_HIGH422PROFILE, + VIRTIO_VIDEO_PROFILE_H264_HIGH444PREDICTIVEPROFILE, + VIRTIO_VIDEO_PROFILE_H264_SCALABLEBASELINE, + VIRTIO_VIDEO_PROFILE_H264_SCALABLEHIGH, + VIRTIO_VIDEO_PROFILE_H264_STEREOHIGH, + VIRTIO_VIDEO_PROFILE_H264_MULTIVIEWHIGH, + VIRTIO_VIDEO_PROFILE_H264_MAX = VIRTIO_VIDEO_PROFILE_H264_MULTIVIEWHIGH, + + /* HEVC */ + VIRTIO_VIDEO_PROFILE_HEVC_MIN = 0x200, + VIRTIO_VIDEO_PROFILE_HEVC_MAIN = VIRTIO_VIDEO_PROFILE_HEVC_MIN, + VIRTIO_VIDEO_PROFILE_HEVC_MAIN10, + VIRTIO_VIDEO_PROFILE_HEVC_MAIN_STILL_PICTURE, + VIRTIO_VIDEO_PROFILE_HEVC_MAX = + VIRTIO_VIDEO_PROFILE_HEVC_MAIN_STILL_PICTURE, + + /* VP8 */ + VIRTIO_VIDEO_PROFILE_VP8_MIN = 0x300, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE0 = VIRTIO_VIDEO_PROFILE_VP8_MIN, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE1, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE2, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE3, + VIRTIO_VIDEO_PROFILE_VP8_MAX = VIRTIO_VIDEO_PROFILE_VP8_PROFILE3, + + /* VP9 */ + VIRTIO_VIDEO_PROFILE_VP9_MIN = 0x400, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE0 = VIRTIO_VIDEO_PROFILE_VP9_MIN, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE1, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE2, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE3, + VIRTIO_VIDEO_PROFILE_VP9_MAX = VIRTIO_VIDEO_PROFILE_VP9_PROFILE3, +}; + +enum virtio_video_level { + /* H.264 */ + VIRTIO_VIDEO_LEVEL_H264_MIN = 0x100, + VIRTIO_VIDEO_LEVEL_H264_1_0 = VIRTIO_VIDEO_LEVEL_H264_MIN, + VIRTIO_VIDEO_LEVEL_H264_1_1, + VIRTIO_VIDEO_LEVEL_H264_1_2, + VIRTIO_VIDEO_LEVEL_H264_1_3, + VIRTIO_VIDEO_LEVEL_H264_2_0, + VIRTIO_VIDEO_LEVEL_H264_2_1, + VIRTIO_VIDEO_LEVEL_H264_2_2, + VIRTIO_VIDEO_LEVEL_H264_3_0, + VIRTIO_VIDEO_LEVEL_H264_3_1, + VIRTIO_VIDEO_LEVEL_H264_3_2, + VIRTIO_VIDEO_LEVEL_H264_4_0, + VIRTIO_VIDEO_LEVEL_H264_4_1, + VIRTIO_VIDEO_LEVEL_H264_4_2, + VIRTIO_VIDEO_LEVEL_H264_5_0, + VIRTIO_VIDEO_LEVEL_H264_5_1, + VIRTIO_VIDEO_LEVEL_H264_MAX = VIRTIO_VIDEO_LEVEL_H264_5_1, +}; + +/* + * Config + */ + +struct virtio_video_config { + __le32 version; + __le32 max_caps_length; + __le32 max_resp_length; +}; + +/* + * Commands + */ + +enum virtio_video_cmd_type { + /* Command */ + VIRTIO_VIDEO_CMD_QUERY_CAPABILITY = 0x0100, + VIRTIO_VIDEO_CMD_STREAM_CREATE, + VIRTIO_VIDEO_CMD_STREAM_DESTROY, + VIRTIO_VIDEO_CMD_STREAM_DRAIN, + VIRTIO_VIDEO_CMD_RESOURCE_CREATE, + VIRTIO_VIDEO_CMD_RESOURCE_QUEUE, + VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL, + VIRTIO_VIDEO_CMD_QUEUE_CLEAR, + VIRTIO_VIDEO_CMD_GET_PARAMS, + VIRTIO_VIDEO_CMD_SET_PARAMS, + VIRTIO_VIDEO_CMD_QUERY_CONTROL, + VIRTIO_VIDEO_CMD_GET_CONTROL, + VIRTIO_VIDEO_CMD_SET_CONTROL, + + /* Response */ + VIRTIO_VIDEO_RESP_OK_NODATA = 0x0200, + VIRTIO_VIDEO_RESP_OK_QUERY_CAPABILITY, + VIRTIO_VIDEO_RESP_OK_RESOURCE_QUEUE, + VIRTIO_VIDEO_RESP_OK_GET_PARAMS, + VIRTIO_VIDEO_RESP_OK_QUERY_CONTROL, + VIRTIO_VIDEO_RESP_OK_GET_CONTROL, + + VIRTIO_VIDEO_RESP_ERR_INVALID_OPERATION = 0x0300, + VIRTIO_VIDEO_RESP_ERR_OUT_OF_MEMORY, + VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID, + VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID, + VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER, + VIRTIO_VIDEO_RESP_ERR_UNSUPPORTED_CONTROL, +}; + +struct virtio_video_cmd_hdr { + __le32 type; /* One of enum virtio_video_cmd_type */ + __le32 stream_id; +}; + +/* VIRTIO_VIDEO_CMD_QUERY_CAPABILITY */ +enum virtio_video_queue_type { + VIRTIO_VIDEO_QUEUE_TYPE_INPUT = 0x100, + VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT, +}; + +struct virtio_video_query_capability { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +enum virtio_video_planes_layout_flag { + VIRTIO_VIDEO_PLANES_LAYOUT_SINGLE_BUFFER = 1 << 0, + VIRTIO_VIDEO_PLANES_LAYOUT_PER_PLANE = 1 << 1, +}; + +struct virtio_video_format_range { + __le32 min; + __le32 max; + __le32 step; + __u8 padding[4]; +}; + +struct virtio_video_format_frame { + struct virtio_video_format_range width; + struct virtio_video_format_range height; + __le32 num_rates; + __u8 padding[4]; + /* Followed by struct virtio_video_format_range frame_rates[] */ +}; + +struct virtio_video_format_desc { + __le64 mask; + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* types */ + __le32 planes_layout; /* Bitmask with VIRTIO_VIDEO_PLANES_LAYOUT_* */ + __le32 plane_align; + __le32 num_frames; + /* Followed by struct virtio_video_format_frame frames[] */ +}; + +struct virtio_video_query_capability_resp { + struct virtio_video_cmd_hdr hdr; + __le32 num_descs; + __u8 padding[4]; + /* Followed by struct virtio_video_format_desc descs[] */ +}; + +/* VIRTIO_VIDEO_CMD_STREAM_CREATE */ +enum virtio_video_mem_type { + VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES, + VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT, +}; + +struct virtio_video_stream_create { + struct virtio_video_cmd_hdr hdr; + __le32 in_mem_type; /* One of VIRTIO_VIDEO_MEM_TYPE_* types */ + __le32 out_mem_type; /* One of VIRTIO_VIDEO_MEM_TYPE_* types */ + __le32 coded_format; /* One of VIRTIO_VIDEO_FORMAT_* types */ + __u8 padding[4]; + __u8 tag[64]; +}; + +/* VIRTIO_VIDEO_CMD_STREAM_DESTROY */ +struct virtio_video_stream_destroy { + struct virtio_video_cmd_hdr hdr; +}; + +/* VIRTIO_VIDEO_CMD_STREAM_DRAIN */ +struct virtio_video_stream_drain { + struct virtio_video_cmd_hdr hdr; +}; + +/* VIRTIO_VIDEO_CMD_RESOURCE_CREATE */ +struct virtio_video_mem_entry { + __le64 addr; + __le32 length; + __u8 padding[4]; +}; + +struct virtio_video_object_entry { + __u8 uuid[16]; +}; + +#define VIRTIO_VIDEO_MAX_PLANES 8 + +struct virtio_video_resource_create { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __le32 resource_id; + __le32 planes_layout; + __le32 num_planes; + __le32 plane_offsets[VIRTIO_VIDEO_MAX_PLANES]; + __le32 num_entries[VIRTIO_VIDEO_MAX_PLANES]; + /** + * Followed by either + * - struct virtio_video_mem_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES + * - struct virtio_video_object_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT + */ +}; + +/* VIRTIO_VIDEO_CMD_RESOURCE_QUEUE */ +struct virtio_video_resource_queue { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __le32 resource_id; + __le64 timestamp; + __le32 num_data_sizes; + __le32 data_sizes[VIRTIO_VIDEO_MAX_PLANES]; + __u8 padding[4]; +}; + +enum virtio_video_buffer_flag { + VIRTIO_VIDEO_BUFFER_FLAG_ERR = 0x0001, + VIRTIO_VIDEO_BUFFER_FLAG_EOS = 0x0002, + + /* Encoder only */ + VIRTIO_VIDEO_BUFFER_FLAG_IFRAME = 0x0004, + VIRTIO_VIDEO_BUFFER_FLAG_PFRAME = 0x0008, + VIRTIO_VIDEO_BUFFER_FLAG_BFRAME = 0x0010, +}; + +struct virtio_video_resource_queue_resp { + struct virtio_video_cmd_hdr hdr; + __le64 timestamp; + __le32 flags; /* One of VIRTIO_VIDEO_BUFFER_FLAG_* flags */ + __le32 size; /* Encoded size */ +}; + +/* VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL */ +struct virtio_video_resource_destroy_all { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +/* VIRTIO_VIDEO_CMD_QUEUE_CLEAR */ +struct virtio_video_queue_clear { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +/* VIRTIO_VIDEO_CMD_GET_PARAMS */ +struct virtio_video_plane_format { + __le32 plane_size; + __le32 stride; +}; + +struct virtio_video_crop { + __le32 left; + __le32 top; + __le32 width; + __le32 height; +}; + +struct virtio_video_params { + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* types */ + __le32 frame_width; + __le32 frame_height; + __le32 min_buffers; + __le32 max_buffers; + struct virtio_video_crop crop; + __le32 frame_rate; + __le32 num_planes; + struct virtio_video_plane_format plane_formats[VIRTIO_VIDEO_MAX_PLANES]; +}; + +struct virtio_video_get_params { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +struct virtio_video_get_params_resp { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_params params; +}; + +/* VIRTIO_VIDEO_CMD_SET_PARAMS */ +struct virtio_video_set_params { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_params params; +}; + +/* VIRTIO_VIDEO_CMD_QUERY_CONTROL */ +enum virtio_video_control_type { + VIRTIO_VIDEO_CONTROL_BITRATE = 1, + VIRTIO_VIDEO_CONTROL_PROFILE, + VIRTIO_VIDEO_CONTROL_LEVEL, + VIRTIO_VIDEO_CONTROL_FORCE_KEYFRAME, +}; + +struct virtio_video_query_control_profile { + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* */ + __u8 padding[4]; +}; + +struct virtio_video_query_control_level { + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* */ + __u8 padding[4]; +}; + +struct virtio_video_query_control { + struct virtio_video_cmd_hdr hdr; + __le32 control; /* One of VIRTIO_VIDEO_CONTROL_* types */ + __u8 padding[4]; + /* + * Followed by a value of struct virtio_video_query_control_* + * in accordance with the value of control. + */ +}; + +struct virtio_video_query_control_resp_profile { + __le32 num; + __u8 padding[4]; + /* Followed by an array le32 profiles[] */ +}; + +struct virtio_video_query_control_resp_level { + __le32 num; + __u8 padding[4]; + /* Followed by an array le32 level[] */ +}; + +struct virtio_video_query_control_resp { + struct virtio_video_cmd_hdr hdr; + /* Followed by one of struct virtio_video_query_control_resp_* */ +}; + +/* VIRTIO_VIDEO_CMD_GET_CONTROL */ +struct virtio_video_get_control { + struct virtio_video_cmd_hdr hdr; + __le32 control; /* One of VIRTIO_VIDEO_CONTROL_* types */ + __u8 padding[4]; +}; + +struct virtio_video_control_val_bitrate { + __le32 bitrate; + __u8 padding[4]; +}; + +struct virtio_video_control_val_profile { + __le32 profile; + __u8 padding[4]; +}; + +struct virtio_video_control_val_level { + __le32 level; + __u8 padding[4]; +}; + +struct virtio_video_get_control_resp { + struct virtio_video_cmd_hdr hdr; + /* Followed by one of struct virtio_video_control_val_* */ +}; + +/* VIRTIO_VIDEO_CMD_SET_CONTROL */ +struct virtio_video_set_control { + struct virtio_video_cmd_hdr hdr; + __le32 control; /* One of VIRTIO_VIDEO_CONTROL_* types */ + __u8 padding[4]; + /* Followed by one of struct virtio_video_control_val_* */ +}; + +struct virtio_video_set_control_resp { + struct virtio_video_cmd_hdr hdr; +}; + +/* + * Events + */ + +enum virtio_video_event_type { + /* For all devices */ + VIRTIO_VIDEO_EVENT_ERROR = 0x0100, + + /* For decoder only */ + VIRTIO_VIDEO_EVENT_DECODER_RESOLUTION_CHANGED = 0x0200, +}; + +struct virtio_video_event { + __le32 event_type; /* One of VIRTIO_VIDEO_EVENT_* types */ + __le32 stream_id; +}; + +#endif /* _UAPI_LINUX_VIRTIO_VIDEO_H */ From patchwork Thu Dec 9 14:55:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522503 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1162586imb; Thu, 9 Dec 2021 07:06:58 -0800 (PST) X-Google-Smtp-Source: ABdhPJyQQkbBcTK213CyzcgzxlAyCYsLNMte88Keqk9UKGdabiQmo4ClwPpCVXVMZ+f2KFtI/qse X-Received: by 2002:a05:6e02:20ea:: with SMTP id q10mr10200902ilv.210.1639062418186; Thu, 09 Dec 2021 07:06:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639062418; cv=none; d=google.com; s=arc-20160816; b=RTWxJGRjd21llcy1yucjESsWBxKASiKbeV1GC3dJhP2xquNq+VOS2pyXHGqYQetLC9 jCjO6NAX4SjLODxSQjljdagsMi8wzR5HxxreA6DlrOo1Fj68S6yh3qys/+gplX8di7vS Kmh7VA7azNEMoqcINIxhQYEY8oozbXIHk/Iv5tK7sIbH5f5As8vlJSW+lJQllkbL0SkN tlehZW7FZPzbzZQ/Ss91bg1rOdv8oG5pN+KOXNT8EPGjBa7sFQKUR+QYNhbqLItL0kOR SOKu/JeszaP4gxE9dPw1K8k22MxU3meJpykjE5WfcVeNnRS9g7HUF/c0YT2Tpb0nYRIc W3bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=GzyL0rwW98AtDdWXVvxfJFra64F6h5kdluyzTy63cnM=; b=x/0fD/yLdmZYXPnTKBdbSoWiihsjlv7NrjVBFAn+cQFHIc3aBHIrWE1W5OKQSBrvjS HjG/5B5VrJFOj5/7IFUwkuylKYS21bljk02GU6CjsDqVidyiwSZW8hVGFEZjtfYpgg/J 94yA+nYUi6qIyGVZcl3jO63d5P0/i/gQURk1Aa6Q83HKhMXGqknQdE22ImxNWcdsjGrQ kF+GgKFo6FgYagkEN6CgC2KxJws1mqtEhI9hlnQi3+Mt2YULE8R7O30ziiAxepKM3Pfy g0tB/9z5nlnmq5EeiQ7HutcVkjtieP038IM8W48k/9ew6iVFUjNmLkMR9d+DCIS8FDWG lTMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=oZDrJa9F; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h1si8949978vsl.704.2021.12.09.07.06.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 07:06:58 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=oZDrJa9F; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34844 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvL0b-00081W-IJ for patch@linaro.org; Thu, 09 Dec 2021 10:06:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59318) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqj-0002be-KW for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:45 -0500 Received: from [2a00:1450:4864:20::333] (port=40926 helo=mail-wm1-x333.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqg-0001Zl-Ok for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:45 -0500 Received: by mail-wm1-x333.google.com with SMTP id j140-20020a1c2392000000b003399ae48f58so6742467wmj.5 for ; Thu, 09 Dec 2021 06:56:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GzyL0rwW98AtDdWXVvxfJFra64F6h5kdluyzTy63cnM=; b=oZDrJa9FfyOHcAOZAaRYIrY9M3SNfGzDUVJXbkTKIVjtaYdT4j83kHej6fp97yinl5 qDZbohsYigwLLzazSs9qIIquQP6VOOAkIB4SXIkhrI69oGNVq/5vl/mxuSR1oG5vsQ5d Z87PiGxh1yVKcUCDn0tPMGk8uYWSWRNtyQlZGkUMMq2F8OyYBCytUG9kvvHLlp55CJSR kUP6FiV7UXVpMfhxyzcWJld9utxksQdKWfLDQTbkLL8LAcx3vHWAQ7K77v5eNC07P7SM SvMIs9tbm9b7D6VTbfwWST08bBltR6089Ov8RnKqFUSjIiJZAr/m0UWnolvyebvXVzxh ZCVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GzyL0rwW98AtDdWXVvxfJFra64F6h5kdluyzTy63cnM=; b=z4BFWAJaHH7+jcBW/Yyjw8BVI+9vg/kkgPkjTTEnONmQ4qUDg4g0XPVfVKjOzL3Jm4 H2Rex/jW6OrX2M+ipakJDYArgaA/cK4Qyfsm/MDekZaZkVNz3j7f4uAzVyzPHoqHLDOY dnrHRZ5Cj8bgYCltoImMG6LHIpOr3lk6K23wlmNFPoOB2l3uqmFJ6gSG9Z82pg9r7/gD Jt+5a5UmfX3+eqeKjhM86U58uqFh7/4mfaeDV1ZxUAytBNmJ4r/XJUSaXtaytOnJRLAP jQ8usYKwCqwEaoZ//iGTB/4usFxvUPxVIU5TA662YcIND8oNZ2SrRe1YKmMy8cYNggRn 6UWQ== X-Gm-Message-State: AOAM531LCvYZM6BymGb4c0/TOw3F0K6Zebbd/YOEKohEMfqY3fsCMYny Ib55FM3KuZgtjyBqbvMN2IM1AppttNmHmQ== X-Received: by 2002:a05:600c:1e87:: with SMTP id be7mr7618376wmb.182.1639061801441; Thu, 09 Dec 2021 06:56:41 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:41 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 6/8] virtio_video: Add Fast Walsh-Hadamard Transform format Date: Thu, 9 Dec 2021 14:55:59 +0000 Message-Id: <20211209145601.331477-7-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::333 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=peter.griffin@linaro.org; helo=mail-wm1-x333.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Linux vicodec (Virtual Codec) test driver in Linux implements FWHT. FWHT was designed to be fast and simple and to have characteristics of other video codecs and therefore face similar issues [1]. https://en.wikipedia.org/wiki/Fast_Walsh%E2%80%93Hadamard_transform Signed-off-by: Peter Griffin --- include/standard-headers/linux/virtio_video.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/standard-headers/linux/virtio_video.h b/include/standard-headers/linux/virtio_video.h index 16b5f642a9..3b517d50c4 100644 --- a/include/standard-headers/linux/virtio_video.h +++ b/include/standard-headers/linux/virtio_video.h @@ -75,6 +75,7 @@ enum virtio_video_format { VIRTIO_VIDEO_FORMAT_HEVC, /* HEVC aka H.265*/ VIRTIO_VIDEO_FORMAT_VP8, /* VP8 */ VIRTIO_VIDEO_FORMAT_VP9, /* VP9 */ + VIRTIO_VIDEO_FORMAT_FWHT, /* FWHT used by vicodec */ VIRTIO_VIDEO_FORMAT_CODED_MAX = VIRTIO_VIDEO_FORMAT_VP9, }; From patchwork Thu Dec 9 14:56:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522501 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1158883imb; Thu, 9 Dec 2021 07:04:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJxCZbtBJx2NCd4396I9kPz/HVs00B8W1BTqqpee5Cpl/jfg34/6UsffgBmzW/0P9mfR7yJP X-Received: by 2002:a7b:cbd8:: with SMTP id n24mr7588045wmi.150.1639062267060; Thu, 09 Dec 2021 07:04:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639062267; cv=none; d=google.com; s=arc-20160816; b=bZhE9KU18kCQBLe9ecyc3ct0BCqG6Vmma6NJHsSqD9YXIV6Pli0yU+eTacZM7zgTGt IQ0dWZ3g4gZmgPk7Tbgn4M/CL8qtplNpbqdRZrFdHrAQSpKqsz2R9jSCWyQ+5RbAcfMW rQO9pQn6PX9LXB6izBRDpyhJKzUtxS+Bhpi7DeiSHDRwotGeWzlClA3TQJ3FcHCQcmWm OEl1zBlehuL+H0twf8GMPg++4Z2F1AJz+ALrJoxy5Ivi/HWV3H2wZ/fKq15sTnclzraz JhnTLADlvAG8NTtPxt8JqjtreSM/8ZZXlew1SCLzSh3R6eAd/M4qRm+E+DBilLO7GAxR n5cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ucRtCT/8uwpnsgdYhV7nphwjB8b91qloRsPjp9e3CEs=; b=dsXKjKPNB1+MUFryaxCcy9Nj7D5I6pPtZPE0mC6+lkeDBHjk6gnTmv73yxLknvmDix dVylwvoIPjxTzlSJIkiuOrr30yasXAixaax4qdYlt+os3Bb30lGzFVro/zkFQVATbXsb bHi8o6Z/uak1Xqx60/ae13GGMtrpapVyt6TxkiDnnl4M2TqB6PnAmoLUGZEbvLC1H7q0 2XvjvSldFo3PVUNfq2ldZEYVKG2H8rWIxkLdL0MPLq0zuUqbc3hFc0OftlB2o304hiDD X56tVzBSCwq8LvpCexBhVpBl+TJ7Bvv/kwXEVaoBc7/72FOQvOuYS1tllkwf9Lc8RJ1x JC0g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=tPTwBOt3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k4si162988wrg.670.2021.12.09.07.04.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 07:04:27 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=tPTwBOt3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvKyA-00055F-6C for patch@linaro.org; Thu, 09 Dec 2021 10:04:26 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqk-0002eB-JR for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:46 -0500 Received: from [2a00:1450:4864:20::32d] (port=46617 helo=mail-wm1-x32d.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqi-0001Zx-Q7 for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:46 -0500 Received: by mail-wm1-x32d.google.com with SMTP id c6-20020a05600c0ac600b0033c3aedd30aso4287914wmr.5 for ; Thu, 09 Dec 2021 06:56:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ucRtCT/8uwpnsgdYhV7nphwjB8b91qloRsPjp9e3CEs=; b=tPTwBOt3NAUJGH+i3yviRxPIS2c+wBQKuKxfG1bARMoa4Q8CQJ/R0grYwMk841Gtda HuKx4Z4t/KrTeFB/Eb6Vd6RLrYwooKKOAiRsE7J/cLqaTWuqhYECBVuWTJ9kwoqmRdUm jkNbPfi4BBkm1uO8XACln1aQYIxpK99HphSQbhIwZ+1gobsNYHVb2QYLweDwEU6YAWp8 kErbFi5msYnw+gvfz0c3UB2+MV8NvPN+WkVBbiplXVOtlFBgcENpfkhCDmdZs3DIAzLo Iuuzx7FSJh9So1LJVU3FR4m7fQE9QFbq5ekTf0VhXiLNDuAjfOmMGEvF/BmvI4W2J0ac DcMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ucRtCT/8uwpnsgdYhV7nphwjB8b91qloRsPjp9e3CEs=; b=DOL9NZt8oZ0igbVsGB9W7BNjoZWi9ynoOl4oL1yKYAvxbk2W0e8hd8wknqK3iV5tso ZB10hJ/zJJOiZptNqKzcCXDz7x5dv/+FugL+9LmZd+nDZIn3P42Ha0ZhxSAvb0HJgFrC OfIUESq71U18ZOLPjHIBZP08wF9tiYn2f43i1BKTfjI8FvVj8V8Kr3O8UbCe2keoTDMS PRdXKWs0iX+jaA7PeKXFlMGodg5msebzF7rrJt72ki27XencmGNAhcWE+to/Vyzg0y/T XirbQ/0R7h8m2sR2yPxx+gtzEL8nSGpjre2dUJLuo7xogEhKCj6MiuPU1XzQcPS8sl+U 6RcA== X-Gm-Message-State: AOAM5301zG9ATHxosy3hfmUzcWIoJsiG1ciBwdmZZkooN6l7RZ6wvfF/ bHZ/Ik7NlaxCdr44LdWulK74Jg== X-Received: by 2002:a05:600c:4f48:: with SMTP id m8mr8050810wmq.50.1639061802337; Thu, 09 Dec 2021 06:56:42 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:41 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 7/8] hw/display: add vhost-user-video-pci Date: Thu, 9 Dec 2021 14:56:00 +0000 Message-Id: <20211209145601.331477-8-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::32d (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::32d; envelope-from=peter.griffin@linaro.org; helo=mail-wm1-x32d.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Add boiler plate code for vhost-user-video-pci. Example -device vhost-user-video-pci,chardev=video,id=video -chardev socket,path=video.sock,id=video Signed-off-by: Peter Griffin Reviewed-by: Alex Bennée --- hw/display/vhost-user-video-pci.c | 82 +++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 hw/display/vhost-user-video-pci.c diff --git a/hw/display/vhost-user-video-pci.c b/hw/display/vhost-user-video-pci.c new file mode 100644 index 0000000000..ceeaad2742 --- /dev/null +++ b/hw/display/vhost-user-video-pci.c @@ -0,0 +1,82 @@ +/* + * Vhost-user VIDEO virtio device PCI glue + * + * Copyright (c) 2021 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "hw/qdev-properties.h" +#include "hw/virtio/vhost-user-video.h" +#include "hw/virtio/virtio-pci.h" + +struct VHostUserVIDEOPCI { + VirtIOPCIProxy parent_obj; + VHostUserVIDEO vdev; +}; + +typedef struct VHostUserVIDEOPCI VHostUserVIDEOPCI; + +#define TYPE_VHOST_USER_VIDEO_PCI "vhost-user-video-pci-base" + +#define VHOST_USER_VIDEO_PCI(obj) \ + OBJECT_CHECK(VHostUserVIDEOPCI, (obj), TYPE_VHOST_USER_VIDEO_PCI) + +static Property vuvideo_pci_properties[] = { + DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags, + VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true), + DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, + DEV_NVECTORS_UNSPECIFIED), + DEFINE_PROP_END_OF_LIST(), +}; + +static void vuvideo_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) +{ + VHostUserVIDEOPCI *dev = VHOST_USER_VIDEO_PCI(vpci_dev); + DeviceState *vdev = DEVICE(&dev->vdev); + + if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { + vpci_dev->nvectors = 1; + } + + qdev_set_parent_bus(vdev, BUS(&vpci_dev->bus), errp); + object_property_set_bool(OBJECT(vdev), "realized", true, errp); +} + +static void vuvideo_pci_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + VirtioPCIClass *k = VIRTIO_PCI_CLASS(klass); + PCIDeviceClass *pcidev_k = PCI_DEVICE_CLASS(klass); + k->realize = vuvideo_pci_realize; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + device_class_set_props(dc, vuvideo_pci_properties); + pcidev_k->vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET; + pcidev_k->device_id = 0; /* Set by virtio-pci based on virtio id */ + pcidev_k->revision = 0x00; + pcidev_k->class_id = PCI_CLASS_STORAGE_OTHER; +} + +static void vuvideo_pci_instance_init(Object *obj) +{ + VHostUserVIDEOPCI *dev = VHOST_USER_VIDEO_PCI(obj); + + virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev), + TYPE_VHOST_USER_VIDEO); +} + +static const VirtioPCIDeviceTypeInfo vuvideo_pci_info = { + .base_name = TYPE_VHOST_USER_VIDEO_PCI, + .non_transitional_name = "vhost-user-video-pci", + .instance_size = sizeof(VHostUserVIDEOPCI), + .instance_init = vuvideo_pci_instance_init, + .class_init = vuvideo_pci_class_init, +}; + +static void vuvideo_pci_register(void) +{ + virtio_pci_types_register(&vuvideo_pci_info); +} + +type_init(vuvideo_pci_register); From patchwork Thu Dec 9 14:56:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Griffin X-Patchwork-Id: 522502 Delivered-To: patch@linaro.org Received: by 2002:a05:6e04:2287:0:0:0:0 with SMTP id bl7csp1159436imb; Thu, 9 Dec 2021 07:04:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJyIn9ph/tRvc6lR1Q6pR2bUE2m+LdYEK8D/zCB7tV+/TAJiHh4hewgTSvl9K8EOakw9nf9v X-Received: by 2002:a5d:6447:: with SMTP id d7mr6997347wrw.118.1639062290343; Thu, 09 Dec 2021 07:04:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639062290; cv=none; d=google.com; s=arc-20160816; b=JadC5YOHD3RB/YCisVxjpR7DUTE7sds0XGr8skjIyZZz70tVCPhkPlycq5ifyEyZWM xgv+uIulfvgUeCczxxRYe8rmVwlvMv18a5o/HUdB1YbNqGNQjERkNj9oUSZbG2DrMzYi AloH1miKa6qRZsMDFcKmX3LPNg1XFz6UzTDKHBHh6YY/kGfxn7azEzRPIxAduY19C2rU a8l8f4ht1qZYs1TTfr//9G5EgUI9zk1DUuhfmo8NZ+0BaqvPLGLoM95Tv/2ykLfai1Ye vMRkRq2jaai8BmV9PBC2hlxtmog5TGQFMCSZN653Hkesx/y816pnNenG+NYaGlnBdEw5 r9EQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=f8QmRx9+DNSp2O4mYDU/ok11SEV2qs8UF4yTcNX4blo=; b=ffF6u1cs6qvVB8ncMCXzoYHdw5jlbZ7unearNzpCdJqU1DfNKeLny/s5pUE7DaMc0n ynB57D4v7tm0hut2T+cow77jnU96cFp4flCIFbpXQbg32y19nvoUGOIIcrTbCer1tGDE 8Ig6FLyz/IgKolNjTI796ccxZ5Ie3htvmTeQ6yQ1kzr113VBIcWKU8c0EIwWUGy/ohxb HtZTYojGCTHTAe8H2hdK8AFhmz761n+cvHQiXt42UTiuXuZ+1HHEKSytaCv0GtLhdEAZ xr0uDcPvjW+SnBKwfgHRPtJhmJPA6+AqC0XYSZqf7M0E8O+nveSWwKD/PBdErxjG0so0 zsgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=E5OJOrZi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r7si167591wrr.655.2021.12.09.07.04.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Dec 2021 07:04:50 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=E5OJOrZi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58994 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvKyX-0005DI-2b for patch@linaro.org; Thu, 09 Dec 2021 10:04:49 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvKqo-0002r6-Su for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:50 -0500 Received: from [2a00:1450:4864:20::32b] (port=40919 helo=mail-wm1-x32b.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvKqk-0001aI-0b for qemu-devel@nongnu.org; Thu, 09 Dec 2021 09:56:50 -0500 Received: by mail-wm1-x32b.google.com with SMTP id j140-20020a1c2392000000b003399ae48f58so6742634wmj.5 for ; Thu, 09 Dec 2021 06:56:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f8QmRx9+DNSp2O4mYDU/ok11SEV2qs8UF4yTcNX4blo=; b=E5OJOrZitnuGnRdRRbJhXnX2GTDYIChslnxRaqybWBcFa2uI5qW93v8vvNVpkAZq8J Dn9NGR5qS+N8KVHHW9jW+f9Z3r4iEzGP7AGOCwdNcFv1Mpr3E31GW5pD307zhPPFjKmS 3FQ3uEKn0hJ36qmH9xef+1AKYESzNPTHCFlA+G1I0GdWCKy+5DXCHF7VnR+yxRwvhMdA 26fpSo/6VSmYuapHPM3KnUx0jCuIuc6maC5qt7HN5QrDVfpfWTFIpPvsYRLG3WToVfE8 fqbc1tzyrsBVj868BCZXYrkbmlwrGxpDqG2XlcP8q3pMhq+Wb0nojwyeqnKMJ6/5H+Bw oJhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f8QmRx9+DNSp2O4mYDU/ok11SEV2qs8UF4yTcNX4blo=; b=ZHj+1DelmF1zD8YDzvu7pzOqukagnh0Qfnb9HOvxeRKRDj25vbpOs8DbwcVaDSoe+A ZPwxQE/1kQn33hbOy22HVdVVVGNwXmpnUHYvi1ryUkZ64xbbLgGmDn812TVZ2ffdd6E+ 3WaVIkvwTylXGRwwYcggoVSD5uj+49/eB5SW0LSBV1Nmg3cwDS22z3a+xWNn1hUrekz2 Z8iIm0uSr/RIuLfBGVj5O46GX/J6vdCRUxCMTGMQVP7r60D8klSWfx6zRdI5e5akAlyH OgFpPxIeYFUesvvhc8QLqVNRCAeS83+LoxwC3ov0kOxxNrQg3tjPel/JzNcBSmjJ6eQ2 saWw== X-Gm-Message-State: AOAM532ZysJMf55UD2mcXtscg6+Zo3U7X/tMF3HrungwZfCMLzeXta5x swLBjkB8T7VTmAnz/YNKT/Msmw== X-Received: by 2002:a05:600c:104b:: with SMTP id 11mr8142088wmx.54.1639061803610; Thu, 09 Dec 2021 06:56:43 -0800 (PST) Received: from xps15-9570.lan (host-92-16-105-103.as13285.net. [92.16.105.103]) by smtp.gmail.com with ESMTPSA id y142sm30845wmc.40.2021.12.09.06.56.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Dec 2021 06:56:43 -0800 (PST) From: Peter Griffin To: marcandre.lureau@redhat.com, mst@redhat.com, alex.bennee@linaro.org Subject: [PATCH 8/8] tools/vhost-user-video: Add initial vhost-user-video vmm Date: Thu, 9 Dec 2021 14:56:01 +0000 Message-Id: <20211209145601.331477-9-peter.griffin@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211209145601.331477-1-peter.griffin@linaro.org> References: <20211209145601.331477-1-peter.griffin@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::32b (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=peter.griffin@linaro.org; helo=mail-wm1-x32b.google.com X-Spam_score_int: 6 X-Spam_score: 0.6 X-Spam_bar: / X-Spam_report: (0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.001, PDS_OTHER_BAD_TLD=1.861, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Griffin , qemu-devel@nongnu.org, stratos-dev@op-lists.linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This vmm translates from virtio-video v3 protocol and writes to a v4l2 mem2mem stateful decoder/encoder device [1]. v3 was chosen as that is what the virtio-video Linux frontend driver implements. This allows for testing with the v4l2 vicodec test codec [2] module in the Linux kernel, and is intended to also be used with Arm SoCs that implement a v4l2 stateful decoder/encoder drivers. The advantage of developing & testing with vicodec is that is allows quick development on a purely virtual setup with qemu and a host Linux kernel. Also it allows ci systems like lkft, kernelci to easily test the virtio interface. Currently conversion from virtio-video to v4l2 stateless m2m codec driver or VAAPI drivers is consiered out ot scope as is emulation of a decoder device using a something like ffmpeg. Although this could be added in the future. Note some virtio & v4l2 helpers were based off virtio-video Linux frontend driver and yavta utility, both GPL v2. Example host commands modprobe vicodec vhost-user-video --v4l2-device=/dev/video3 -v --socket-path=video.sock Run Qemu with -device vhost-user-video-pci,chardev=video,id=video Guest decoder v4l2-ctl -d0 -x width=640,height=480 -v width=640,height=480,pixelformat=YU12 --stream-mmap --stream-out-mmap --stream-from jelly_640_480-420P.fwht --stream-to out-jelly-640-480.YU12 [1] https://www.kernel.org/doc/html/latest/userspace-api/media/ v4l/dev-decoder.html [2] https://lwn.net/Articles/760650/ Signed-off-by: Peter Griffin --- tools/vhost-user-video/50-qemu-rpmb.json.in | 5 + tools/vhost-user-video/main.c | 1680 ++++++++++++++++ tools/vhost-user-video/meson.build | 10 + tools/vhost-user-video/v4l2_backend.c | 1777 +++++++++++++++++ tools/vhost-user-video/v4l2_backend.h | 99 + tools/vhost-user-video/virtio_video_helpers.c | 462 +++++ tools/vhost-user-video/virtio_video_helpers.h | 166 ++ tools/vhost-user-video/vuvideo.h | 43 + 8 files changed, 4242 insertions(+) create mode 100644 tools/vhost-user-video/50-qemu-rpmb.json.in create mode 100644 tools/vhost-user-video/main.c create mode 100644 tools/vhost-user-video/meson.build create mode 100644 tools/vhost-user-video/v4l2_backend.c create mode 100644 tools/vhost-user-video/v4l2_backend.h create mode 100644 tools/vhost-user-video/virtio_video_helpers.c create mode 100644 tools/vhost-user-video/virtio_video_helpers.h create mode 100644 tools/vhost-user-video/vuvideo.h diff --git a/tools/vhost-user-video/50-qemu-rpmb.json.in b/tools/vhost-user-video/50-qemu-rpmb.json.in new file mode 100644 index 0000000000..2b033cda56 --- /dev/null +++ b/tools/vhost-user-video/50-qemu-rpmb.json.in @@ -0,0 +1,5 @@ +{ + "description": "QEMU vhost-user-rpmb", + "type": "block", + "binary": "@libexecdir@/vhost-user-rpmb" +} diff --git a/tools/vhost-user-video/main.c b/tools/vhost-user-video/main.c new file mode 100644 index 0000000000..a944efadb6 --- /dev/null +++ b/tools/vhost-user-video/main.c @@ -0,0 +1,1680 @@ +/* + * VIRTIO Video Emulation via vhost-user + * + * Copyright (c) 2021 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#define G_LOG_DOMAIN "vhost-user-video" +#define G_LOG_USE_STRUCTURED 1 + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "libvhost-user-glib.h" +#include "libvhost-user.h" +#include "standard-headers/linux/virtio_video.h" + +#include "qemu/compiler.h" +#include "qemu/iov.h" + +#include "vuvideo.h" +#include "v4l2_backend.h" +#include "virtio_video_helpers.h" + +#ifndef container_of +#define container_of(ptr, type, member) ({ \ + const typeof(((type *) 0)->member) * __mptr = (ptr); \ + (type *) ((char *) __mptr - offsetof(type, member)); }) +#endif + +static gchar *socket_path; +static gchar *v4l2_path; +static gint socket_fd = -1; +static gboolean print_cap; +static gboolean verbose; +static gboolean debug; + +static GOptionEntry options[] = { + { "socket-path", 0, 0, G_OPTION_ARG_FILENAME, &socket_path, + "Location of vhost-user Unix domain socket, " + "incompatible with --fd", "PATH" }, + { "v4l2-device", 0, 0, G_OPTION_ARG_FILENAME, &v4l2_path, + "Location of v4l2 device node", "PATH" }, + { "fd", 0, 0, G_OPTION_ARG_INT, &socket_fd, + "Specify the fd of the backend, " + "incompatible with --socket-path", "FD" }, + { "print-capabilities", 0, 0, G_OPTION_ARG_NONE, &print_cap, + "Output to stdout the backend capabilities " + "in JSON format and exit", NULL}, + { "verbose", 'v', 0, G_OPTION_ARG_NONE, &verbose, + "Be more verbose in output", NULL}, + { "debug", 0, 0, G_OPTION_ARG_NONE, &debug, + "Include debug output", NULL}, + { NULL } +}; + +enum { + VHOST_USER_VIDEO_MAX_QUEUES = 2, +}; + +/* taken from util/iov.c */ +size_t video_iov_size(const struct iovec *iov, const unsigned int iov_cnt) +{ + size_t len; + unsigned int i; + + len = 0; + for (i = 0; i < iov_cnt; i++) { + len += iov[i].iov_len; + } + return len; +} + +static size_t video_iov_to_buf(const struct iovec *iov, + const unsigned int iov_cnt, + size_t offset, void *buf, size_t bytes) +{ + size_t done; + unsigned int i; + for (i = 0, done = 0; (offset || done < bytes) && i < iov_cnt; i++) { + if (offset < iov[i].iov_len) { + size_t len = MIN(iov[i].iov_len - offset, bytes - done); + memcpy(buf + done, iov[i].iov_base + offset, len); + done += len; + offset = 0; + } else { + offset -= iov[i].iov_len; + } + } + assert(offset == 0); + return done; +} + +static size_t video_iov_from_buf(const struct iovec *iov, unsigned int iov_cnt, + size_t offset, const void *buf, size_t bytes) +{ + size_t done; + unsigned int i; + for (i = 0, done = 0; (offset || done < bytes) && i < iov_cnt; i++) { + if (offset < iov[i].iov_len) { + size_t len = MIN(iov[i].iov_len - offset, bytes - done); + memcpy(iov[i].iov_base + offset, buf + done, len); + done += len; + offset = 0; + } else { + offset -= iov[i].iov_len; + } + } + assert(offset == 0); + return done; +} + +static void video_panic(VuDev *dev, const char *msg) +{ + g_critical("%s\n", msg); + exit(EXIT_FAILURE); +} + +static uint64_t video_get_features(VuDev *dev) +{ + g_info("%s: replying", __func__); + return 0; +} + +static void video_set_features(VuDev *dev, uint64_t features) +{ + if (features) { + g_autoptr(GString) s = g_string_new("Requested un-handled feature"); + g_string_append_printf(s, " 0x%" PRIx64 "", features); + g_info("%s: %s", __func__, s->str); + } +} + +/* + * The configuration of the device is static and set when we start the + * daemon. + */ +static int +video_get_config(VuDev *dev, uint8_t *config, uint32_t len) +{ + VuVideo *v = container_of(dev, VuVideo, dev.parent); + + g_return_val_if_fail(len <= sizeof(struct virtio_video_config), -1); + v->virtio_config.version = 0; + v->virtio_config.max_caps_length = MAX_CAPS_LEN; + v->virtio_config.max_resp_length = MAX_CAPS_LEN; + + memcpy(config, &v->virtio_config, len); + + g_debug("%s: config.max_caps_length = %d", __func__ + , ((struct virtio_video_config *)config)->max_caps_length); + g_debug("%s: config.max_resp_length = %d", __func__ + , ((struct virtio_video_config *)config)->max_resp_length); + + return 0; +} + +static int +video_set_config(VuDev *dev, const uint8_t *data, + uint32_t offset, uint32_t size, + uint32_t flags) +{ + g_debug("%s: ", __func__); + /* ignore */ + return 0; +} + +/* + * Handlers for individual control messages + */ + +static void +handle_set_params_cmd(struct VuVideo *v, struct vu_video_ctrl_command *vio_cmd) +{ + int ret = 0; + enum v4l2_buf_type buf_type; + struct virtio_video_set_params *cmd = + (struct virtio_video_set_params *) vio_cmd->cmd_buf; + struct stream *s; + + g_debug("%s: type(x%x) stream_id(%d) %s ", __func__, + cmd->hdr.type, cmd->hdr.stream_id, + vio_queue_name(le32toh(cmd->params.queue_type))); + g_debug("%s: format=0x%x frame_width(%d) frame_height(%d)", + __func__, le32toh(cmd->params.format), + le32toh(cmd->params.frame_width), + le32toh(cmd->params.frame_height)); + g_debug("%s: min_buffers(%d) max_buffers(%d)", __func__, + le32toh(cmd->params.min_buffers), le32toh(cmd->params.max_buffers)); + g_debug("%s: frame_rate(%d) num_planes(%d)", __func__, + le32toh(cmd->params.frame_rate), le32toh(cmd->params.num_planes)); + g_debug("%s: crop top=%d, left=%d, width=%d, height=%d", __func__, + le32toh(cmd->params.crop.left), le32toh(cmd->params.crop.top), + le32toh(cmd->params.crop.width), le32toh(cmd->params.crop.height)); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, cmd->hdr.stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + g_mutex_lock(&s->mutex); + + buf_type = get_v4l2_buf_type(le32toh(cmd->params.queue_type), + s->has_mplane); + + ret = v4l2_video_set_format(s->fd, buf_type, &cmd->params); + if (ret < 0) { + g_error("%s: v4l2_video_set_format() failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + if (is_capture_queue(buf_type)) { + /* decoder supports composing on CAPTURE */ + struct v4l2_selection sel; + memset(&sel, 0, sizeof(struct v4l2_selection)); + + sel.r.left = le32toh(cmd->params.crop.left); + sel.r.top = le32toh(cmd->params.crop.top); + sel.r.width = le32toh(cmd->params.crop.width); + sel.r.height = le32toh(cmd->params.crop.height); + + ret = v4l2_video_set_selection(s->fd, buf_type, &sel); + if (ret < 0) { + g_printerr("%s: v4l2_video_set_selection failed: %s (%d).\n" + , __func__, g_strerror(errno), errno); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + } + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +out: + return; +} + +static void +handle_get_params_cmd(struct VuVideo *v, struct vu_video_ctrl_command *vio_cmd) +{ + int ret; + struct v4l2_format fmt; + struct v4l2_selection sel; + enum v4l2_buf_type buf_type; + struct virtio_video_get_params *cmd = + (struct virtio_video_get_params *) vio_cmd->cmd_buf; + struct virtio_video_get_params_resp getparams_reply; + struct stream *s; + + g_debug("%s: type(0x%x) stream_id(%d) %s", __func__, + cmd->hdr.type, cmd->hdr.stream_id, + vio_queue_name(le32toh(cmd->queue_type))); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found\n" + , __func__, cmd->hdr.stream_id); + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + g_mutex_lock(&s->mutex); + + getparams_reply.hdr.stream_id = cmd->hdr.stream_id; + getparams_reply.params.queue_type = cmd->queue_type; + + buf_type = get_v4l2_buf_type(cmd->queue_type, s->has_mplane); + + ret = v4l2_video_get_format(s->fd, buf_type, &fmt); + if (ret < 0) { + g_printerr("v4l2_video_get_format failed\n"); + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + if (is_capture_queue(buf_type)) { + ret = v4l2_video_get_selection(s->fd, buf_type, &sel); + if (ret < 0) { + g_printerr("v4l2_video_get_selection failed\n"); + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + } + + /* convert from v4l2 to virtio */ + v4l2_to_virtio_video_params(v->v4l2_dev, &fmt, &sel, + &getparams_reply); + + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&getparams_reply, + sizeof(struct virtio_video_get_params_resp)); + g_mutex_unlock(&s->mutex); +out: + return; +} + +struct stream *find_stream(struct VuVideo *v, uint32_t stream_id) +{ + GList *l; + struct stream *s; + + for (l = v->streams; l != NULL; l = l->next) { + s = (struct stream *)l->data; + if (s->stream_id == stream_id) { + return s; + } + } + + return NULL; +} + +int add_resource(struct stream *s, struct resource *r, uint32_t queue_type) +{ + + if (!s || !r) { + return -EINVAL; + } + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + s->inputq_resources = g_list_append(s->inputq_resources, r); + break; + + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + s->outputq_resources = g_list_append(s->outputq_resources, r); + break; + default: + return -EINVAL; + } + + return 0; +} + +void free_resource_mem(struct resource *r) +{ + + /* + * Frees the memory allocated for resource_queue_cmd + * not the memory allocated in resource_create + */ + + if (r->vio_q_cmd) { + g_free(r->vio_q_cmd->cmd_buf); + r->vio_q_cmd->cmd_buf = NULL; + free(r->vio_q_cmd); + r->vio_q_cmd = NULL; + } +} + +void remove_all_resources(struct stream *s, uint32_t queue_type) +{ + GList **resource_list; + struct resource *r; + + /* assumes stream mutex is held by caller */ + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + resource_list = &s->inputq_resources; + break; + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + resource_list = &s->outputq_resources; + break; + default: + g_critical("%s: Invalid virtio queue!", __func__); + return; + } + + g_debug("%s: resource_list has %d elements", __func__ + , g_list_length(*resource_list)); + + GList *l = *resource_list; + while (l != NULL) { + GList *next = l->next; + r = (struct resource *)l->data; + if (r) { + g_debug("%s: Removing resource_id(%d) resource=%p" + , __func__, r->vio_resource.resource_id, r); + + /* + * Assumes that either QUEUE_CLEAR or normal dequeuing + * of buffers will have freed resource_queue cmd memory + */ + + /* free resource memory allocated in resource_create() */ + g_free(r->iov); + g_free(r); + *resource_list = g_list_delete_link(*resource_list, l); + } + l = next; + } +} + +struct resource *find_resource(struct stream *s, uint32_t resource_id, + uint32_t queue_type) +{ + GList *l; + struct resource *r; + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + l = s->inputq_resources; + break; + + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + l = s->outputq_resources; + break; + default: + g_error("%s: Invalid queue type!", __func__); + } + + for (; l != NULL; l = l->next) { + r = (struct resource *)l->data; + if (r->vio_resource.resource_id == resource_id) { + return r; + } + } + + return NULL; +} + +struct resource *find_resource_by_v4l2index(struct stream *s, + enum v4l2_buf_type buf_type, + uint32_t v4l2_index) +{ + GList *l = NULL; + struct resource *r; + + switch (buf_type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + l = s->outputq_resources; + break; + + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + l = s->inputq_resources; + break; + + default: + g_error("Unsupported buffer type\n"); + } + + for (; l != NULL; l = l->next) { + r = (struct resource *)l->data; + if (r->v4l2_index == v4l2_index) { + g_debug("%s: found Resource=%p streamid(%d) resourceid(%d) " + "numplanes(%d) planes_layout(0x%x) vio_q_cmd=%p", __func__, + r, r->stream_id, r->vio_resource.resource_id, + r->vio_resource.num_planes, r->vio_resource.planes_layout, + r->vio_q_cmd); + return r; + } + } + return NULL; +} + +#define EVENT_WQ_IDX 1 + +static void *stream_worker_thread(gpointer data) +{ + int ret; + struct stream *s = data; + VuVideo *v = s->video; + VugDev *vugdev = &v->dev; + VuDev *vudev = &vugdev->parent; + VuVirtq *vq = vu_get_queue(vudev, EVENT_WQ_IDX); + VuVirtqElement *elem; + size_t len; + + struct v4l2_event ev; + struct virtio_video_event vio_event; + + /* select vars */ + fd_set efds, rfds, wfds; + bool have_event, have_read, have_write; + enum v4l2_buf_type buf_type; + + fcntl(s->fd, F_SETFL, fcntl(s->fd, F_GETFL) | O_NONBLOCK); + + while (true) { + int res; + + g_mutex_lock(&s->mutex); + + /* wait for STREAMING or DESTROYING state */ + while (s->stream_state != STREAM_DESTROYING && + s->stream_state != STREAM_STREAMING && + s->stream_state != STREAM_DRAINING) + g_cond_wait(&s->stream_cond, &s->mutex); + + if (s->stream_state == STREAM_DESTROYING) { + g_debug("stream worker thread exiting!"); + s->stream_state = STREAM_DESTROYED; + g_cond_signal(&s->stream_cond); + g_mutex_unlock(&s->mutex); + g_thread_exit(0); + } + + g_mutex_unlock(&s->mutex); + + FD_ZERO(&efds); + FD_SET(s->fd, &efds); + FD_ZERO(&rfds); + FD_SET(s->fd, &rfds); + FD_ZERO(&wfds); + FD_SET(s->fd, &wfds); + + struct timeval tv = { 0 , 500000 }; + res = select(s->fd + 1, &rfds, &wfds, &efds, &tv); + if (res < 0) { + g_printerr("%s:%d - select() failed errno(%s)\n", __func__, + __LINE__, g_strerror(errno)); + break; + } + + if (res == 0) { + g_debug("%s:%d - select() timeout", __func__, __LINE__); + continue; + } + + have_event = FD_ISSET(s->fd, &efds); + have_read = FD_ISSET(s->fd, &rfds); + have_write = FD_ISSET(s->fd, &wfds); + /* read is capture queue, write is output queue */ + + g_debug("%s:%d have_event=%d, have_write=%d, have_read=%d\n" + , __func__, __LINE__, FD_ISSET(s->fd, &efds) + , FD_ISSET(s->fd, &wfds), FD_ISSET(s->fd, &rfds)); + + g_mutex_lock(&s->mutex); + + if (have_event) { + g_debug("%s: have_event!", __func__); + res = ioctl(s->fd, VIDIOC_DQEVENT, &ev); + if (res < 0) { + g_printerr("%s:%d - VIDIOC_DQEVENT failed: errno(%s)\n", + __func__, __LINE__, g_strerror(errno)); + break; + } + v4l2_to_virtio_event(&ev, &vio_event); + + /* get event workqueue */ + elem = vu_queue_pop(vudev, vq, sizeof(struct VuVirtqElement)); + if (!elem) { + g_debug("%s:%d\n", __func__, __LINE__); + break; + } + + len = video_iov_from_buf(elem->in_sg, + elem->in_num, 0, (void *) &vio_event, + sizeof(struct virtio_video_event)); + + vu_queue_push(vudev, vq, elem, len); + vu_queue_notify(vudev, vq); + } + + if (have_read && s->capture_streaming == true) { + /* TODO assumes decoder */ + buf_type = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + + ret = v4l2_dequeue_buffer(s->fd, buf_type, s); + if (ret < 0) { + g_info("%s: v4l2_dequeue_buffer() failed CAPTURE ret(%d)" + , __func__, ret); + + if (errno == EPIPE) { + /* dequeued last buf, so stop streaming */ + ioctl_streamoff(s, buf_type); + } + } + } + + if (have_write && s->output_streaming == true) { + buf_type = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + ret = v4l2_dequeue_buffer(s->fd, buf_type, s); + if (ret < 0) { + g_info("%s: v4l2_dequeue_buffer() failed OUTPUT ret(%d)" + , __func__, ret); + } + } + + g_mutex_unlock(&s->mutex); + } + + return NULL; +} + +void handle_queue_clear_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct virtio_video_queue_clear *cmd = + (struct virtio_video_queue_clear *)vio_cmd->cmd_buf; + int ret = 0; + struct stream *s; + uint32_t stream_id = le32toh(cmd->hdr.stream_id); + enum virtio_video_queue_type queue_type = le32toh(cmd->queue_type); + + g_debug("%s: stream_id(%d) %s\n", __func__, stream_id, + vio_queue_name(queue_type)); + + if (!v || !cmd) { + return; + } + + s = find_stream(v, stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + g_mutex_lock(&s->mutex); + + enum v4l2_buf_type buf_type = + get_v4l2_buf_type(le32toh(cmd->queue_type), s->has_mplane); + + /* + * QUEUE_CLEAR behaviour from virtio-video spec + * Return already queued buffers back from the input or the output queue + * of the device. The device SHOULD return all of the buffers from the + * respective queue as soon as possible without pushing the buffers through + * the processing pipeline. + * + * From v4l2 PoV we issue a VIDIOC_STREAMOFF on the queue which will abort + * or finish any DMA in progress, unlocks any user pointer buffers locked + * in physical memory, and it removes all buffers from the incoming and + * outgoing queues. + */ + + /* issue streamoff */ + ret = ioctl_streamoff(s, buf_type); + if (ret < 0) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + /* iterate the queues resources list - and send a reply to each one */ + + /* + * If the processing was stopped due to VIRTIO_VIDEO_CMD_QUEUE_CLEAR, + * the device MUST respond with VIRTIO_VIDEO_RESP_OK_NODATA as a response + * type and VIRTIO_- VIDEO_BUFFER_FLAG_ERR in flags. + */ + + g_list_foreach(get_resource_list(s, queue_type), + (GFunc)send_qclear_res_reply, s); + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +out: + return; +} + +GList *get_resource_list(struct stream *s, uint32_t queue_type) +{ + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + return s->inputq_resources; + break; + + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + return s->outputq_resources; + break; + default: + g_critical("%s: Unknown queue type!", __func__); + return NULL; + } +} + +void send_ctrl_response(struct vu_video_ctrl_command *vio_cmd, + uint8_t *resp, size_t resp_len) +{ + size_t len; + + virtio_video_ctrl_hdr_htole((struct virtio_video_cmd_hdr *)resp); + + /* send virtio_video_resource_queue_resp */ + len = video_iov_from_buf(vio_cmd->elem.in_sg, + vio_cmd->elem.in_num, 0, resp, resp_len); + + if (len != resp_len) { + g_critical("%s: response size incorrect %zu vs %zu", + __func__, len, resp_len); + } + + vu_queue_push(vio_cmd->dev, vio_cmd->vq, &vio_cmd->elem, len); + vu_queue_notify(vio_cmd->dev, vio_cmd->vq); + + if (vio_cmd->finished) { + g_free(vio_cmd->cmd_buf); + free(vio_cmd); + } +} + +void send_ctrl_response_nodata(struct vu_video_ctrl_command *vio_cmd) +{ + send_ctrl_response(vio_cmd, vio_cmd->cmd_buf, + sizeof(struct virtio_video_cmd_hdr)); +} + +void send_qclear_res_reply(gpointer data, gpointer user_data) +{ + struct resource *r = data; + struct vu_video_ctrl_command *vio_cmd = r->vio_q_cmd; + struct virtio_video_queue_clear *cmd = + (struct virtio_video_queue_clear *) vio_cmd->cmd_buf; + struct virtio_video_resource_queue_resp resp; + + /* + * only need to send replies for buffers that are + * inflight + */ + + if (r->queued) { + + resp.hdr.stream_id = cmd->hdr.stream_id; + resp.hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + resp.flags = htole32(VIRTIO_VIDEO_BUFFER_FLAG_ERR); + resp.timestamp = htole64(r->vio_res_q.timestamp); + + g_debug("%s: stream_id=%d type=0x%x flags=0x%x resource_id=%d t=%llx" + , __func__, resp.hdr.stream_id, resp.hdr.type, resp.flags, + r->vio_resource.resource_id, resp.timestamp); + + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *) &resp, + sizeof(struct virtio_video_resource_queue_resp)); + } + return; +} + +static int +handle_resource_create_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + int ret = 0, i; + uint32_t total_entries = 0; + uint32_t stream_id ; + struct virtio_video_resource_create *cmd = + (struct virtio_video_resource_create *)vio_cmd->cmd_buf; + struct virtio_video_mem_entry *mem; + struct resource *res; + struct virtio_video_resource_create *r; + struct stream *s; + enum virtio_video_mem_type mem_type; + + stream_id = cmd->hdr.stream_id; + + s = find_stream(v, stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + g_mutex_lock(&s->mutex); + + if (le32toh(cmd->resource_id) == 0) { + g_critical("%s: resource id 0 is not allowed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + /* check resource id doesn't already exist */ + res = find_resource(s, le32toh(cmd->resource_id), le32toh(cmd->queue_type)); + if (res) { + g_critical("%s: resource_id:%d already exists" + , __func__, le32toh(cmd->resource_id)); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } else { + res = g_new0(struct resource, 1); + res->vio_resource.resource_id = le32toh(cmd->resource_id); + res->vio_resource.queue_type = le32toh(cmd->queue_type); + res->vio_resource.planes_layout = le32toh(cmd->planes_layout); + + res->vio_resource.num_planes = le32toh(cmd->num_planes); + r = &res->vio_resource; + + ret = add_resource(s, res, le32toh(cmd->queue_type)); + if (ret) { + g_critical("%s: resource_add id:%d failed" + , __func__, le32toh(cmd->resource_id)); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } + + g_debug("%s: resource=%p streamid(%d) resourceid(%d) numplanes(%d)" + "planes_layout(0x%x) %s", + __func__, res, res->stream_id, r->resource_id, r->num_planes, + r->planes_layout, vio_queue_name(r->queue_type)); + } + + if (r->planes_layout & VIRTIO_VIDEO_PLANES_LAYOUT_PER_PLANE) { + g_debug("%s: streamid(%d) resourceid(%d) planes_layout(0x%x)" + , __func__, res->stream_id, r->resource_id, r->planes_layout); + + for (i = 0; i < r->num_planes; i++) { + total_entries += le32toh(cmd->num_entries[i]); + g_debug("%s: streamid(%d) resourceid(%d) num_entries[%d]=%d" + , __func__, res->stream_id, r->resource_id, + i, le32toh(cmd->num_entries[i])); + } + } else { + total_entries = 1; + } + + /* + * virtio_video_resource_create is followed by either + * - struct virtio_video_mem_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES + * - struct virtio_video_object_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT + */ + + if (r->queue_type == VIRTIO_VIDEO_QUEUE_TYPE_INPUT) { + mem_type = s->vio_stream.in_mem_type; + } else { + mem_type = s->vio_stream.out_mem_type; + } + /* + * Followed by either + * - struct virtio_video_mem_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES + * - struct virtio_video_object_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT + */ + + if (mem_type == VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES) { + mem = (void *)cmd + sizeof(struct virtio_video_resource_create); + + res->iov = g_malloc0(sizeof(struct iovec) * total_entries); + for (i = 0; i < total_entries; i++) { + uint64_t len = le32toh(mem[i].length); + g_debug("%s: mem[%d] addr=0x%lx", __func__ + , i, le64toh(mem[i].addr)); + + res->iov[i].iov_len = le32toh(mem[i].length); + res->iov[i].iov_base = + vu_gpa_to_va(&v->dev.parent, &len, le64toh(mem[i].addr)); + g_debug("%s: [%d] iov_len = 0x%lx", __func__ + , i, res->iov[i].iov_len); + g_debug("%s: [%d] iov_base = 0x%p", __func__ + , i, res->iov[i].iov_base); + } + res->iov_count = total_entries; + + } else if (mem_type == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT) { + g_critical("%s: VIRTIO_OBJECT not implemented!", __func__); + /* TODO implement VIRTIO_OBJECT support */ + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + /* check underlying driver supports GUEST_PAGES */ + enum v4l2_buf_type buf_type = + get_v4l2_buf_type(r->queue_type, s->has_mplane); + + ret = v4l2_resource_create(s, buf_type, mem_type, res); + if (ret < 0) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +out: + return ret; +} + +static int +handle_resource_queue_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct virtio_video_resource_queue *cmd = + (struct virtio_video_resource_queue *)vio_cmd->cmd_buf; + struct resource *res; + struct stream *s; + uint32_t stream_id; + int ret = 0; + + g_debug("%s: type(0x%x) %s resource_id(%d)", __func__, + cmd->hdr.type, vio_queue_name(le32toh(cmd->queue_type)), + le32toh(cmd->resource_id)); + g_debug("%s: num_data_sizes = %d", __func__, le32toh(cmd->num_data_sizes)); + g_debug("%s: data_sizes[0] = %d", __func__, le32toh(cmd->data_sizes[0])); + + stream_id = cmd->hdr.stream_id; + + s = find_stream(v, stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + g_mutex_lock(&s->mutex); + + if (cmd->resource_id == 0) { + g_critical("%s: resource id 0 is not allowed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } + + /* get resource object */ + res = find_resource(s, le32toh(cmd->resource_id), le32toh(cmd->queue_type)); + if (!res) { + g_critical("%s: resource_id:%d does not exist!" + , __func__, le32toh(cmd->resource_id)); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } + + res->vio_res_q.timestamp = le64toh(cmd->timestamp); + res->vio_res_q.num_data_sizes = le32toh(cmd->num_data_sizes); + res->vio_res_q.queue_type = le32toh(cmd->queue_type); + res->vio_q_cmd = vio_cmd; + + g_debug("%s: res=%p res->vio_q_cmd=0x%p", __func__, res, res->vio_q_cmd); + + enum v4l2_buf_type buf_type = get_v4l2_buf_type( + cmd->queue_type, s->has_mplane); + + + ret = v4l2_queue_buffer(s->fd, buf_type, cmd, res, s, v->v4l2_dev); + if (ret < 0) { + g_critical("%s: v4l2_queue_buffer failed", __func__); + /* virtio error set by v4l2_queue_buffer */ + goto out_unlock; + } + + /* + * let the stream worker thread do the dequeueing of output and + * capture queue buffers and send the resource_queue replies + */ + + g_mutex_unlock(&s->mutex); + return ret; + +out_unlock: + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +out: + return ret; +} + + +static void +handle_resource_destroy_all_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct virtio_video_resource_destroy_all *cmd = + (struct virtio_video_resource_destroy_all *)vio_cmd->cmd_buf; + enum v4l2_buf_type buf_type; + struct stream *s; + int ret = 0; + + g_debug("%s: type(0x%x) %s stream_id(%d)", __func__, + cmd->hdr.type, vio_queue_name(le32toh(cmd->queue_type)), + cmd->hdr.stream_id); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, cmd->hdr.stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + g_mutex_lock(&s->mutex); + + buf_type = get_v4l2_buf_type(le32toh(cmd->queue_type), s->has_mplane); + + ret = v4l2_free_buffers(s->fd, buf_type); + if (ret) { + g_critical("%s: v4l2_free_buffers() failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + remove_all_resources(s, le32toh(cmd->queue_type)); + + /* free resource objects from queue list */ + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +} + +static void +handle_stream_create_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + int ret = 0; + struct stream *s; + uint32_t req_stream_id; + uint32_t coded_format; + + struct virtio_video_stream_create *cmd = + (struct virtio_video_stream_create *)vio_cmd->cmd_buf; + + g_debug("%s: type(0x%x) stream_id(%d) in_mem_type(0x%x) " + "out_mem_type(0x%x) coded_format(0x%x)", + __func__, cmd->hdr.type, cmd->hdr.stream_id, + le32toh(cmd->in_mem_type), le32toh(cmd->out_mem_type), + le32toh(cmd->coded_format)); + + req_stream_id = cmd->hdr.stream_id; + coded_format = le32toh(cmd->coded_format); + + if ((le32toh(cmd->in_mem_type) == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT) || + (le32toh(cmd->out_mem_type) == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT)) { + /* TODO implement VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT */ + g_printerr("%s: MEM_TYPE_VIRTIO_OBJECT not supported yet", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + if (!find_stream(v, req_stream_id)) { + s = g_new0(struct stream, 1); + /* copy but bswap */ + s->vio_stream.in_mem_type = le32toh(cmd->in_mem_type); + s->vio_stream.out_mem_type = le32toh(cmd->out_mem_type); + s->vio_stream.coded_format = le32toh(cmd->coded_format); + strncpy((char *)&s->vio_stream.tag, (char *)cmd->tag, + sizeof(cmd->tag) - 1); + s->vio_stream.tag[sizeof(cmd->tag) - 1] = 0; + s->stream_id = req_stream_id; + s->video = v; + s->stream_state = STREAM_STOPPED; + s->has_mplane = v->v4l2_dev->has_mplane; + g_mutex_init(&s->mutex); + g_cond_init(&s->stream_cond); + v->streams = g_list_append(v->streams, s); + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + } else { + g_debug("%s: Stream ID in use - ", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID; + goto out; + } + + /* set the requested coded format */ + ret = v4l2_stream_create(v->v4l2_dev, coded_format, s); + if (ret < 0) { + g_printerr("%s: v4l2_stream_create() failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + + v->streams = g_list_remove(v->streams, s); + g_free(s); + } + + /* + * create thread to handle + * - dequeing buffers from output & capture queues + * - sending resource replies for buffers + * - handling EOS and dynamic-resoltion events + */ + s->worker_thread = g_thread_new("vio-video stream worker", + stream_worker_thread, s); + +out: + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); +} + +static void +handle_stream_drain_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + int ret; + struct stream *s; + uint32_t stream_id; + struct virtio_video_stream_drain *cmd = + (struct virtio_video_stream_drain *)vio_cmd->cmd_buf; + + stream_id = cmd->hdr.stream_id; + + g_debug("%s: stream_id(%d)", __func__, stream_id); + + s = find_stream(v, stream_id); + if (!s) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID; + return; + } + + g_debug("%s: Found stream=0x%p", __func__, s); + + g_mutex_lock(&s->mutex); + + ret = v4l2_issue_cmd(s->fd, V4L2_DEC_CMD_STOP, 0); + if (ret < 0) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + s->stream_state = STREAM_DRAINING; + g_cond_signal(&s->stream_cond); + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +} + +static void +handle_stream_destroy_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct stream *s; + uint32_t stream_id; + struct virtio_video_stream_destroy *cmd = + (struct virtio_video_stream_destroy *)vio_cmd->cmd_buf; + enum v4l2_buf_type buftype; + + if (!v || !vio_cmd) { + return; + } + + stream_id = cmd->hdr.stream_id; + + g_debug("%s: stream_id=(%d)", __func__, stream_id); + + s = find_stream(v, stream_id); + + if (!s) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID; + return; + } + + if (s) { + g_debug("%s: Found stream=0x%p", __func__, s); + + g_mutex_lock(&s->mutex); + /* TODO assumes decoder */ + buftype = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + v4l2_streamoff(buftype, s); + + /* signal worker thread */ + s->stream_state = STREAM_DESTROYING; + g_cond_signal(&s->stream_cond); + g_mutex_unlock(&s->mutex); + + /* wait for DESTROYED state */ + g_mutex_lock(&s->mutex); + while (s->stream_state != STREAM_DESTROYED) { + g_cond_wait(&s->stream_cond, &s->mutex); + } + + /* stream worker thread now exited */ + + /* deallocate the buffers */ + v4l2_free_buffers(s->fd, buftype); + remove_all_resources(s, VIRTIO_VIDEO_QUEUE_TYPE_INPUT); + + buftype = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : + V4L2_BUF_TYPE_VIDEO_CAPTURE; + + v4l2_free_buffers(s->fd, buftype); + remove_all_resources(s, VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT); + + g_cond_clear(&s->stream_cond); + + v4l2_close(s->fd); + + v->streams = g_list_remove(v->streams, (gconstpointer) s); + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + } + + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); + g_mutex_clear(&s->mutex); + g_free(s); + + return; +} + +struct virtio_video_get_control_resp_level { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_control_val_level level; +}; + +struct virtio_video_get_control_resp_profile { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_control_val_profile profile; +}; + +struct virtio_video_get_control_resp_bitrate { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_control_val_bitrate bitrate; +}; + +static int +handle_get_control_cmd(struct VuVideo *v, struct vu_video_ctrl_command *vio_cmd) +{ + int ret; + uint32_t v4l2_control; + int32_t value; + + struct virtio_video_get_control_resp ctl_resp_err; + struct virtio_video_get_control_resp_level ctl_resp_level; + struct virtio_video_get_control_resp_profile ctl_resp_profile; + struct virtio_video_get_control_resp_bitrate ctl_resp_bitrate; + + struct stream *s; + + struct virtio_video_query_control *cmd = + (struct virtio_video_query_control *)vio_cmd->cmd_buf; + + g_debug("%s: type(0x%x) stream_id(%d) control(0x%x)", __func__, + cmd->hdr.type, cmd->hdr.stream_id, le32toh(cmd->control)); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, cmd->hdr.stream_id); + goto out; + } + + g_mutex_lock(&s->mutex); + + v4l2_control = virtio_video_control_to_v4l2(le32toh(cmd->control)); + if (!v4l2_control) { + goto out_err_unlock; + } + + + switch (le32toh(cmd->control)) { + case VIRTIO_VIDEO_CONTROL_BITRATE: + g_debug("%s: VIRTIO_VIDEO_CONTROL_BITRATE", __func__); + + ctl_resp_bitrate.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_bitrate.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + + if (v->v4l2_dev->dev_type == STATEFUL_ENCODER) { + ret = v4l2_video_get_control(s->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_video_get_control() failed\n"); + goto out_err_unlock; + } + ctl_resp_bitrate.bitrate.bitrate = htole32(value); + + } else { + g_debug("%s: CONTROL_BITRATE unsupported for decoders!", __func__); + goto out_err_unlock; + } + + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&ctl_resp_bitrate, + sizeof(struct virtio_video_get_control_resp_bitrate)); + break; + + case VIRTIO_VIDEO_CONTROL_PROFILE: + g_debug("%s: VIRTIO_VIDEO_CONTROL_PROFILE", __func__); + + ctl_resp_profile.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_profile.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + + ret = v4l2_video_get_control(s->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_video_get_control() failed\n"); + goto out_err_unlock; + } + + ctl_resp_profile.profile.profile = htole32(value); + + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&ctl_resp_profile, + sizeof(struct virtio_video_get_control_resp_profile)); + + /* + * TODO need to determine "in use" codec to (h264/vp8/vp9) to map to + * v4l2 control for PROFILE? + */ + + break; + + case VIRTIO_VIDEO_CONTROL_LEVEL: + g_debug("%s: VIRTIO_VIDEO_CONTROL_LEVEL", __func__); + + ctl_resp_level.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_level.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + + ret = v4l2_video_get_control(s->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_video_get_control() failed\n"); + goto out_err_unlock; + } + + ctl_resp_level.level.level = htole32(value); + + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&ctl_resp_level, + sizeof(struct virtio_video_get_control_resp_level)); + break; + + default: + g_critical("Unknown control requested!"); + goto out_err_unlock; + break; + } + + return 0; + +out_err_unlock: + ctl_resp_err.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_err.hdr.type = VIRTIO_VIDEO_RESP_ERR_UNSUPPORTED_CONTROL; + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&ctl_resp_err, + sizeof(struct virtio_video_get_control_resp)); + g_mutex_unlock(&s->mutex); +out: + return -EINVAL; +} + +static int +handle_query_capability_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *cmd) +{ + GList *fmt_l; + int ret; + enum v4l2_buf_type buf_type; + struct virtio_video_query_capability *qcmd = + (struct virtio_video_query_capability *)cmd->cmd_buf; + GByteArray *querycapresp; + + /* hdr bswapped already */ + g_debug("%s: type(0x%x) stream_id(%d) %s", __func__, + qcmd->hdr.type, qcmd->hdr.stream_id, + vio_queue_name(le32toh(qcmd->queue_type))); + + buf_type = get_v4l2_buf_type(le32toh(qcmd->queue_type), + v->v4l2_dev->has_mplane); + + /* enumerate formats */ + ret = video_enum_formats(v->v4l2_dev, buf_type, &fmt_l, false); + if (ret < 0) { + g_printerr("video_enum_formats failed"); + return ret; + } + + querycapresp = g_byte_array_new(); + querycapresp = create_query_cap_resp(qcmd, &fmt_l, querycapresp); + cmd->finished = true; + send_ctrl_response(cmd, querycapresp->data, querycapresp->len); + + video_free_formats(&fmt_l); + g_byte_array_free(querycapresp, true); + + return 0; +} + +/* for v3 virtio-video spec currently */ +static void +video_handle_ctrl(VuDev *dev, int qidx) +{ + VuVirtq *vq = vu_get_queue(dev, qidx); + VuVideo *video = container_of(dev, VuVideo, dev.parent); + size_t cmd_len, len; + + struct vu_video_ctrl_command *cmd; + + for (;;) { + + cmd = vu_queue_pop(dev, vq, sizeof(struct vu_video_ctrl_command)); + if (!cmd) { + break; + } + + cmd->vq = vq; + cmd->error = 0; + cmd->finished = false; + cmd->dev = dev; + + cmd_len = video_iov_size(cmd->elem.out_sg, cmd->elem.out_num); + cmd->cmd_buf = g_malloc0(cmd_len); + len = video_iov_to_buf(cmd->elem.out_sg, cmd->elem.out_num, + 0, cmd->cmd_buf, cmd_len); + + if (len != cmd_len) { + g_warning("%s: command size incorrect %zu vs %zu\n", + __func__, len, cmd_len); + } + + /* header is first on every cmd struct */ + cmd->cmd_hdr = (struct virtio_video_cmd_hdr *) cmd->cmd_buf; + /*bswap header */ + virtio_video_ctrl_hdr_letoh(cmd->cmd_hdr); + + switch (cmd->cmd_hdr->type) { + case VIRTIO_VIDEO_CMD_QUERY_CAPABILITY: + g_debug("Received VIRTIO_VIDEO_CMD_QUERY_CAPABILITY cmd"); + handle_query_capability_cmd(video, cmd); + break; + + case VIRTIO_VIDEO_CMD_STREAM_CREATE: + g_debug("Received VIRTIO_VIDEO_CMD_STREAM_CREATE cmd"); + handle_stream_create_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_STREAM_DESTROY: + g_debug("Received VIRTIO_VIDEO_CMD_STREAM_DESTROY cmd"); + handle_stream_destroy_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_STREAM_DRAIN: + g_debug("Received VIRTIO_VIDEO_CMD_STREAM_DRAIN cmd"); + handle_stream_drain_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_RESOURCE_CREATE: + g_debug("Received VIRTIO_VIDEO_CMD_RESOURCE_CREATE cmd"); + handle_resource_create_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_RESOURCE_QUEUE: + g_debug("Received VIRTIO_VIDEO_CMD_RESOURCE_QUEUE cmd"); + handle_resource_queue_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL: + g_debug("Received VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL cmd"); + handle_resource_destroy_all_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_QUEUE_CLEAR: + g_debug("Received VIRTIO_VIDEO_CMD_QUEUE_CLEAR cmd"); + handle_queue_clear_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_GET_PARAMS: + g_debug("Received VIRTIO_VIDEO_CMD_GET_PARAMS cmd"); + handle_get_params_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_SET_PARAMS: + g_debug("Received VIRTIO_VIDEO_CMD_SET_PARAMS cmd"); + handle_set_params_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_QUERY_CONTROL: + g_error("**** VIRTIO_VIDEO_CMD_QUERY_CONTROL unimplemented!"); + break; + case VIRTIO_VIDEO_CMD_GET_CONTROL: + g_debug("Received VIRTIO_VIDEO_CMD_GET_CONTROL cmd"); + handle_get_control_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_SET_CONTROL: + g_error("**** VIRTIO_VIDEO_CMD_SET_CONTROL unimplemented!"); + break; + default: + g_error("Unknown VIRTIO_VIDEO command!"); + break; + } + } +} + +static void +video_queue_set_started(VuDev *dev, int qidx, bool started) +{ + VuVirtq *vq = vu_get_queue(dev, qidx); + + g_debug("queue started %d:%d\n", qidx, started); + + switch (qidx) { + case 0: + vu_set_queue_handler(dev, vq, started ? video_handle_ctrl : NULL); + break; + default: + break; + } +} + +/* + * video_process_msg: process messages of vhost-user interface + * + * Any that are not handled here are processed by the libvhost library + * itself. + */ +static int video_process_msg(VuDev *dev, VhostUserMsg *msg, int *do_reply) +{ + VuVideo *r = container_of(dev, VuVideo, dev.parent); + + g_debug("%s: msg %d", __func__, msg->request); + + switch (msg->request) { + case VHOST_USER_NONE: + g_main_loop_quit(r->loop); + return 1; + default: + return 0; + } + + return 0; +} + +static const VuDevIface vuiface = { + .set_features = video_set_features, + .get_features = video_get_features, + .queue_set_started = video_queue_set_started, + .process_msg = video_process_msg, + .get_config = video_get_config, + .set_config = video_set_config, +}; + +static void video_destroy(VuVideo *v) +{ + vug_deinit(&v->dev); + if (socket_path) { + unlink(socket_path); + } + + v4l2_backend_free(v->v4l2_dev); +} + +/* Print vhost-user.json backend program capabilities */ +static void print_capabilities(void) +{ + printf("{\n"); + printf(" \"type\": \"misc\"\n"); + printf("}\n"); +} + +static gboolean hangup(gpointer user_data) +{ + GMainLoop *loop = (GMainLoop *) user_data; + g_info("%s: caught hangup/quit signal, quitting main loop", __func__); + g_main_loop_quit(loop); + return true; +} + +int main(int argc, char *argv[]) +{ + GError *error = NULL; + GOptionContext *context; + g_autoptr(GSocket) socket = NULL; + VuVideo video = { }; + + context = g_option_context_new("vhost-user emulation of video device"); + g_option_context_add_main_entries(context, options, "vhost-user-video"); + if (!g_option_context_parse(context, &argc, &argv, &error)) { + g_printerr("option parsing failed: %s\n", error->message); + exit(1); + } + + g_option_context_free(context); + + if (print_cap) { + print_capabilities(); + exit(0); + } + + if (!socket_path && socket_fd < 0) { + g_printerr("Please specify either --fd or --socket-path\n"); + exit(EXIT_FAILURE); + } + + if (verbose || debug) { + g_log_set_handler(NULL, G_LOG_LEVEL_MASK, g_log_default_handler, NULL); + if (debug) { + g_setenv("G_MESSAGES_DEBUG", "all", true); + } + } else { + g_log_set_handler(NULL, + G_LOG_LEVEL_WARNING | G_LOG_LEVEL_CRITICAL + | G_LOG_LEVEL_ERROR, + g_log_default_handler, NULL); + } + + /* + * Open the v4l2 device and enumerate supported formats. + * Use this to determine whether it is a stateful encoder/decoder. + */ + if (!v4l2_path || !g_file_test(v4l2_path, G_FILE_TEST_EXISTS)) { + g_printerr("Please specify a valid --v4l2-device\n"); + exit(EXIT_FAILURE); + } else { + video.v4l2_dev = v4l2_backend_init(v4l2_path); + if (!video.v4l2_dev) { + g_printerr("v4l2 backend init failed!\n"); + exit(EXIT_FAILURE); + } + } + + /* + * Now create a vhost-user socket that we will receive messages + * on. Once we have our handler set up we can enter the glib main + * loop. + */ + if (socket_path) { + g_autoptr(GSocketAddress) addr = g_unix_socket_address_new(socket_path); + g_autoptr(GSocket) bind_socket = + g_socket_new(G_SOCKET_FAMILY_UNIX, G_SOCKET_TYPE_STREAM, + G_SOCKET_PROTOCOL_DEFAULT, &error); + + if (!g_socket_bind(bind_socket, addr, false, &error)) { + g_printerr("Failed to bind to socket at %s (%s).\n", + socket_path, error->message); + exit(EXIT_FAILURE); + } + if (!g_socket_listen(bind_socket, &error)) { + g_printerr("Failed to listen on socket %s (%s).\n", + socket_path, error->message); + } + g_message("awaiting connection to %s", socket_path); + socket = g_socket_accept(bind_socket, NULL, &error); + if (!socket) { + g_printerr("Failed to accept on socket %s (%s).\n", + socket_path, error->message); + } + } else { + socket = g_socket_new_from_fd(socket_fd, &error); + if (!socket) { + g_printerr("Failed to connect to FD %d (%s).\n", + socket_fd, error->message); + exit(EXIT_FAILURE); + } + } + + /* + * Create the main loop first so all the various sources can be + * added. As well as catching signals we need to ensure vug_init + * can add it's GSource watches. + */ + + video.loop = g_main_loop_new(NULL, FALSE); + /* catch exit signals */ + g_unix_signal_add(SIGHUP, hangup, video.loop); + g_unix_signal_add(SIGINT, hangup, video.loop); + + if (!vug_init(&video.dev, VHOST_USER_VIDEO_MAX_QUEUES, + g_socket_get_fd(socket), + video_panic, &vuiface)) { + g_printerr("Failed to initialize libvhost-user-glib.\n"); + exit(EXIT_FAILURE); + } + + g_message("entering main loop, awaiting messages"); + g_main_loop_run(video.loop); + g_message("finished main loop, cleaning up"); + + g_main_loop_unref(video.loop); + video_destroy(&video); +} diff --git a/tools/vhost-user-video/meson.build b/tools/vhost-user-video/meson.build new file mode 100644 index 0000000000..931e73c15d --- /dev/null +++ b/tools/vhost-user-video/meson.build @@ -0,0 +1,10 @@ +executable('vhost-user-video', files( + 'main.c', 'v4l2_backend.c', 'virtio_video_helpers.c'), + dependencies: [qemuutil, glib, gio, vhost_user], + install: true, + install_dir: get_option('libexecdir')) + +configure_file(input: '50-qemu-rpmb.json.in', + output: '50-qemu-rpmb.json', + configuration: config_host, + install_dir: qemu_datadir / 'vhost-user') diff --git a/tools/vhost-user-video/v4l2_backend.c b/tools/vhost-user-video/v4l2_backend.c new file mode 100644 index 0000000000..f90b76b172 --- /dev/null +++ b/tools/vhost-user-video/v4l2_backend.c @@ -0,0 +1,1777 @@ +/* + * virtio-video video v4l2 backend + * + * The purpose of this backend is to interface with + * v4l2 stateful encoder and decoder devices in the kernel. + * + * v4l2 stateless devices are NOT supported currently. + * + * Some v4l2 helper functions taken from yatva + * + * Copyright (c) 2021 Linaro Ltd + * Copyright (C) 2005-2010 Laurent Pinchart + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include +#include "virtio_video_helpers.h" +#include "v4l2_backend.h" +#include "standard-headers/linux/virtio_video.h" +#include "vuvideo.h" + +/* function prototypes */ + +static const struct v4l2_format_info *v4l2_format_by_fourcc(unsigned int fourcc); +static const char *v4l2_format_name(unsigned int fourcc); +static const char *v4l2_buf_type_name(enum v4l2_buf_type type); +static const char *v4l2_field_name(enum v4l2_field field); + +static int video_enum_frame_intervals(struct v4l2_device *dev, + __u32 pixelformat, + unsigned int width, unsigned int height, + GList **p_vid_fmt_frm_rate_l); + +static int video_enum_frame_sizes(struct v4l2_device *dev, __u32 pixelformat, + GList **p_vid_fmt_frm_l); +static int video_querycap(struct v4l2_device *dev); +static GByteArray *iterate_frame_rate_list(GByteArray *resp, + GList *frm_rate_l); +static GByteArray *iterate_format_frame_list(GByteArray *resp, + GList *fmt_frm_l); +static GByteArray *iterate_format_desc_list(GByteArray *resp, + GList *fmt_desc_l); + +/* v4l2 to str tables & helpers taken from yavta to make prettier logs */ + +static struct v4l2_format_info { + const char *name; + unsigned int fourcc; + unsigned char n_planes; +} pixel_formats[] = { + { "RGB332", V4L2_PIX_FMT_RGB332, 1 }, + { "RGB444", V4L2_PIX_FMT_RGB444, 1 }, + { "ARGB444", V4L2_PIX_FMT_ARGB444, 1 }, + { "XRGB444", V4L2_PIX_FMT_XRGB444, 1 }, + { "RGB555", V4L2_PIX_FMT_RGB555, 1 }, + { "ARGB555", V4L2_PIX_FMT_ARGB555, 1 }, + { "XRGB555", V4L2_PIX_FMT_XRGB555, 1 }, + { "RGB565", V4L2_PIX_FMT_RGB565, 1 }, + { "RGB555X", V4L2_PIX_FMT_RGB555X, 1 }, + { "RGB565X", V4L2_PIX_FMT_RGB565X, 1 }, + { "BGR666", V4L2_PIX_FMT_BGR666, 1 }, + { "BGR24", V4L2_PIX_FMT_BGR24, 1 }, + { "RGB24", V4L2_PIX_FMT_RGB24, 1 }, + { "BGR32", V4L2_PIX_FMT_BGR32, 1 }, + { "ABGR32", V4L2_PIX_FMT_ABGR32, 1 }, + { "XBGR32", V4L2_PIX_FMT_XBGR32, 1 }, + { "RGB32", V4L2_PIX_FMT_RGB32, 1 }, + { "ARGB32", V4L2_PIX_FMT_ARGB32, 1 }, + { "XRGB32", V4L2_PIX_FMT_XRGB32, 1 }, + { "HSV24", V4L2_PIX_FMT_HSV24, 1 }, + { "HSV32", V4L2_PIX_FMT_HSV32, 1 }, + { "Y8", V4L2_PIX_FMT_GREY, 1 }, + { "Y10", V4L2_PIX_FMT_Y10, 1 }, + { "Y12", V4L2_PIX_FMT_Y12, 1 }, + { "Y16", V4L2_PIX_FMT_Y16, 1 }, + { "UYVY", V4L2_PIX_FMT_UYVY, 1 }, + { "VYUY", V4L2_PIX_FMT_VYUY, 1 }, + { "YUYV", V4L2_PIX_FMT_YUYV, 1 }, + { "YVYU", V4L2_PIX_FMT_YVYU, 1 }, + { "NV12", V4L2_PIX_FMT_NV12, 1 }, + { "NV12M", V4L2_PIX_FMT_NV12M, 2 }, + { "NV21", V4L2_PIX_FMT_NV21, 1 }, + { "NV21M", V4L2_PIX_FMT_NV21M, 2 }, + { "NV16", V4L2_PIX_FMT_NV16, 1 }, + { "NV16M", V4L2_PIX_FMT_NV16M, 2 }, + { "NV61", V4L2_PIX_FMT_NV61, 1 }, + { "NV61M", V4L2_PIX_FMT_NV61M, 2 }, + { "NV24", V4L2_PIX_FMT_NV24, 1 }, + { "NV42", V4L2_PIX_FMT_NV42, 1 }, + { "YU12", V4L2_PIX_FMT_YVU420, 1}, + { "YUV420M", V4L2_PIX_FMT_YUV420M, 3 }, + { "YUV422M", V4L2_PIX_FMT_YUV422M, 3 }, + { "YUV444M", V4L2_PIX_FMT_YUV444M, 3 }, + { "YVU420M", V4L2_PIX_FMT_YVU420M, 3 }, + { "YVU422M", V4L2_PIX_FMT_YVU422M, 3 }, + { "YVU444M", V4L2_PIX_FMT_YVU444M, 3 }, + { "SBGGR8", V4L2_PIX_FMT_SBGGR8, 1 }, + { "SGBRG8", V4L2_PIX_FMT_SGBRG8, 1 }, + { "SGRBG8", V4L2_PIX_FMT_SGRBG8, 1 }, + { "SRGGB8", V4L2_PIX_FMT_SRGGB8, 1 }, + { "SBGGR10_DPCM8", V4L2_PIX_FMT_SBGGR10DPCM8, 1 }, + { "SGBRG10_DPCM8", V4L2_PIX_FMT_SGBRG10DPCM8, 1 }, + { "SGRBG10_DPCM8", V4L2_PIX_FMT_SGRBG10DPCM8, 1 }, + { "SRGGB10_DPCM8", V4L2_PIX_FMT_SRGGB10DPCM8, 1 }, + { "SBGGR10", V4L2_PIX_FMT_SBGGR10, 1 }, + { "SGBRG10", V4L2_PIX_FMT_SGBRG10, 1 }, + { "SGRBG10", V4L2_PIX_FMT_SGRBG10, 1 }, + { "SRGGB10", V4L2_PIX_FMT_SRGGB10, 1 }, + { "SBGGR10P", V4L2_PIX_FMT_SBGGR10P, 1 }, + { "SGBRG10P", V4L2_PIX_FMT_SGBRG10P, 1 }, + { "SGRBG10P", V4L2_PIX_FMT_SGRBG10P, 1 }, + { "SRGGB10P", V4L2_PIX_FMT_SRGGB10P, 1 }, + { "SBGGR12", V4L2_PIX_FMT_SBGGR12, 1 }, + { "SGBRG12", V4L2_PIX_FMT_SGBRG12, 1 }, + { "SGRBG12", V4L2_PIX_FMT_SGRBG12, 1 }, + { "SRGGB12", V4L2_PIX_FMT_SRGGB12, 1 }, + { "IPU3_SBGGR10", V4L2_PIX_FMT_IPU3_SBGGR10, 1 }, + { "IPU3_SGBRG10", V4L2_PIX_FMT_IPU3_SGBRG10, 1 }, + { "IPU3_SGRBG10", V4L2_PIX_FMT_IPU3_SGRBG10, 1 }, + { "IPU3_SRGGB10", V4L2_PIX_FMT_IPU3_SRGGB10, 1 }, + { "DV", V4L2_PIX_FMT_DV, 1 }, + { "MJPEG", V4L2_PIX_FMT_MJPEG, 1 }, + { "MPEG", V4L2_PIX_FMT_MPEG, 1 }, + { "FWHT", V4L2_PIX_FMT_FWHT, 1 }, +}; + +bool video_is_mplane(enum v4l2_buf_type type) +{ + return type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE || + type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; +} + +bool video_is_splane(enum v4l2_buf_type type) +{ + return type == V4L2_BUF_TYPE_VIDEO_CAPTURE || + type == V4L2_BUF_TYPE_VIDEO_OUTPUT; +} +bool video_is_meta(enum v4l2_buf_type type) +{ + return type == V4L2_BUF_TYPE_META_CAPTURE || + type == V4L2_BUF_TYPE_META_OUTPUT; +} + +bool is_capture_queue(enum v4l2_buf_type type) +{ + return type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE || + type == V4L2_BUF_TYPE_VIDEO_CAPTURE || + type == V4L2_BUF_TYPE_META_CAPTURE; +} + +bool is_output_queue(enum v4l2_buf_type type) +{ + return type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE || + type == V4L2_BUF_TYPE_VIDEO_OUTPUT || + type == V4L2_BUF_TYPE_META_OUTPUT; +} + +static const struct v4l2_format_info *v4l2_format_by_fourcc(unsigned int fourcc) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(pixel_formats); ++i) { + if (pixel_formats[i].fourcc == fourcc) { + return &pixel_formats[i]; + } + } + + return NULL; +} + +static const char *v4l2_format_name(unsigned int fourcc) +{ + const struct v4l2_format_info *info; + static char name[5]; + unsigned int i; + + info = v4l2_format_by_fourcc(fourcc); + if (info) { + return info->name; + } + + for (i = 0; i < 4; ++i) { + name[i] = fourcc & 0xff; + fourcc >>= 8; + } + + name[4] = '\0'; + return name; +} + +static struct { + enum v4l2_buf_type type; + bool supported; + const char *name; +} buf_types_array[] = { + { V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, 1, "Video capture mplanes", }, + { V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, 1, "Video output mplanes", }, + { V4L2_BUF_TYPE_VIDEO_CAPTURE, 1, "Video capture", }, + { V4L2_BUF_TYPE_VIDEO_OUTPUT, 1, "Video output", }, + { V4L2_BUF_TYPE_VIDEO_OVERLAY, 0, "Video overlay", }, + { V4L2_BUF_TYPE_META_CAPTURE, 0, "Meta-data capture", }, + { V4L2_BUF_TYPE_META_OUTPUT, 0, "Meta-data output", }, +}; + +static const char *v4l2_buf_type_name(enum v4l2_buf_type type) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(buf_types_array); ++i) { + if (buf_types_array[i].type == type) { + return buf_types_array[i].name; + } + } + + if (type & V4L2_BUF_TYPE_PRIVATE) { + return "Private"; + } else { + return "Unknown"; + } +} + +static const struct { + const char *name; + enum v4l2_field field; +} fields[] = { + { "any", V4L2_FIELD_ANY }, + { "none", V4L2_FIELD_NONE }, + { "top", V4L2_FIELD_TOP }, + { "bottom", V4L2_FIELD_BOTTOM }, + { "interlaced", V4L2_FIELD_INTERLACED }, + { "seq-tb", V4L2_FIELD_SEQ_TB }, + { "seq-bt", V4L2_FIELD_SEQ_BT }, + { "alternate", V4L2_FIELD_ALTERNATE }, + { "interlaced-tb", V4L2_FIELD_INTERLACED_TB }, + { "interlaced-bt", V4L2_FIELD_INTERLACED_BT }, +}; + +static const char *v4l2_field_name(enum v4l2_field field) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(fields); ++i) { + if (fields[i].field == field) { + return fields[i].name; + } + } + + return "unknown"; +} + +int v4l2_open(const gchar *devname) +{ + int fd; + + if (!devname) { + return -EINVAL; + } + + fd = open(devname, O_RDWR | O_NONBLOCK); + if (fd < 0) { + g_printerr("Error opening device %s: %s (%d).\n", devname, + g_strerror(errno), errno); + return fd; + } + + g_print("Device %s opened fd(%d).\n", devname, fd); + + return fd; +} + +int v4l2_close(int fd) +{ + int ret; + ret = close(fd); + + if (ret < 0) { + g_printerr("%s: close failed errno(%s)", __func__, g_strerror(errno)); + } + return ret; +} + +static int video_enum_frame_intervals(struct v4l2_device *dev, + __u32 pixelformat, + unsigned int width, unsigned int height, + GList **p_vid_fmt_frm_rate_l) +{ + struct v4l2_frmivalenum ival; + GList *vid_fmt_frm_rate_l = NULL; + struct video_format_frame_rates *fmt_frm_rate; + unsigned int i; + int ret = 0; + + for (i = 0; ; ++i) { + memset(&ival, 0, sizeof ival); + ival.index = i; + ival.pixel_format = pixelformat; + ival.width = width; + ival.height = height; + ret = ioctl(dev->fd, VIDIOC_ENUM_FRAMEINTERVALS, &ival); + if (ret < 0) { + if (errno == EINVAL) /* EINVAL means no more frame intervals */ + ret = 0; + else + g_printerr("%s: VIDIOC_ENUM_FRAMEINTERVALS failed %s\n" + , __func__, g_strerror(errno)); + break; + } + + /* driver sanity checks */ + if (i != ival.index) + g_printerr("Warning: driver returned wrong ival index " + "%u.\n", ival.index); + if (pixelformat != ival.pixel_format) + g_printerr("Warning: driver returned wrong ival pixel " + "format %08x.\n", ival.pixel_format); + if (width != ival.width) + g_printerr("Warning: driver returned wrong ival width " + "%u.\n", ival.width); + if (height != ival.height) + g_printerr("Warning: driver returned wrong ival height " + "%u.\n", ival.height); + + if (i != 0) { + g_print(", "); + } + + /* allocate video_format_frame */ + fmt_frm_rate = g_new0(struct video_format_frame_rates, 1); + /* keep a copy of v4l2 frmsizeenum struct */ + memcpy(&fmt_frm_rate->v4l_ival, &ival, + sizeof(struct v4l2_frmivalenum)); + vid_fmt_frm_rate_l = + g_list_append(vid_fmt_frm_rate_l, fmt_frm_rate); + + switch (ival.type) { + case V4L2_FRMIVAL_TYPE_DISCRETE: + g_debug("%u/%u", + ival.discrete.numerator, + ival.discrete.denominator); + + fmt_frm_rate->frame_rates.min = ival.discrete.denominator; + + break; + + case V4L2_FRMIVAL_TYPE_CONTINUOUS: + g_debug("%u/%u - %u/%u", + ival.stepwise.min.numerator, + ival.stepwise.min.denominator, + ival.stepwise.max.numerator, + ival.stepwise.max.denominator); + + fmt_frm_rate->frame_rates.min = ival.stepwise.min.denominator; + fmt_frm_rate->frame_rates.max = ival.stepwise.max.denominator; + fmt_frm_rate->frame_rates.step = 1; + + goto out; + + case V4L2_FRMIVAL_TYPE_STEPWISE: + g_debug("%u/%u - %u/%u (by %u/%u)", + ival.stepwise.min.numerator, + ival.stepwise.min.denominator, + ival.stepwise.max.numerator, + ival.stepwise.max.denominator, + ival.stepwise.step.numerator, + ival.stepwise.step.denominator); + + fmt_frm_rate->frame_rates.min = ival.stepwise.min.denominator; + fmt_frm_rate->frame_rates.max = ival.stepwise.max.denominator; + fmt_frm_rate->frame_rates.step = ival.stepwise.step.denominator; + + goto out; + + default: + break; + } + } + +out: + if (ret == 0) { + g_print("\n%s: Enumerated %d frame intervals\n", __func__ + , g_list_length(vid_fmt_frm_rate_l)); + g_return_val_if_fail(i == g_list_length(vid_fmt_frm_rate_l), -EINVAL); + *p_vid_fmt_frm_rate_l = vid_fmt_frm_rate_l; + } + + return ret; +} + +static int video_enum_frame_sizes(struct v4l2_device *dev, + __u32 pixelformat, GList **p_vid_fmt_frm_l) +{ + struct v4l2_frmsizeenum frame; + struct video_format_frame *vid_frame = NULL; + GList *vid_fmt_frm_l = NULL; + unsigned int i; + int ret; + + if (!dev) { + return -EINVAL; + } + + for (i = 0; ; ++i) { + memset(&frame, 0, sizeof frame); + frame.index = i; + frame.pixel_format = pixelformat; + ret = ioctl(dev->fd, VIDIOC_ENUM_FRAMESIZES, &frame); + if (ret < 0) { + if (errno == EINVAL) /* EINVAL means no more frame sizes */ + ret = 0; + else + g_printerr("%s: VIDIOC_ENUM_FRAMESIZES failed %s\n", + __func__, g_strerror(errno)); + break; + } + + /* driver sanity checks */ + if (i != frame.index) + g_printerr("Warning: driver returned wrong frame index " + "%u.\n", frame.index); + if (pixelformat != frame.pixel_format) + g_printerr("Warning: driver returned wrong frame pixel " + "format %08x.\n", frame.pixel_format); + + /* allocate video_format_frame */ + vid_frame = g_new0(struct video_format_frame, 1); + /* keep a copy of v4l2 frmsizeenum struct */ + memcpy(&vid_frame->v4l_framesize, &frame, + sizeof(struct v4l2_frmsizeenum)); + vid_fmt_frm_l = g_list_append(vid_fmt_frm_l, vid_frame); + + switch (frame.type) { + case V4L2_FRMSIZE_TYPE_DISCRETE: + g_debug("\tFrame size (D): %ux%u (", frame.discrete.width, + frame.discrete.height); + + vid_frame->frame.width.min = htole32(frame.discrete.width); + vid_frame->frame.width.max = htole32(frame.discrete.width); + vid_frame->frame.height.min = htole32(frame.discrete.height); + vid_frame->frame.height.max = htole32(frame.discrete.height); + + if (video_enum_frame_intervals(dev, frame.pixel_format, + frame.discrete.width, + frame.discrete.height, + &vid_frame->frm_rate_l) < 0) + g_printerr("%s: video_enum_frame_intervals failed!", __func__); + g_debug(")"); + break; + + case V4L2_FRMSIZE_TYPE_CONTINUOUS: + g_debug("\tFrame size (C): %ux%u - %ux%u (", + frame.stepwise.min_width, + frame.stepwise.min_height, + frame.stepwise.max_width, + frame.stepwise.max_height); + + vid_frame->frame.width.min = htole32(frame.stepwise.min_width); + vid_frame->frame.width.max = htole32(frame.stepwise.max_width); + vid_frame->frame.width.step = htole32(frame.stepwise.step_width); + vid_frame->frame.height.min = htole32(frame.stepwise.min_height); + vid_frame->frame.height.max = htole32(frame.stepwise.max_height); + vid_frame->frame.height.step = htole32(frame.stepwise.step_height); + + /* driver sanity check */ + if (frame.stepwise.step_height != 1 || + frame.stepwise.step_width != 1) { + g_printerr("Warning: invalid step for continuous framesize"); + } + + if (video_enum_frame_intervals(dev, frame.pixel_format, + frame.stepwise.max_width, + frame.stepwise.max_height, + &vid_frame->frm_rate_l) < 0) + g_printerr("%s: video_enum_frame_intervals failed!\n" + , __func__); + + g_debug(")"); + break; + + case V4L2_FRMSIZE_TYPE_STEPWISE: + g_debug("\tFrame size (S): %ux%u - %ux%u (by %ux%u) (", + frame.stepwise.min_width, + frame.stepwise.min_height, + frame.stepwise.max_width, + frame.stepwise.max_height, + frame.stepwise.step_width, + frame.stepwise.step_height); + + vid_frame->frame.width.min = htole32(frame.stepwise.min_width); + vid_frame->frame.width.max = htole32(frame.stepwise.max_width); + vid_frame->frame.width.step = htole32(frame.stepwise.step_width); + vid_frame->frame.height.min = htole32(frame.stepwise.min_height); + vid_frame->frame.height.max = htole32(frame.stepwise.max_height); + vid_frame->frame.height.step = htole32(frame.stepwise.step_height); + + if (video_enum_frame_intervals(dev, frame.pixel_format, + frame.stepwise.max_width, + frame.stepwise.max_height, + &vid_frame->frm_rate_l) < 0) + g_printerr("%s: video_enum_frame_intervals failed!\n" + , __func__); + + g_debug(")"); + break; + + default: + break; + } + } + if (ret == 0) { + g_print("%s: Enumerated %d frame sizes and %d frame intervals\n", + __func__, g_list_length(vid_fmt_frm_l), + g_list_length(vid_frame->frm_rate_l)); + + vid_frame->frame.num_rates = + htole32(g_list_length(vid_frame->frm_rate_l)); + + g_return_val_if_fail(i == g_list_length(vid_fmt_frm_l), -EINVAL); + *p_vid_fmt_frm_l = vid_fmt_frm_l; + } + + return ret; +} + +int video_send_decoder_start_cmd(struct v4l2_device *dev) +{ + int ret = 0; + struct v4l2_decoder_cmd cmd; + g_debug("%s: ", __func__); + + cmd.cmd = V4L2_DEC_CMD_START; + cmd.flags = 0; + cmd.start.speed = 1000; + + memset(&cmd, 0, sizeof cmd); + + ret = ioctl(dev->fd, VIDIOC_DECODER_CMD, &cmd); + if (ret < 0) { + g_printerr("%s: %s (%d)\n", __func__, g_strerror(errno), errno); + } + + return ret; +} + +static int video_querycap(struct v4l2_device *dev) +{ + struct v4l2_capability cap; + unsigned int caps; + bool has_video; + bool has_meta; + bool has_capture; + bool has_output; + bool has_mplane; + int ret; + + memset(&cap, 0, sizeof cap); + ret = ioctl(dev->fd, VIDIOC_QUERYCAP, &cap); + if (ret < 0) { + return 0; + } + + caps = cap.capabilities & V4L2_CAP_DEVICE_CAPS + ? cap.device_caps : cap.capabilities; + + has_video = caps & (V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_VIDEO_CAPTURE | + V4L2_CAP_VIDEO_OUTPUT_MPLANE | + V4L2_CAP_VIDEO_OUTPUT); + has_meta = caps & (V4L2_CAP_META_CAPTURE | + V4L2_CAP_META_OUTPUT); + has_capture = caps & (V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_VIDEO_CAPTURE | + V4L2_CAP_META_CAPTURE); + has_output = caps & (V4L2_CAP_VIDEO_OUTPUT_MPLANE | + V4L2_CAP_VIDEO_OUTPUT | + V4L2_CAP_META_OUTPUT); + has_mplane = caps & (V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_VIDEO_OUTPUT_MPLANE | + V4L2_CAP_VIDEO_M2M_MPLANE); + + g_print("Device `%s' on `%s' (driver '%s') " + "supports%s%s%s%s %s mplanes.\n", + cap.card, cap.bus_info, cap.driver, + has_video ? " video," : "", + has_meta ? " meta-data," : "", + has_capture ? " capture," : "", + has_output ? " output," : "", + has_mplane ? "with" : "without"); + + dev->capabilities = caps; + dev->has_mplane = has_mplane; + + return 0; +} + +void v4l2_set_device_type(struct v4l2_device *dev, enum v4l2_buf_type type, + struct v4l2_fmtdesc *fmt_desc) +{ + if (fmt_desc->flags & V4L2_FMT_FLAG_COMPRESSED) { + + switch (fmt_desc->pixelformat) { + case V4L2_PIX_FMT_H263: + case V4L2_PIX_FMT_H264: + case V4L2_PIX_FMT_H264_NO_SC: + case V4L2_PIX_FMT_H264_MVC: + case V4L2_PIX_FMT_MPEG1: + case V4L2_PIX_FMT_MPEG2: + case V4L2_PIX_FMT_MPEG4: + case V4L2_PIX_FMT_XVID: + case V4L2_PIX_FMT_VC1_ANNEX_G: + case V4L2_PIX_FMT_VC1_ANNEX_L: + case V4L2_PIX_FMT_VP8: + case V4L2_PIX_FMT_VP9: + case V4L2_PIX_FMT_HEVC: + case V4L2_PIX_FMT_FWHT: + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT) { + dev->dev_type |= STATEFUL_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + dev->dev_type |= STATEFUL_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE) { + dev->dev_type |= STATEFUL_ENCODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + dev->dev_type |= STATEFUL_ENCODER; + } + break; + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_FWHT_STATELESS: + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT) { + dev->dev_type |= STATELESS_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + dev->dev_type |= STATELESS_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE) { + dev->dev_type |= STATELESS_ENCODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + dev->dev_type |= STATELESS_ENCODER; + } + break; + default: + break; + } + } +} + +enum v4l2_buf_type get_v4l2_buf_type (enum virtio_video_queue_type queue_type, + bool has_mplane) +{ + enum v4l2_buf_type buf_type; + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + buf_type = has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + break; + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + buf_type = has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + break; + + default: + g_warning("%s: Unknown queue_type!", __func__); + } + + g_debug("%s: queue_type(0x%x) has_mplane(%d), buf_type(%s)" + , __func__, queue_type, has_mplane, v4l2_buf_type_name(buf_type)); + + return buf_type; +} + +int v4l2_free_buffers(int fd, enum v4l2_buf_type type) +{ + struct v4l2_requestbuffers reqbuf; + int ret; + g_debug("%s: v4l2_buf_type: %s: Issuing REQBUFS 0" + , __func__, v4l2_buf_type_name(type)); + + memset(&reqbuf, 0, sizeof(reqbuf)); + reqbuf.type = type; + reqbuf.count = 0; + + /*TODO must save this when creating resource on queue */ + reqbuf.memory = V4L2_MEMORY_USERPTR; + + /* + * Applications can call ioctl VIDIOC_REQBUFS again to change the number + * of buffers. Note that if any buffers are still mapped or exported via + * DMABUF, then ioctl VIDIOC_REQBUFS can only succeed if the + * V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS capability is set. Otherwise ioctl + * VIDIOC_REQBUFS will return the EBUSY error code. If + * V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS is set, then these buffers are + * orphanedand will be freed when they are unmapped or when the exported + * DMABUF fds are closed. A count value of zero frees or orphans all + * buffers, after aborting or finishing any DMA in progress, an implicit + * VIDIOC_STREAMOFF. + */ + + /* TODO support V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS */ + + ret = ioctl(fd, VIDIOC_REQBUFS, &reqbuf); + if (ret == -1) { + if (errno == EBUSY) { + g_critical("%s: EBUSY: buffers for %s still mapped or exported!\n" + , __func__, v4l2_buf_type_name(type)); + } else { + g_printerr("VIDIOC_REQBUFS failed: %s (%d)\n" + , g_strerror(errno), errno); + } + goto out; + } + g_debug("%s: VIDIOC_REQBUFS capabilities(0x%x) granted(%d)" + , __func__, reqbuf.capabilities, reqbuf.count); + +out: + return ret; +} + +int v4l2_resource_create(struct stream *s, enum v4l2_buf_type type, + enum virtio_video_mem_type mem_type, + struct resource *res) +{ + struct v4l2_requestbuffers reqbuf; + int ret; + g_debug("%s: v4l2_buf_type: %s", __func__, v4l2_buf_type_name(type)); + + memset(&reqbuf, 0, sizeof(reqbuf)); + reqbuf.type = type; + reqbuf.count = 1; + + if (is_output_queue(type)) { + reqbuf.count = s->output_bufcount + 1; + } else if (is_capture_queue(type)) { + reqbuf.count = s->capture_bufcount + 1; + } + + if (mem_type == VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES) { + reqbuf.memory = V4L2_MEMORY_USERPTR; + } else if (mem_type == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT) { + /* TODO */ + g_error("%s: VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT not implemented\n" + , __func__); + ret = -EINVAL; + goto out; + } + + ret = ioctl(s->fd, VIDIOC_REQBUFS, &reqbuf); + if (ret == -1) { + g_printerr("VIDIOC_REQBUFS failed: %s (%d)\n" + , g_strerror(errno), errno); + goto out; + } + g_debug("%s: VIDIOC_REQBUFS capabilities(0x%x) granted(%d)!" + , __func__, reqbuf.capabilities, reqbuf.count); + + if (is_output_queue(type)) { + s->output_bufcount = reqbuf.count; + res->v4l2_index = reqbuf.count - 1; + } else if (is_capture_queue(type)) { + s->capture_bufcount = reqbuf.count; + res->v4l2_index = reqbuf.count - 1; + } + + res->type = type; +out: + return ret; +} + +/* timestamp in nsecs */ +void convert_to_timeval(uint64_t timestamp, struct timeval *t) +{ + uint64_t f_nsecs; + + uint64_t nsecs; + + /* convert to seconds */ + t->tv_sec = timestamp / 1000000000; + + /* deal with fraction of a second */ + f_nsecs = t->tv_sec * 1000000000; + t->tv_usec = (timestamp - f_nsecs) / 1000; + + /* sanity check above conversion */ + nsecs = t->tv_sec * 1000000000; + nsecs += (t->tv_usec * 1000); + + if (timestamp != nsecs) { + g_critical("%s: timestamp != nsecs", __func__); + } +} + +int ioctl_streamon(struct stream *s, enum v4l2_buf_type type) +{ + int ret = 0; + ret = ioctl(s->fd, VIDIOC_STREAMON, &type); + if (ret < 0) { + g_printerr("VIDIOC_STREAMON failed: fd=(%d) buf type=%s %s (%d).\n" + , s->fd, v4l2_buf_type_name(type), g_strerror(errno), errno); + } else { + g_debug("%s: VIDIOC_STREAMON OK fd=(%d) buf type: %s" + , __func__, s->fd, v4l2_buf_type_name(type)); + if (is_output_queue(type)) { + s->output_streaming = true; + } + if (is_capture_queue(type)) { + s->capture_streaming = true; + } + } + return ret; +} + +int ioctl_streamoff(struct stream *s, enum v4l2_buf_type type) +{ + int ret = 0; + ret = ioctl(s->fd, VIDIOC_STREAMOFF, &type); + if (ret < 0) { + g_printerr("VIDIOC_STREAMOFF failed: fd=(%d) buf type=%s: %s (%d).\n" + , s->fd, v4l2_buf_type_name(type), g_strerror(errno), errno); + } else { + g_debug("%s: VIDIOC_STREAMOFF OK buf type: %s" + , __func__, v4l2_buf_type_name(type)); + + if (is_output_queue(type)) { + s->output_streaming = false; + } + if (is_capture_queue(type)) { + s->capture_streaming = false; + } + + /* + * if either queue has STREAMOFF applied, then we enter STOPPED + * Assumes that s->mutex is held by calling function + */ + s->stream_state = STREAM_STOPPED; + g_cond_signal(&s->stream_cond); + } + return ret; +} + +/* activate streaming on both queues */ +int v4l2_streamon(struct v4l2_device *dev, enum v4l2_buf_type type, + struct stream *s) +{ + int ret = 0; + bool is_mplane = video_is_mplane(type); + enum v4l2_buf_type type2; + + if (!s->subscribed_events) { + /* subscribe for SOURCE_CHANGE event */ + if (dev->sup_dyn_res_switching) { + ret = v4l2_subscribe_event(s, V4L2_EVENT_SOURCE_CHANGE, 0); + if (ret < 0) { + g_printerr("%s: V4L2_EVENT_SOURCE_CHANGE failed\n", __func__); + } + } + /* subscribe for EOS event */ + ret = v4l2_subscribe_event(s, V4L2_EVENT_EOS, 0); + if (ret < 0) { + g_printerr("%s: V4L2_EVENT_EOS failed\n", __func__); + } + s->subscribed_events = true; + } + + switch (type) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_META_OUTPUT: + if (s->output_streaming == false) { + ret |= ioctl_streamon(s, type); + } + + if (s->capture_streaming == false) { + type2 = is_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : + V4L2_BUF_TYPE_VIDEO_CAPTURE; + ret |= ioctl_streamon(s, type2); + } + break; + + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_META_CAPTURE: + if (s->capture_streaming == false) { + ret |= ioctl_streamon(s, type); + } + if (s->output_streaming == false) { + type2 = is_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE : + V4L2_BUF_TYPE_VIDEO_OUTPUT; + ret |= ioctl_streamon(s, type2); + } + break; + + default: + g_printerr("%s: unknown v4l2 buffer type!", __func__); + ret = EINVAL; + } + + if (s->stream_state != STREAM_DRAINING) { + s->stream_state = STREAM_STREAMING; + g_cond_signal(&s->stream_cond); + } + + return ret; +} + +int v4l2_streamoff(enum v4l2_buf_type type, struct stream *s) +{ + int ret = 0; + bool is_mplane = video_is_mplane(type); + enum v4l2_buf_type type2; + + switch (type) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_META_OUTPUT: + if (s->output_streaming == true) { + ret |= ioctl_streamoff(s, type); + } + + if (s->capture_streaming == true) { + type2 = is_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + ret |= ioctl_streamoff(s, type2); + } + break; + + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_META_CAPTURE: + if (s->capture_streaming == true) { + ret |= ioctl_streamoff(s, type); + } + if (s->output_streaming == true) { + type2 = is_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + ret |= ioctl_streamoff(s, type2); + } + break; + + default: + g_printerr("%s: unknown v4l2 buffer type!", __func__); + ret = EINVAL; + } + + return ret; +} + +int v4l2_subscribe_event(struct stream *s, + uint32_t event_type, uint32_t id) +{ + int ret = 0; + struct v4l2_event_subscription sub; + + memset(&sub, 0, sizeof(sub)); + sub.type = event_type; + sub.id = 0; + + if (event_type == V4L2_EVENT_SOURCE_CHANGE) { + sub.id = id; + } + + ret = ioctl(s->fd, VIDIOC_SUBSCRIBE_EVENT, &sub); + if (ret < 0) { + g_printerr("%s: VIDIOC_SUBSCRIBE_EVENT failed", __func__); + return ret; + } + + g_debug("%s event(0x%x) OK!", __func__, event_type); + + return ret; +} + +void v4l2_print_event(const struct v4l2_event *ev) +{ + g_debug("%s: %ld.%06ld: event %u, pending %u: ", __func__, + ev->timestamp.tv_sec, ev->timestamp.tv_nsec / 1000, + ev->sequence, ev->pending); + switch (ev->type) { + case V4L2_EVENT_VSYNC: + g_debug("%s: vsync\n", __func__); + break; + case V4L2_EVENT_EOS: + g_debug("%s: eos\n", __func__); + break; + case V4L2_EVENT_CTRL: + g_debug("%s: eos\n", __func__); + break; + case V4L2_EVENT_FRAME_SYNC: + g_debug("%s: frame_sync %d\n", __func__, ev->u.frame_sync.frame_sequence); + break; + case V4L2_EVENT_SOURCE_CHANGE: + g_debug("%s: source_change!: pad/input=%d changes: %x\n" + , __func__, ev->id, ev->u.src_change.changes); + break; + case V4L2_EVENT_MOTION_DET: + if (ev->u.motion_det.flags & V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ) { + g_debug("%s: motion_det frame %d, regions 0x%x\n", __func__, + ev->u.motion_det.frame_sequence, + ev->u.motion_det.region_mask); + } else { + g_debug("%s: motion_det regions 0x%x\n", __func__ + , ev->u.motion_det.region_mask); + } + break; + default: + if (ev->type >= V4L2_EVENT_PRIVATE_START) { + g_debug("unknown private event (%08x)\n", ev->type); + } else { + g_debug("unknown event (%08x)\n", ev->type); + } + break; + } +} + +int v4l2_queue_buffer(int fd, enum v4l2_buf_type type, + struct virtio_video_resource_queue *qcmd, + struct resource *res, struct stream *s, + struct v4l2_device *dev) +{ + struct v4l2_buffer vbuf; + int ret = 0; + + memset(&vbuf, 0, sizeof(vbuf)); + vbuf.index = res->v4l2_index; + + vbuf.type = type; + vbuf.field = V4L2_FIELD_NONE; + vbuf.flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + g_debug("%s: type=%s index=%d", __func__, + v4l2_buf_type_name(type), vbuf.index); + + convert_to_timeval(le64toh(qcmd->timestamp), &vbuf.timestamp); + + /* if using GUEST_PAGES queued using USERPTR mechanism */ + vbuf.memory = V4L2_MEMORY_USERPTR; + + if (video_is_mplane(type)) { + /* for mplane length field is number of elements in planes array */ + vbuf.length = res->vio_resource.num_planes; + vbuf.m.planes = g_malloc0(sizeof(struct v4l2_plane) + * res->vio_resource.num_planes); + + for (int i = 0; i < vbuf.length; i++) { + vbuf.m.planes[i].m.userptr = (unsigned long)res->iov[i].iov_base; + vbuf.m.planes[i].length = (unsigned long)res->iov[i].iov_len; + } + } else { + /* m is a union of userptr, *planes and fd */ + vbuf.m.userptr = (unsigned long)res->iov[0].iov_base; + vbuf.length = res->iov[0].iov_len; + g_debug("%s: iov_base = 0x%p", __func__, res->iov[0].iov_base); + g_debug("%s: iov_len = 0x%lx", __func__, res->iov[0].iov_len); + } + + ret = v4l2_streamon(dev, type, s); + if (ret < 0) { + g_printerr("v4l2_streamon failed (%d)", ret); + /* only print error, as v4l2_streamon() does both queues */ + } + + ret = ioctl(fd, VIDIOC_QBUF, &vbuf); + if (ret < 0) { + qcmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + g_printerr("Unable to QBUF: %s (%d).\n", g_strerror(errno), errno); + return ret; + } + + res->queued = true; + if (video_is_mplane(type)) { + g_free(vbuf.m.planes); + } + + g_debug("%s: Queued resource-id(%d) buf_type=%s v4l2_index(%d) " + "virtio_queue(0x%x)", __func__, res->vio_resource.resource_id, + v4l2_buf_type_name(type), res->v4l2_index, + res->vio_resource.queue_type); + + return ret; + +} + +int v4l2_dequeue_buffer(int fd, enum v4l2_buf_type type, + struct stream *s) +{ + struct v4l2_buffer vbuf; + int ret = 0; + struct resource *r; + struct virtio_video_resource_queue_resp resp; + struct vu_video_ctrl_command *vio_cmd; + + memset(&vbuf, 0, sizeof(vbuf)); + + vbuf.type = type; + vbuf.memory = V4L2_MEMORY_USERPTR; + + vbuf.field = V4L2_FIELD_NONE; + vbuf.flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + if (video_is_mplane(type)) { + /* for mplane length field is number of elements in planes array */ + vbuf.length = VIRTIO_VIDEO_MAX_PLANES; + vbuf.m.planes = g_malloc0(sizeof(struct v4l2_plane) + * VIRTIO_VIDEO_MAX_PLANES); + + g_debug("%s: mplane allocating planes array", __func__); + } + + ret = ioctl(fd, VIDIOC_DQBUF, &vbuf); + if (ret < 0) { + g_printerr("Unable to DQBUF: %s (%d).\n", g_strerror(errno), errno); + return ret; + } + + g_debug("%s: VIDIOC_DQBUF OK index(%d)!", __func__, vbuf.index); + + if (video_is_mplane(type)) { + g_free(vbuf.m.planes); + } + + r = find_resource_by_v4l2index(s, type, vbuf.index); + if (!r) { + g_printerr("%s: Can't find resource for dequeued buffer!", __func__); + return -EINVAL; + } + + r->queued = false; + vio_cmd = r->vio_q_cmd; + + resp.hdr.stream_id = r->stream_id; + resp.hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + resp.timestamp = htole64(r->vio_res_q.timestamp); + + /* encoder only */ + resp.size = htole32(vbuf.bytesused); + + if (vbuf.flags & V4L2_BUF_FLAG_LAST && + s->stream_state == STREAM_DRAINING) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_EOS; + s->stream_state = STREAM_STOPPED; + g_cond_signal(&s->stream_cond); + } + + if (vbuf.flags & V4L2_BUF_FLAG_KEYFRAME) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_IFRAME; + } + if (vbuf.flags & V4L2_BUF_FLAG_PFRAME) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_PFRAME; + } + if (vbuf.flags & V4L2_BUF_FLAG_BFRAME) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_PFRAME; + } + + if (vbuf.flags & V4L2_BUF_FLAG_ERROR) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_ERR; + g_critical("%s: V4L2_BUF_FLAG_ERROR\n", __func__); + } + + g_debug("%s: Send queue_buffer reply: stream_id=%d type=0x%x " + "flags=0x%x resource_id=%d t=%llx", __func__, + resp.hdr.stream_id, resp.hdr.type, resp.flags, + r->vio_resource.resource_id, resp.timestamp); + + send_ctrl_response(vio_cmd, (uint8_t *) &resp, + sizeof(struct virtio_video_resource_queue_resp)); + + vio_cmd->finished = true; + free_resource_mem(r); + + return ret; +} + +int v4l2_video_get_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel) +{ + int ret = 0; + + if (!sel) { + return -EINVAL; + } + + memset(sel, 0, sizeof(struct v4l2_selection)); + + sel->type = type; + + if (is_capture_queue(type)) { + sel->target = V4L2_SEL_TGT_COMPOSE; + } else if (is_output_queue(type)) { + sel->target = V4L2_SEL_TGT_CROP; + } + + ret = ioctl(fd, VIDIOC_G_SELECTION, sel); + if (ret < 0) { + g_printerr("Unable to get selection: %s (%d).\n", g_strerror(errno), + errno); + return ret; + } + + g_debug("%s: VIDIOC_G_SELECTION: fd=(%d) %s: left=(%d) " + "top=(%d) width=(%d) height=(%d)", + __func__, fd, v4l2_buf_type_name(type), sel->r.left, + sel->r.top, sel->r.width, sel->r.height); + + return ret; +} + +int v4l2_video_set_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel) +{ + int ret = 0; + + if (!sel) { + return -EINVAL; + } + + sel->type = type; + sel->target = V4L2_SEL_TGT_COMPOSE; + /* flags 0 - the driver can adjust the rect size freely */ + sel->flags = 0; + + ret = ioctl(fd, VIDIOC_S_SELECTION, sel); + if (ret < 0) { + g_printerr("Unable to set selection: fd=(%d) left=(%d) top=(%d)" + "width=(%d) height=(%d): %s (%d).\n", fd, sel->r.left, + sel->r.top, sel->r.width, sel->r.height, g_strerror(errno), + errno); + return ret; + } + + g_debug("%s: VIDIOC_S_SELECTION: fd=(%d) left=(%d) " + "top=(%d) width=(%d) height=(%d)", + __func__, fd, sel->r.left, sel->r.top, sel->r.width, sel->r.height); + + return ret; +} + +int v4l2_issue_cmd(int fd, uint32_t cmd, uint32_t flags) +{ + int ret = 0; + struct v4l2_decoder_cmd decoder_cmd; + memset(&decoder_cmd, 0, sizeof(struct v4l2_decoder_cmd)); + + decoder_cmd.cmd = cmd; + decoder_cmd.flags = flags; + + ret = ioctl(fd, VIDIOC_DECODER_CMD, &decoder_cmd); + if (ret < 0) { + g_printerr("%s: VIDIOC_DECODER_CMD(%d) failed fd(%d): %s: (%d).\n" + , __func__, cmd, fd, g_strerror(errno), errno); + return ret; + } + + g_debug("%s: VIDIOC_DECODER_CMD(%d) fd(%d)OK\n", __func__, cmd, fd); + + return ret; +} + +int v4l2_video_get_param(int fd, enum v4l2_buf_type type, + struct v4l2_streamparm *sparam) +{ + int ret = 0; + + if (!sparam) { + return -EINVAL; + } + + memset(sparam, 0, sizeof(struct v4l2_streamparm)); + sparam->type = type; + + ret = ioctl(fd, VIDIOC_G_PARM, sparam); + if (ret < 0) { + g_printerr("Unable to VIDIOC_G_PARAM: %s (%d).\n", g_strerror(errno), + errno); + return ret; + } + + g_debug("%s: VIDIOC_G_PARM timeperframe (%d/%d)", __func__, + sparam->parm.capture.timeperframe.numerator, + sparam->parm.capture.timeperframe.denominator); + + return ret; +} + +int v4l2_video_get_format(int fd, enum v4l2_buf_type type, + struct v4l2_format *fmt) +{ + unsigned int i; + int ret; + + if (!fmt) { + return -EINVAL; + } + + memset(fmt, 0, sizeof(struct v4l2_format)); + fmt->type = type; + + ret = ioctl(fd, VIDIOC_G_FMT, fmt); + if (ret < 0) { + g_printerr("Unable to get format: %s (%d).\n", g_strerror(errno), + errno); + return ret; + } + + if (video_is_mplane(type)) { + g_print("Video format: %s (%08x) %ux%u field %s, %u planes:\n", + v4l2_format_name(fmt->fmt.pix_mp.pixelformat), + fmt->fmt.pix_mp.pixelformat, + fmt->fmt.pix_mp.width, fmt->fmt.pix_mp.height, + v4l2_field_name(fmt->fmt.pix_mp.field), + fmt->fmt.pix_mp.num_planes); + + for (i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + g_print(" * Stride %u, buffer size %u\n", + fmt->fmt.pix_mp.plane_fmt[i].bytesperline, + fmt->fmt.pix_mp.plane_fmt[i].sizeimage); + } + } else if (video_is_meta(type)) { + g_print("Meta-data format: %s (%08x) buffer size %u\n", + v4l2_format_name(fmt->fmt.meta.dataformat), + fmt->fmt.meta.dataformat, + fmt->fmt.meta.buffersize); + } else { + g_print("Video format: %s (%08x) %ux%u (stride %u) field %s " + "buffer size %u\n", + v4l2_format_name(fmt->fmt.pix.pixelformat), + fmt->fmt.pix.pixelformat, + fmt->fmt.pix.width, fmt->fmt.pix.height, + fmt->fmt.pix.bytesperline, + v4l2_field_name(fmt->fmt.pix_mp.field), + fmt->fmt.pix.sizeimage); + } + + return 0; +} + +int v4l2_video_get_control(int fd , uint32_t control, int32_t *value) +{ + int ret = 0; + struct v4l2_control ctrl; + + g_debug("%s:%d", __func__, __LINE__); + + ctrl.id = control; + + ret = ioctl(fd, VIDIOC_G_CTRL, &ctrl); + if (ret < 0) { + g_printerr("Unable to get control: %s (%d).\n", g_strerror(errno), + errno); + return ret; + } + + *value = ctrl.value; + g_debug("%s: ctrl=0x%x value=0x%x", __func__, control, *value); + + return ret; +} + +int v4l2_video_set_format(int fd, enum v4l2_buf_type type, + struct virtio_video_params *p) +{ + struct v4l2_format fmt; + int ret = 0; + unsigned int i; + uint32_t pixfmt; + + if (!p) { + return -EINVAL; + } + + memset(&fmt, 0, sizeof fmt); + fmt.type = type; + pixfmt = virtio_video_format_to_v4l2(le32toh(p->format)); + + if (video_is_mplane(type)) { + fmt.fmt.pix_mp.width = le32toh(p->frame_width); + fmt.fmt.pix_mp.height = le32toh(p->frame_height); + fmt.fmt.pix_mp.pixelformat = pixfmt; + /* + * V4L2_FIELD_NONE - matches what Linux frontend driver does in + * virtio_video_format_from_info() + */ + fmt.fmt.pix_mp.field = V4L2_FIELD_NONE; + /*fmt.fmt.pix_mp.num_planes = info->n_planes;*/ + fmt.fmt.pix_mp.num_planes = le32toh(p->num_planes); + fmt.fmt.pix_mp.flags = 0; + + for (i = 0; i < le32toh(p->num_planes); i++) { + fmt.fmt.pix_mp.plane_fmt[i].bytesperline = + le32toh(p->plane_formats[i].stride); + fmt.fmt.pix_mp.plane_fmt[i].sizeimage = + le32toh(p->plane_formats[i].plane_size); + } + } else if (video_is_splane(type)) { + fmt.fmt.pix.width = le32toh(p->frame_width); + fmt.fmt.pix.height = le32toh(p->frame_height); + fmt.fmt.pix.pixelformat = pixfmt; + fmt.fmt.pix.field = V4L2_FIELD_NONE; + fmt.fmt.pix.bytesperline = le32toh(p->plane_formats[0].stride); + fmt.fmt.pix.sizeimage = le32toh(p->plane_formats[0].plane_size); + fmt.fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; + fmt.fmt.pix.flags = 0; + } + + ret = ioctl(fd, VIDIOC_S_FMT, &fmt); + if (ret < 0) { + g_printerr("Unable to set format: %s (%d).\n", g_strerror(errno), + errno); + } + return ret; +} + +int v4l2_set_pixel_format(int fd, enum v4l2_buf_type buf_type, + uint32_t pixelformat) +{ + int ret = 0; + struct v4l2_format cur_fmt; + + g_debug("%s: buf_type=0x%x pixelformat=0x%x", __func__, + buf_type, pixelformat); + + /* get the currently set format */ + ret = v4l2_video_get_format(fd, buf_type, &cur_fmt); + if (ret < 0) { + g_printerr("%s: v4l2_video_get_format() failed\n", __func__); + return ret; + } + + /* keep defaults and set correct pixel format */ + if (video_is_mplane(cur_fmt.type)) { + g_print("%s: Format is mplane\n", __func__); + cur_fmt.fmt.pix_mp.pixelformat = pixelformat; + } else if (video_is_splane(cur_fmt.type)) { + g_print("%s: Format is splane\n", __func__); + cur_fmt.fmt.pix.pixelformat = pixelformat; + } + + ret = ioctl(fd, VIDIOC_S_FMT, &cur_fmt); + if (ret < 0) { + g_printerr("Unable to set format: %s (%d).\n", g_strerror(errno), + errno); + } + + return ret; +} + +int video_enum_formats(struct v4l2_device *dev, enum v4l2_buf_type type, + GList **p_fmt_list, bool only_enum_fmt) +{ + struct v4l2_fmtdesc fmt; + struct video_format *vid_fmt = NULL; + GList *fmt_list = NULL; + unsigned int index; + int ret = 0; + + if (!dev) { + return -EINVAL; + } + + for (index = 0; ; ++index) { + memset(&fmt, 0, sizeof fmt); + fmt.index = index; + fmt.type = type; + ret = ioctl(dev->fd, VIDIOC_ENUM_FMT, &fmt); + + if (ret < 0) { + if (errno == EINVAL) { + ret = 0; + } else{ + g_printerr("%s: VIDIOC_ENUM_FMT failed %s\n", __func__, + g_strerror(errno)); + } + break; + } + + /* do some driver sanity checks */ + if (index != fmt.index) { + g_warning("v4l2 driver modified index %u.\n", fmt.index); + } + if (type != fmt.type) { + g_warning("v4l2 driver modified type %u.\n", fmt.type); + } + g_debug("\tFormat %u: %s (%08x)", index, + v4l2_format_name(fmt.pixelformat), fmt.pixelformat); + g_debug("\tType: %s (%u)", v4l2_buf_type_name(fmt.type), + fmt.type); + g_debug("\tName: %.32s", fmt.description); + g_debug("\tFlags: 0x%x", fmt.flags); + + if (fmt.flags & V4L2_FMT_FLAG_DYN_RESOLUTION && + fmt.flags & V4L2_FMT_FLAG_COMPRESSED) { + g_print("dynamic resolution switching supported\n"); + dev->sup_dyn_res_switching = true; + } + + /* test if pixelformat converts to virtio */ + if (!virtio_video_v4l2_format_to_virtio(fmt.pixelformat)) { + g_info("Skipping Format %s (%08x) - no virtio-video equivalent" + , v4l2_format_name(fmt.pixelformat), fmt.pixelformat); + continue; + } + + /* allocate video_format struct */ + vid_fmt = g_new0(struct video_format, 1); + + /* keep a copy of v4l2 struct */ + memcpy(&vid_fmt->fmt, &fmt, sizeof(struct v4l2_fmtdesc)); + + /* add it to linked list */ + fmt_list = g_list_append(fmt_list, vid_fmt); + + if (!only_enum_fmt) { + /* pass video_format to enum_frame_sizes */ + ret = video_enum_frame_sizes(dev, fmt.pixelformat, + &vid_fmt->vid_fmt_frm_l); + if (ret < 0) { + g_printerr("video_enum_frame_sizes failed\n"); + } + + /* convert to virtio format */ + v4l2_to_virtio_fmtdesc(dev, vid_fmt, type); + } + + /* determine type of v4l2 device */ + v4l2_set_device_type(dev, type, &fmt); + } + + if (ret == 0) { + g_print("%s: Enumerated %d formats on v4l2 %s queue", __func__ + , index, v4l2_buf_type_name(type)); + g_print(" %d formats are representable by virtio-video\n" + , g_list_length(fmt_list)); + if (!only_enum_fmt) + g_print("%s: Enumerated %d frame sizes\n", __func__, + g_list_length(vid_fmt->vid_fmt_frm_l)); + + *p_fmt_list = fmt_list; + } + + return ret; +} + +void video_free_frame_intervals(GList *frm_intervals_l) +{ + GList *l; + struct video_format_frame_rates *vid_fmt_frm_rate; + for (l = frm_intervals_l; l != NULL; l = l->next) { + vid_fmt_frm_rate = l->data; + g_free(vid_fmt_frm_rate); + } +} + +void video_free_frame_sizes(GList *frm_sz_l) +{ + GList *l; + struct video_format_frame *vid_frame; + for (l = frm_sz_l; l != NULL; l = l->next) { + vid_frame = l->data; + if (vid_frame->frm_rate_l) { + video_free_frame_intervals(vid_frame->frm_rate_l); + } + g_free(vid_frame); + } +} + +void video_free_formats(GList **fmt_l) +{ + GList *l; + struct video_format *vid_fmt; + + for (l = *fmt_l; l != NULL; l = l->next) { + vid_fmt = l->data; + if (vid_fmt->vid_fmt_frm_l) { + video_free_frame_sizes(vid_fmt->vid_fmt_frm_l); + } + + g_free(vid_fmt); + } +} + + +static GByteArray *iterate_frame_rate_list(GByteArray *resp, GList *frm_rate_l) +{ + struct video_format_frame_rates *vid_fmt_frm_rate; + + /* iterate frame_rate list */ + for (; frm_rate_l != NULL; frm_rate_l = frm_rate_l->next) { + vid_fmt_frm_rate = frm_rate_l->data; + + resp = g_byte_array_append(resp, + (guint8 *) &vid_fmt_frm_rate->frame_rates, + sizeof(struct virtio_video_format_range)); + } + return resp; +} + +static GByteArray *iterate_format_frame_list(GByteArray *resp, GList *fmt_frm_l) +{ + struct video_format_frame *vid_fmt_frm; + GList *frm_rate_l = NULL; + + /* iterate format_frame list */ + for (; fmt_frm_l != NULL; fmt_frm_l = fmt_frm_l->next) { + vid_fmt_frm = fmt_frm_l->data; + + if (!vid_fmt_frm->frm_rate_l) { + vid_fmt_frm->frame.num_rates = htole32(0); + } else { + frm_rate_l = vid_fmt_frm->frm_rate_l; + vid_fmt_frm->frame.num_rates = htole32(g_list_length(frm_rate_l)); + + } + + g_debug("%s: num_rates(%d)", __func__, + le32toh(vid_fmt_frm->frame.num_rates)); + + resp = g_byte_array_append(resp, + (guint8 *) &vid_fmt_frm->frame, + sizeof(struct virtio_video_format_frame)); + + if (frm_rate_l) { + resp = iterate_frame_rate_list(resp, frm_rate_l); + } + } + + return resp; +} + +static GByteArray *iterate_format_desc_list(GByteArray *resp, GList *fmt_desc_l) +{ + struct video_format *vid_fmt; + GList *fmt_frm_l = NULL; + + for (; fmt_desc_l != NULL; fmt_desc_l = fmt_desc_l->next) { + vid_fmt = fmt_desc_l->data; + + /* does video_format have a list of format_frame? */ + if (!vid_fmt->vid_fmt_frm_l) { + vid_fmt->desc.num_frames = htole32(0); + } else { + fmt_frm_l = vid_fmt->vid_fmt_frm_l; + vid_fmt->desc.num_frames = htole32(g_list_length(fmt_frm_l)); + } + + g_debug("%s: num_frames(%d)", __func__, + le32toh(vid_fmt->desc.num_frames)); + + resp = g_byte_array_append(resp, + (guint8 *) &vid_fmt->desc, + sizeof(struct virtio_video_format_desc)); + + if (fmt_frm_l) { + resp = iterate_format_frame_list(resp, fmt_frm_l); + } + } + + return resp; +} + +GByteArray *create_query_cap_resp(struct virtio_video_query_capability *qcmd, + GList **fmt_l, GByteArray *resp) +{ + GList *fmt_desc_l; + struct virtio_video_query_capability_resp cap_resp; + + fmt_desc_l = *fmt_l; + + cap_resp.hdr.type = VIRTIO_VIDEO_RESP_OK_QUERY_CAPABILITY; + cap_resp.hdr.stream_id = qcmd->hdr.stream_id; + cap_resp.num_descs = htole32(g_list_length(fmt_desc_l)); + + assert(le32toh(cap_resp.num_descs) < MAX_FMT_DESCS); + + resp = g_byte_array_append(resp, (guint8 *) &cap_resp, sizeof(cap_resp)); + resp = iterate_format_desc_list(resp, fmt_desc_l); + + return resp; +} + +void v4l2_backend_free(struct v4l2_device *dev) +{ + if (dev && dev->opened) { + close(dev->fd); + } + g_free(dev); +} + +struct v4l2_device *v4l2_backend_init(const gchar *devname) +{ + struct v4l2_device *dev; + int ret = 0; + GList *vid_output_fmt_l = NULL; + GList *vid_capture_fmt_l = NULL; + enum v4l2_buf_type buf_type; + + if (!devname) { + return NULL; + } + + dev = g_malloc0(sizeof(struct v4l2_device)); + + /* open the device */ + dev->fd = v4l2_open(devname); + if (dev->fd < 0) { + g_printerr("v4l2_open() failed!\n"); + goto err; + } + + dev->opened = 1; + dev->devname = devname; + + ret = video_querycap(dev); + + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + /* enumerate coded formats on OUTPUT */ + ret = video_enum_formats(dev, buf_type, + &vid_output_fmt_l, true); + if (ret < 0) { + g_printerr("video_enum_formats() failed OUTPUT\n"); + goto err; + } + + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + + /* enumerate coded formats on CAPTURE */ + ret = video_enum_formats(dev, buf_type, + &vid_capture_fmt_l, true); + if (ret < 0) { + g_printerr("video_enum_formats() failed CAPTURE\n"); + goto err2; + } + + if (dev->dev_type & STATEFUL_ENCODER) + g_print("%s: %s is a stateful encoder (0x%x)!\n", __func__, + devname, dev->dev_type); + + if (dev->dev_type & STATEFUL_DECODER) + g_print("%s: %s is a stateful decoder (0x%x)!\n", __func__, + devname, dev->dev_type); + + video_free_formats(&vid_output_fmt_l); + video_free_formats(&vid_capture_fmt_l); + + if (!(dev->dev_type & STATEFUL_ENCODER || + dev->dev_type & STATEFUL_DECODER)) { + g_printerr("v4l2 device not supported! v4l2 backend only supports " + "stateful codec devices currently(%d)!\n", dev->dev_type); + goto err3; + } + + g_debug("%s: success!\n", __func__); + return dev; + +err3: + video_free_formats(&vid_capture_fmt_l); +err2: + video_free_formats(&vid_output_fmt_l); +err: + v4l2_backend_free(dev); + return NULL; +} diff --git a/tools/vhost-user-video/v4l2_backend.h b/tools/vhost-user-video/v4l2_backend.h new file mode 100644 index 0000000000..e3617256b3 --- /dev/null +++ b/tools/vhost-user-video/v4l2_backend.h @@ -0,0 +1,99 @@ +/* + * Virtio vhost-user VIDEO Device + * + * Copyright Linaro 2021 + * + * Authors: * Peter Griffin + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef V4L2_BACKEND_H +#define V4L2_BACKEND_H + +#include "standard-headers/linux/virtio_video.h" +#include "virtio_video_helpers.h" + +#define ARRAY_SIZE(a) (sizeof(a) / sizeof((a)[0])) + +#define MAX_CAPS_LEN 4096 +#define MAX_FMT_DESCS 64 + +#define STATEFUL_ENCODER (1 << 0) +#define STATEFUL_DECODER (1 << 1) +#define STATELESS_ENCODER (1 << 2) +#define STATELESS_DECODER (1 << 3) + +/* Function protoypes */ + +struct v4l2_device *v4l2_backend_init(const gchar *devname); +void v4l2_backend_free(struct v4l2_device *dev); + +GByteArray *create_query_cap_resp(struct virtio_video_query_capability *qcmd, + GList **fmt_l, GByteArray *querycapresp); +enum v4l2_buf_type get_v4l2_buf_type (enum virtio_video_queue_type queue_type, + bool has_mplane); + +void v4l2_set_device_type(struct v4l2_device *dev, enum v4l2_buf_type type, + struct v4l2_fmtdesc *fmt_desc); +int v4l2_video_get_format(int fd, enum v4l2_buf_type type, + struct v4l2_format *fmt); + +int v4l2_video_set_format(int fd, enum v4l2_buf_type type, + struct virtio_video_params *p); + +int v4l2_video_get_control(int fd, uint32_t control, int32_t *value); + +int v4l2_queue_buffer(int fd, enum v4l2_buf_type type, + struct virtio_video_resource_queue *qcmd, + struct resource *res, struct stream *s, + struct v4l2_device *dev); + +int v4l2_dequeue_buffer(int fd, enum v4l2_buf_type type, + struct stream *s); + +int v4l2_dequeue_event(struct v4l2_device *dev); + +int v4l2_set_pixel_format(int fd, enum v4l2_buf_type buf_type, + uint32_t pixelformat); +int v4l2_release_buffers(int fd, enum v4l2_buf_type type); +int v4l2_resource_create(struct stream *s, enum v4l2_buf_type type, + enum virtio_video_mem_type mem_type, + struct resource *res); +int v4l2_subscribe_event(struct stream *s, + uint32_t event_type, uint32_t id); + +int v4l2_video_get_param(int fd, enum v4l2_buf_type type, + struct v4l2_streamparm *param); + +int v4l2_video_get_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel); + +int v4l2_video_set_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel); + +int video_send_decoder_start_cmd(struct v4l2_device *dev); +void video_free_frame_intervals(GList *frm_intervals_l); +void video_free_frame_sizes(GList *frm_sz_l); +int video_enum_formats(struct v4l2_device *dev, enum v4l2_buf_type type, + GList **p_fmt_list, bool only_enum_fmt); +void video_free_formats(GList **fmt_l); +bool video_is_mplane(enum v4l2_buf_type type); +bool video_is_splane(enum v4l2_buf_type type); +bool video_is_meta(enum v4l2_buf_type type); +bool is_capture_queue(enum v4l2_buf_type type); +bool is_output_queue(enum v4l2_buf_type type); +int ioctl_streamon(struct stream *s, enum v4l2_buf_type type); +int ioctl_streamoff(struct stream *s, enum v4l2_buf_type type); +int v4l2_streamon(struct v4l2_device *dev, enum v4l2_buf_type type, + struct stream *s); +int v4l2_streamoff(enum v4l2_buf_type type, struct stream *s); +void v4l2_print_event(const struct v4l2_event *ev); +int v4l2_open(const gchar *devname); +int v4l2_close(int fd); +int v4l2_free_buffers(int fd, enum v4l2_buf_type type); +void convert_to_timeval(uint64_t timestamp, struct timeval *t); +int v4l2_issue_cmd(int fd, uint32_t cmd, uint32_t flags); + +#endif diff --git a/tools/vhost-user-video/virtio_video_helpers.c b/tools/vhost-user-video/virtio_video_helpers.c new file mode 100644 index 0000000000..71353804ea --- /dev/null +++ b/tools/vhost-user-video/virtio_video_helpers.c @@ -0,0 +1,462 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * virtio-video helpers + * + * Copyright Linaro 2021 + * Copyright 2019 OpenSynergy GmbH. + * + * Authors: + * Peter Griffin + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include "standard-headers/linux/virtio_video.h" +#include +#include "v4l2_backend.h" +#include "virtio_video_helpers.h" + +struct virtio_video_convert_table { + uint32_t virtio_value; + uint32_t v4l2_value; +}; + +#define ARRAY_SIZE(a) (sizeof(a) / sizeof((a)[0])) + +static struct virtio_video_convert_table level_table[] = { + { VIRTIO_VIDEO_LEVEL_H264_1_0, V4L2_MPEG_VIDEO_H264_LEVEL_1_0 }, + { VIRTIO_VIDEO_LEVEL_H264_1_1, V4L2_MPEG_VIDEO_H264_LEVEL_1_1 }, + { VIRTIO_VIDEO_LEVEL_H264_1_2, V4L2_MPEG_VIDEO_H264_LEVEL_1_2 }, + { VIRTIO_VIDEO_LEVEL_H264_1_3, V4L2_MPEG_VIDEO_H264_LEVEL_1_3 }, + { VIRTIO_VIDEO_LEVEL_H264_2_0, V4L2_MPEG_VIDEO_H264_LEVEL_2_0 }, + { VIRTIO_VIDEO_LEVEL_H264_2_1, V4L2_MPEG_VIDEO_H264_LEVEL_2_1 }, + { VIRTIO_VIDEO_LEVEL_H264_2_2, V4L2_MPEG_VIDEO_H264_LEVEL_2_2 }, + { VIRTIO_VIDEO_LEVEL_H264_3_0, V4L2_MPEG_VIDEO_H264_LEVEL_3_0 }, + { VIRTIO_VIDEO_LEVEL_H264_3_1, V4L2_MPEG_VIDEO_H264_LEVEL_3_1 }, + { VIRTIO_VIDEO_LEVEL_H264_3_2, V4L2_MPEG_VIDEO_H264_LEVEL_3_2 }, + { VIRTIO_VIDEO_LEVEL_H264_4_0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0 }, + { VIRTIO_VIDEO_LEVEL_H264_4_1, V4L2_MPEG_VIDEO_H264_LEVEL_4_1 }, + { VIRTIO_VIDEO_LEVEL_H264_4_2, V4L2_MPEG_VIDEO_H264_LEVEL_4_2 }, + { VIRTIO_VIDEO_LEVEL_H264_5_0, V4L2_MPEG_VIDEO_H264_LEVEL_5_0 }, + { VIRTIO_VIDEO_LEVEL_H264_5_1, V4L2_MPEG_VIDEO_H264_LEVEL_5_1 }, + { 0 }, +}; + +uint32_t virtio_video_level_to_v4l2(uint32_t level) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(level_table); idx++) { + if (level_table[idx].virtio_value == level) { + return level_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_level_to_virtio(uint32_t v4l2_level) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(level_table); idx++) { + if (level_table[idx].v4l2_value == v4l2_level) { + return level_table[idx].virtio_value; + } + } + + return 0; +} + +static struct virtio_video_convert_table profile_table[] = { + { VIRTIO_VIDEO_PROFILE_H264_BASELINE, + V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE }, + { VIRTIO_VIDEO_PROFILE_H264_MAIN, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN }, + { VIRTIO_VIDEO_PROFILE_H264_EXTENDED, + V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED }, + { VIRTIO_VIDEO_PROFILE_H264_HIGH, V4L2_MPEG_VIDEO_H264_PROFILE_HIGH }, + { VIRTIO_VIDEO_PROFILE_H264_HIGH10PROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_10 }, + { VIRTIO_VIDEO_PROFILE_H264_HIGH422PROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422}, + { VIRTIO_VIDEO_PROFILE_H264_HIGH444PREDICTIVEPROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_444_PREDICTIVE }, + { VIRTIO_VIDEO_PROFILE_H264_SCALABLEBASELINE, + V4L2_MPEG_VIDEO_H264_PROFILE_SCALABLE_BASELINE }, + { VIRTIO_VIDEO_PROFILE_H264_SCALABLEHIGH, + V4L2_MPEG_VIDEO_H264_PROFILE_SCALABLE_HIGH }, + { VIRTIO_VIDEO_PROFILE_H264_STEREOHIGH, + V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH }, + { VIRTIO_VIDEO_PROFILE_H264_MULTIVIEWHIGH, + V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH }, + { 0 }, +}; + +uint32_t virtio_video_profile_to_v4l2(uint32_t profile) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(profile_table); idx++) { + if (profile_table[idx].virtio_value == profile) { + return profile_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_profile_to_virtio(uint32_t v4l2_profile) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(profile_table); idx++) { + if (profile_table[idx].v4l2_value == v4l2_profile) { + return profile_table[idx].virtio_value; + } + } + + return 0; +} + + +static struct virtio_video_convert_table format_table[] = { + { VIRTIO_VIDEO_FORMAT_ARGB8888, V4L2_PIX_FMT_ARGB32 }, + { VIRTIO_VIDEO_FORMAT_BGRA8888, V4L2_PIX_FMT_ABGR32 }, + { VIRTIO_VIDEO_FORMAT_NV12, V4L2_PIX_FMT_NV12 }, + { VIRTIO_VIDEO_FORMAT_YUV420, V4L2_PIX_FMT_YUV420 }, + { VIRTIO_VIDEO_FORMAT_YVU420, V4L2_PIX_FMT_YVU420 }, + { VIRTIO_VIDEO_FORMAT_MPEG2, V4L2_PIX_FMT_MPEG2 }, + { VIRTIO_VIDEO_FORMAT_MPEG4, V4L2_PIX_FMT_MPEG4 }, + { VIRTIO_VIDEO_FORMAT_H264, V4L2_PIX_FMT_H264 }, + { VIRTIO_VIDEO_FORMAT_HEVC, V4L2_PIX_FMT_HEVC }, + { VIRTIO_VIDEO_FORMAT_VP8, V4L2_PIX_FMT_VP8 }, + { VIRTIO_VIDEO_FORMAT_VP9, V4L2_PIX_FMT_VP9 }, + { VIRTIO_VIDEO_FORMAT_FWHT, V4L2_PIX_FMT_FWHT }, + { 0 }, +}; + +uint32_t virtio_video_format_to_v4l2(uint32_t format) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(format_table); idx++) { + if (format_table[idx].virtio_value == format) { + return format_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_format_to_virtio(uint32_t v4l2_format) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(format_table); idx++) { + if (format_table[idx].v4l2_value == v4l2_format) { + return format_table[idx].virtio_value; + } + } + + return 0; +} + +/* + * TODO FIXME PROFILE and LEVEL seem wrong here as tied to H264 codec. + * V4L2_CID_MPEG_VIDEO_VP9_PROFILE + * e.g. https://elixir.bootlin.com/linux/v5.12.1/source/ + * include/uapi/linux/v4l2-controls.h#L669 + */ +static struct virtio_video_convert_table control_table[] = { + { VIRTIO_VIDEO_CONTROL_BITRATE, V4L2_CID_MPEG_VIDEO_BITRATE }, + { VIRTIO_VIDEO_CONTROL_PROFILE, V4L2_CID_MPEG_VIDEO_H264_PROFILE }, + { VIRTIO_VIDEO_CONTROL_LEVEL, V4L2_CID_MPEG_VIDEO_H264_LEVEL }, + { VIRTIO_VIDEO_CONTROL_FORCE_KEYFRAME, + V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME }, + { 0 }, +}; + +uint32_t virtio_video_control_to_v4l2(uint32_t control) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(control_table); idx++) { + if (control_table[idx].virtio_value == control) { + return control_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_control_to_virtio(uint32_t v4l2_control) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(control_table); idx++) { + if (control_table[idx].v4l2_value == v4l2_control) { + return control_table[idx].virtio_value; + } + } + + return 0; +} + +/* new helper functions (not from Linux frontend driver) */ + +const char *vio_queue_name(enum virtio_video_queue_type queue) +{ + if (queue == VIRTIO_VIDEO_QUEUE_TYPE_INPUT) { + return "Queue Input"; + } + if (queue == VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT) { + return "Queue Output"; + } + + return "Queue type unknown"; +} + + +__le64 virtio_fmtdesc_generate_mask(GList **p_list) +{ + uint64_t mask = 0; + unsigned int bit = 0; + GList *l; + + for (l = *p_list; l != NULL; l = l->next) { + mask |= (1 << bit); + bit++; + } + g_debug("%s: mask(0x%lx)\n", __func__, mask); + + return mask; +} + +/* vio_codedformat endian swapped by upper level */ + +int v4l2_stream_create(struct v4l2_device *dev, uint32_t vio_codedformat, + struct stream *s) +{ + enum v4l2_buf_type buf_type; + uint32_t v4l2_pixformat; + int ret; + + /* buf type for coded format depends on device type */ + if (dev->dev_type & STATEFUL_DECODER) { + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + } else if (dev->dev_type & STATEFUL_ENCODER) { + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : + V4L2_BUF_TYPE_VIDEO_CAPTURE; + } else { + g_critical("Unknown device type %d!", dev->dev_type); + return -EINVAL; + } + + s->fd = v4l2_open(dev->devname); + if (s->fd < 0) { + g_printerr("Error opening device %s: %s (%d).\n", dev->devname, + g_strerror(errno), errno); + } + + v4l2_pixformat = virtio_video_format_to_v4l2(vio_codedformat); + if (v4l2_pixformat == 0) { + g_error("%s: virtio to v4l2 format translation failed!", __func__); + ret = -EINVAL; + return ret; + } + + /* set the requested coded format */ + ret = v4l2_set_pixel_format(s->fd, buf_type, v4l2_pixformat); + if (ret < 0) { + g_printerr("%s: v4l2_video_set_pixel_format() failed", __func__); + } + + return ret; +} + +void v4l2_to_virtio_fmtdesc(struct v4l2_device *dev, + struct video_format *vid_fmt, + enum v4l2_buf_type type) +{ + struct v4l2_fmtdesc *v4l2_fmtdsc = &vid_fmt->fmt; + struct virtio_video_format_desc *virtio_fmtdesc = &vid_fmt->desc; + enum v4l2_buf_type buftype; + int ret; + + if (!vid_fmt) { + return; + } + + virtio_fmtdesc->format = + htole32(virtio_video_v4l2_format_to_virtio(v4l2_fmtdsc->pixelformat)); + + /* + * To generate the mask we need to check the FORMAT is already set. + * before we enumerate the other queue to generate the mask + */ + + ret = v4l2_set_pixel_format(dev->fd, type, vid_fmt->fmt.pixelformat); + if (ret < 0) { + g_printerr("%s: v4l2_video_get_format() failed\n", __func__); + } + + /* enumerate formats on the other queue now the format is set */ + GList *vid_fmts_l = NULL; + + if (is_output_queue(type)) { + buftype = video_is_mplane(type) ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + } + + if (is_capture_queue(type)) { + buftype = video_is_mplane(type) ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE : + V4L2_BUF_TYPE_VIDEO_OUTPUT; + } + + ret = video_enum_formats(dev, buftype, &vid_fmts_l, true); + + /* + * generate the capability mask. bitset represents the supported + * combinations of input and output formats. + */ + + virtio_fmtdesc->mask = htole64(virtio_fmtdesc_generate_mask(&vid_fmts_l)); + + virtio_fmtdesc->planes_layout = + htole32(VIRTIO_VIDEO_PLANES_LAYOUT_SINGLE_BUFFER); + + /* TODO need to set plane_align */ + if ((!v4l2_fmtdsc->flags & V4L2_FMT_FLAG_COMPRESSED) && + (le32toh(virtio_fmtdesc->planes_layout) & + VIRTIO_VIDEO_PLANES_LAYOUT_SINGLE_BUFFER)) { + g_critical("%s: TODO need to set plane_align field", __func__); + } + + virtio_fmtdesc->num_frames = htole32(g_list_length(vid_fmt->vid_fmt_frm_l)); + + video_free_formats(&vid_fmts_l); +} + +void v4l2_to_virtio_video_params(struct v4l2_device *dev, + struct v4l2_format *fmt, + struct v4l2_selection *sel, + struct virtio_video_get_params_resp *resp) +{ + struct virtio_video_params *vid_params = &resp->params; + int i; + + /* min/max_buffers default (taken from crosvm) */ + vid_params->min_buffers = htole32(1); + vid_params->max_buffers = htole32(32); + + if (video_is_mplane(fmt->type)) { + + vid_params->format = + virtio_video_v4l2_format_to_virtio(fmt->fmt.pix.pixelformat); + vid_params->frame_width = htole32(fmt->fmt.pix_mp.width); + vid_params->frame_height = htole32(fmt->fmt.pix_mp.height); + + vid_params->num_planes = htole32(fmt->fmt.pix_mp.num_planes); + + for (i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + vid_params->plane_formats[i].stride = \ + htole32(fmt->fmt.pix_mp.plane_fmt[i].bytesperline); + + vid_params->plane_formats[i].plane_size = \ + htole32(fmt->fmt.pix_mp.plane_fmt[i].sizeimage); + + g_debug(" ** Stride %u, buffer size %u\n", + fmt->fmt.pix_mp.plane_fmt[i].bytesperline, + fmt->fmt.pix_mp.plane_fmt[i].sizeimage); + } + } else if (video_is_splane(fmt->type)) { + + vid_params->format = + virtio_video_v4l2_format_to_virtio(fmt->fmt.pix.pixelformat); + vid_params->frame_width = htole32(fmt->fmt.pix.width); + vid_params->frame_height = htole32(fmt->fmt.pix.height); + vid_params->num_planes = htole32(1); + + vid_params->plane_formats[0].stride = \ + htole32(fmt->fmt.pix.bytesperline); + + vid_params->plane_formats[0].plane_size = \ + htole32(fmt->fmt.pix.sizeimage); + + } + + /* cropping rectangle */ + if (is_capture_queue(fmt->type)) { + vid_params->crop.left = htole32(sel->r.left); + vid_params->crop.top = htole32(sel->r.top); + vid_params->crop.width = htole32(sel->r.width); + vid_params->crop.height = htole32(sel->r.height); + + g_debug("%s: crop: left=(%d) top=(%d) width=(%d) height=(%d)" + , __func__, sel->r.left, sel->r.top, sel->r.width, + sel->r.height); + } + + /* TODO frame_rate field for encoder */ +} + +void v4l2_to_virtio_event(struct v4l2_event *ev, + struct virtio_video_event *vio_ev) +{ + g_debug("%s:%d", __func__, __LINE__); + g_debug("%ld.%06ld: event %u, pending %u: ", + ev->timestamp.tv_sec, ev->timestamp.tv_nsec / 1000, + ev->sequence, ev->pending); + + switch (ev->type) { + case V4L2_EVENT_VSYNC: + g_debug("vsync\n"); + break; + case V4L2_EVENT_EOS: + g_debug("eos\n"); + break; + case V4L2_EVENT_CTRL: + g_debug("eos\n"); + break; + case V4L2_EVENT_FRAME_SYNC: + g_debug("frame_sync %d\n", ev->u.frame_sync.frame_sequence); + break; + case V4L2_EVENT_SOURCE_CHANGE: + g_debug("source_change!: pad/input=%d changes: %x\n" + , ev->id, ev->u.src_change.changes); + + vio_ev->event_type = + htole32(VIRTIO_VIDEO_EVENT_DECODER_RESOLUTION_CHANGED); + /* TODO need proper mapping from v4l2 streamid to virtio streamid */ + vio_ev->stream_id = htole32(ev->id) + 1; + break; + case V4L2_EVENT_MOTION_DET: + if (ev->u.motion_det.flags & V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ) { + g_debug("motion_det frame %d, regions 0x%x\n", + ev->u.motion_det.frame_sequence, + ev->u.motion_det.region_mask); + } else { + g_debug("motion_det regions 0x%x\n", ev->u.motion_det.region_mask); + } + break; + default: + if (ev->type >= V4L2_EVENT_PRIVATE_START) { + g_debug("unknown private event (%08x)\n", ev->type); + } else { + g_debug("unknown event (%08x)\n", ev->type); + } + break; + } +} diff --git a/tools/vhost-user-video/virtio_video_helpers.h b/tools/vhost-user-video/virtio_video_helpers.h new file mode 100644 index 0000000000..9c46f4105a --- /dev/null +++ b/tools/vhost-user-video/virtio_video_helpers.h @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * virtio-video helpers + * + * Copyright Linaro 2021 + * + * Authors: + * Peter Griffin + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef VIRTIO_VIDEO_HELPERS_H +#define VIRTIO_VIDEO_HELPERS_H + +#include +#include "standard-headers/linux/virtio_video.h" +#include +#include "libvhost-user-glib.h" +#include "libvhost-user.h" + +/* + * Structure to track internal state of VIDEO Device + */ + +typedef struct VuVideo { + VugDev dev; + struct virtio_video_config virtio_config; + GMainLoop *loop; + struct v4l2_device *v4l2_dev; + GList *streams; +} VuVideo; + +struct v4l2_device { + gchar *devname; + unsigned int dev_type; + unsigned int capabilities; + int fd; + int epollfd; + int opened; + bool has_mplane; + bool sup_dyn_res_switching; +}; + +struct vu_video_ctrl_command { + VuVirtqElement elem; + VuVirtq *vq; + VuDev *dev; + struct virtio_video_cmd_hdr *cmd_hdr; + uint32_t error; + bool finished; + uint8_t *cmd_buf; +}; + + +/* + * Structure to track internal state of a Stream + */ + +struct stream { + struct virtio_video_stream_create vio_stream; + uint32_t stream_id; + GList *inputq_resources; + GList *outputq_resources; + VuVideo *video; + GThread *worker_thread; + uint32_t stream_state; + GMutex mutex; + GCond stream_cond; + bool output_streaming; + bool capture_streaming; + bool subscribed_events; + bool has_mplane; + int fd; + uint32_t output_bufcount; + uint32_t capture_bufcount; +}; + +#define STREAM_STOPPED 1 +#define STREAM_STREAMING 2 +#define STREAM_DRAINING 3 +#define STREAM_DESTROYING 4 +#define STREAM_DESTROYED 5 + +/* Structure to track resources */ + +struct resource { + uint32_t stream_id; + struct virtio_video_resource_create vio_resource; + struct virtio_video_resource_queue vio_res_q; + struct iovec *iov; + uint32_t iov_count; + uint32_t v4l2_index; + enum v4l2_buf_type type; + struct vu_video_ctrl_command *vio_q_cmd; + bool queued; +}; + +struct video_format_frame_rates { + struct virtio_video_format_range frame_rates; + struct v4l2_frmivalenum v4l_ival; +}; + +struct video_format_frame { + struct virtio_video_format_frame frame; + struct v4l2_frmsizeenum v4l_framesize; + GList *frm_rate_l; +}; + +struct video_format { + struct v4l2_fmtdesc fmt; + struct virtio_video_format_desc desc; + GList *vid_fmt_frm_l; +}; + +/* function prototypes */ +int v4l2_stream_create(struct v4l2_device *dev, + uint32_t vio_codedformat, struct stream *s); +void v4l2_to_virtio_video_params(struct v4l2_device *dev, + struct v4l2_format *fmt, + struct v4l2_selection *sel, + struct virtio_video_get_params_resp *resp); + +void v4l2_to_virtio_fmtdesc(struct v4l2_device *dev, + struct video_format *vid_fmt, + enum v4l2_buf_type type); + +void v4l2_to_virtio_event(struct v4l2_event *ev, + struct virtio_video_event *vio_ev); + +struct resource *find_resource_by_v4l2index(struct stream *s, + enum v4l2_buf_type buf_type, + uint32_t v4l2_index); +/* + * The following conversion helpers and tables taken from Linux + * frontend driver from opensynergy + */ + +uint32_t virtio_video_level_to_v4l2(uint32_t level); +uint32_t virtio_video_v4l2_level_to_virtio(uint32_t v4l2_level); +uint32_t virtio_video_profile_to_v4l2(uint32_t profile); +uint32_t virtio_video_v4l2_profile_to_virtio(uint32_t v4l2_profile); +uint32_t virtio_video_format_to_v4l2(uint32_t format); +uint32_t virtio_video_v4l2_format_to_virtio(uint32_t v4l2_format); +uint32_t virtio_video_control_to_v4l2(uint32_t control); +uint32_t virtio_video_v4l2_control_to_virtio(uint32_t v4l2_control); +__le64 virtio_fmtdesc_generate_mask(GList **p_list); + +/* Helpers for logging */ +const char *vio_queue_name(enum virtio_video_queue_type queue); + +static inline void +virtio_video_ctrl_hdr_letoh(struct virtio_video_cmd_hdr *hdr) +{ + hdr->type = le32toh(hdr->type); + hdr->stream_id = le32toh(hdr->stream_id); +} + +static inline void +virtio_video_ctrl_hdr_htole(struct virtio_video_cmd_hdr *hdr) +{ + hdr->type = htole32(hdr->type); + hdr->stream_id = htole32(hdr->stream_id); +} +#endif diff --git a/tools/vhost-user-video/vuvideo.h b/tools/vhost-user-video/vuvideo.h new file mode 100644 index 0000000000..de02e05b46 --- /dev/null +++ b/tools/vhost-user-video/vuvideo.h @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * vhost-user-video header + * + * Copyright Linaro 2021 + * + * Authors: + * Peter Griffin + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef VUVIDEO_H +#define VUVIDEO_H + +#include "virtio_video_helpers.h" +#include "v4l2_backend.h" +#include "vuvideo.h" + +size_t video_iov_size(const struct iovec *iov, const unsigned int iov_cnt); + +GList *get_resource_list(struct stream *s, uint32_t queue_type); +void send_qclear_res_reply(gpointer data, gpointer user_data); + +struct stream *find_stream(struct VuVideo *v, uint32_t stream_id); +int add_resource(struct stream *s, struct resource *r, uint32_t queue_type); +int remove_resource(struct stream *s, struct resource *r, uint32_t queue_type); +struct resource *find_resource(struct stream *s, uint32_t resource_id, + uint32_t queue_type); + +void send_ctrl_response(struct vu_video_ctrl_command *vio_cmd, + uint8_t *resp, size_t resp_len); + +void send_ctrl_response_nodata(struct vu_video_ctrl_command *vio_cmd); + +void free_resource_mem(struct resource *r); +void remove_all_resources(struct stream *s, uint32_t queue_type); + +void handle_queue_clear_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd); + +#endif