From patchwork Mon Nov 21 13:59:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7BA1C43217 for ; Mon, 21 Nov 2022 14:02:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229977AbiKUOC5 (ORCPT ); Mon, 21 Nov 2022 09:02:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229814AbiKUOCD (ORCPT ); Mon, 21 Nov 2022 09:02:03 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 211ECCD2; Mon, 21 Nov 2022 06:02:02 -0800 (PST) Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AL7xJa7019467; Mon, 21 Nov 2022 14:01:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=TrJ8uoa6gXngqmx07bGZr8m1fkWRmoABc/gVNOSpwiw=; b=BIvvekxHFwNu6LVtecdOhePUMdRYtbX2OmC/UT860GHCLAEAguBNlhAvP4qVn5li4XVq bLA0mL0lyRCiuYH37i4pR6X/O29mD00I5/ag0e0/gXa0dPs9NWHR/IM5bdw6efFjnYv9 0t8jb0ZjmD5bCAs76dcMpBMrR2LWiGIpE5u0RifGGWeEb+1E6ivIVAA6F5qYa7fcqZei IeotuX91sMjNhqO5G5xABgcEwmkq/H47JIUVgngK8Ay55tmocJDZH/mA/MZnK2bARa96 w1Y93FYeJdPMmbAp2zPbo/3ZYeBRu/2GH5YpENaGke9jPNnXPeijwyo9ILJ/taaBBcfj oA== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrfbvnvf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:34 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE1X94006762 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:33 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:32 -0800 From: Elliot Berman To: Bjorn Andersson , Jonathan Corbet , Bagas Sanjaya , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 01/20] docs: gunyah: Introduce Gunyah Hypervisor Date: Mon, 21 Nov 2022 05:59:50 -0800 Message-ID: <20221121140009.2353512-2-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: fzY7Np0Vq2vYRnCwPec-3GUb13oz_XBI X-Proofpoint-ORIG-GUID: fzY7Np0Vq2vYRnCwPec-3GUb13oz_XBI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 adultscore=0 lowpriorityscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 phishscore=0 priorityscore=1501 clxscore=1015 impostorscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It does not depend on any lower-privileged OS/kernel code for its core functionality. This increases its security and can support a smaller trusted computing based when compared to Type-2 hypervisors. Add documentation describing the Gunyah hypervisor and the main components of the Gunyah hypervisor which are of interest to Linux virtualization development. Reviewed-by: Bagas Sanjaya Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 114 ++++++++++++++++++++ Documentation/virt/gunyah/message-queue.rst | 56 ++++++++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 7 ++ 4 files changed, 178 insertions(+) create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst new file mode 100644 index 000000000000..fbadbdd24da7 --- /dev/null +++ b/Documentation/virt/gunyah/index.rst @@ -0,0 +1,114 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Gunyah Hypervisor +================= + +.. toctree:: + :maxdepth: 1 + + message-queue + +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in +a higher CPU privilege level. It does not depend on any lower-privileged operating system +for its core functionality. This increases its security and can support a much smaller +trusted computing base than a Type-2 hypervisor. + +Gunyah is an open source hypervisor. The source repo is available at +https://github.com/quic/gunyah-hypervisor. + +Gunyah provides these following features. + +- Scheduling: + + A scheduler for virtual CPUs (vCPUs) on physical CPUs and enables time-sharing + of the CPUs. Gunyah supports two models of scheduling: + + 1. "Behind the back" scheduling in which Gunyah hypervisor schedules vCPUS on its own. + 2. "Proxy" scheduling in which a delegated VM can donate part of one of its vCPU slice + to another VM's vCPU via a hypercall. + +- Memory Management: + + APIs handling memory, abstracted as objects, limiting direct use of physical + addresses. Memory ownership and usage tracking of all memory under its control. + Memory partitioning between VMs is a fundamental security feature. + +- Interrupt Virtualization: + + Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled + in the hypervisor and routed to the assigned VM. + +- Inter-VM Communication: + + There are several different mechanisms provided for communicating between VMs. + +- Virtual platform: + + Architectural devices such as interrupt controllers and CPU timers are directly provided + by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI. + +- Device Virtualization: + + Para-virtualization of devices is supported using inter-VM communication. + +Architectures supported +======================= +AArch64 with a GIC + +Resources and Capabilities +========================== + +Some services or resources provided by the Gunyah hypervisor are described to a virtual machine by +capability IDs. For instance, inter-VM communication is performed with doorbells and message queues. +Gunyah allows access to manipulate that doorbell via the capability ID. These devices are described +in Linux as a struct gunyah_resource. + +High level management of these resources is performed by the resource manager VM. RM informs a +guest VM about resources it can access through either the device tree or via guest-initiated RPC. + +For each virtual machine, Gunyah maintains a table of resources which can be accessed by that VM. +An entry in this table is called a "capability" and VMs can only access resources via this +capability table. Hence, virtual Gunyah devices are referenced by a "capability IDs" and not a +"resource IDs". A VM can have multiple capability IDs mapping to the same resource. If 2 VMs have +access to the same resource, they may not be using the same capability ID to access that resource +since the tables are independent per VM. + +Resource Manager +================ + +The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor. +It provides policy enforcement aspects of the virtualization system. The resource manager can +be treated as an extension of the Hypervisor but is separated to its own partition to ensure +that the hypervisor layer itself remains small and secure and to maintain a separation of policy +and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines. + +Communication with the resource manager from each guest VM happens with message-queue.rst. Details +about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c + +:: + + +-------+ +--------+ +--------+ + | RM | | VM_A | | VM_B | + +-.-.-.-+ +---.----+ +---.----+ + | | | | + +-.-.-----------.------------.----+ + | | \==========/ | | + | \========================/ | + | Gunyah | + +---------------------------------+ + +The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager. + +The resource manager provides the following features: + +- VM lifecycle management: allocating a VM, starting VMs, destruction of VMs +- VM access control policy, including memory sharing and lending +- Interrupt routing configuration +- Forwarding of system-level events (e.g. VM shutdown) to owner VM + +When booting a virtual machine which uses a devicetree, resource manager overlays a +/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM, +how to communicate with resource manager, and basic description and capabilities of +this VM. See Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml for a description +of this node. diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst new file mode 100644 index 000000000000..be4ab289236a --- /dev/null +++ b/Documentation/virt/gunyah/message-queue.rst @@ -0,0 +1,56 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Message Queues +============== +Message queue is a simple low-capacity IPC channel between two VMs. It is +intended for sending small control and configuration messages. Each message +queue object is unidirectional, so a full-duplex IPC channel requires a pair of +objects. + +Messages can be up to 240 bytes in length. Longer messages require a further +protocol on top of the message queue messages themselves. For instance, communication +with the resource manager adds a header field for sending longer messages via multiple +message fragments. + +The diagram below shows how message queue works. A typical configuration involves +2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message +queue 2 allows VM_B to send messages to VM_A. + +1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall + with the message to inform the hypervisor to add the message to + message queue 1's queue. + +2. Gunyah raises the corresponding interrupt for VM_B when any of these happens: + + a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case. + b. Explicility with gh_msgq_push command from VM_A. + c. Message queue has reached a threshold depth. + +3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer. + +For VM_B to send a message to VM_A, the process is identical, except that hypercalls +reference message queue 2's capability ID. Each message queue has its own independent +vIRQ: two TX message queues will have two vIRQs (and two capability IDs). + +:: + + +---------------+ +-----------------+ +---------------+ + | VM_A | |Gunyah hypervisor| | VM_B | + | | | | | | + | | | | | | + | | Tx | | | | + | |-------->| | Rx vIRQ | | + |gh_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gh_msgq_recv() | + | |<------- | | | | + | | | | | | + | Message Queue | | | | Message Queue | + | driver | | | | driver | + | | | | | | + | | | | | | + | | | | Tx | | + | | Rx vIRQ | |<--------| | + |gh_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gh_msgq_send() | + | | | |-------->| | + | | | | | | + | | | | | | + +---------------+ +-----------------+ +---------------+ diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index 2f1cffa87b1b..418d540f5484 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -15,6 +15,7 @@ Linux Virtualization Support acrn/index coco/sev-guest hyperv/index + gunyah/index .. only:: html and subproject diff --git a/MAINTAINERS b/MAINTAINERS index 256f03904987..3c6c7cbe6888 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8938,6 +8938,13 @@ L: linux-efi@vger.kernel.org S: Maintained F: block/partitions/efi.* +GUNYAH HYPERVISOR DRIVER +M: Elliot Berman +M: Murali Nalajala +L: linux-arm-msm@vger.kernel.org +S: Supported +F: Documentation/virt/gunyah/ + HABANALABS PCI DRIVER M: Oded Gabbay S: Supported From patchwork Mon Nov 21 13:59:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627288 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8482BC4332F for ; Mon, 21 Nov 2022 14:02:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230099AbiKUOCF (ORCPT ); Mon, 21 Nov 2022 09:02:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231565AbiKUOBi (ORCPT ); Mon, 21 Nov 2022 09:01:38 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3B27183AC; Mon, 21 Nov 2022 06:01:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039296; x=1700575296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rWpgfF+grvxhwlUn7RH6TlVBHLE7hev1bxoVYI1JmjA=; b=i9DmCQrypBKEDVOt+5zD8jYw0ShgLSzFxmB9XXrPjBEBIEYho+/ylJU1 K6cVpQuuvIWEcvGxe5LgL9hG1iGs3xFjamExHjMg4xzIKxBeTqErQ1OpV NNCw9vc2uAu4vRGaVB6pRsnyf8iPeT3VFxqWuC9HU6S5VbSeeZp3M/hj4 s=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 21 Nov 2022 06:01:36 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:01:34 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:33 -0800 From: Elliot Berman To: Bjorn Andersson , Rob Herring , Krzysztof Kozlowski , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , "Jonathan Corbet" , Catalin Marinas , "Will Deacon" , Jassi Brar , Sudeep Holla , Mark Rutland , "Lorenzo Pieralisi" , , , , , , Subject: [PATCH v7 02/20] dt-bindings: Add binding for gunyah hypervisor Date: Mon, 21 Nov 2022 05:59:51 -0800 Message-ID: <20221121140009.2353512-3-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org When Linux is booted as a guest under the Gunyah hypervisor, the Gunyah Resource Manager applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. This information is not otherwise discoverable by a VM: the Gunyah hypervisor core does not provide a direct interface to discover capability IDs nor a way to communicate with RM without having already known the corresponding message queue capability ID. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Signed-off-by: Elliot Berman Reviewed-by: Rob Herring --- .../bindings/firmware/gunyah-hypervisor.yaml | 82 +++++++++++++++++++ MAINTAINERS | 1 + 2 files changed, 83 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 000000000000..9fd1d254b156 --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Murali Nalajala + - Elliot Berman + +description: |+ + Gunyah virtual machines use this information to determine the capability IDs + of the message queues used to communicate with the Gunyah Resource Manager. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + const: gunyah-resource-manager + + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + + additionalProperties: false + + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "gunyah-hypervisor"; + + gunyah-resource-mgr@0 { + compatible = "gunyah-resource-manager"; + interrupts = , /* TX full IRQ */ + ; /* RX empty IRQ */ + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; + /* TX, RX cap ids */ + }; + }; diff --git a/MAINTAINERS b/MAINTAINERS index 3c6c7cbe6888..ecca14b28b7c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8943,6 +8943,7 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Supported +F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ HABANALABS PCI DRIVER From patchwork Mon Nov 21 13:59:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB4C2C43217 for ; Mon, 21 Nov 2022 14:02:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231221AbiKUOCG (ORCPT ); Mon, 21 Nov 2022 09:02:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231567AbiKUOBi (ORCPT ); Mon, 21 Nov 2022 09:01:38 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A969714D2B; Mon, 21 Nov 2022 06:01:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039296; x=1700575296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QbhRaPjXj++IurS0vOW7ALgIU74DZrFaOGSG30viE4s=; b=OWCPxmfGu0HLoLke/3uzG0ydcGPBC5EbirSG1h/PkTYbWMVrAbVFSeMM 0t/+xAYpHgu1/Wu5ZGccysO7e0AauIlcHBZbO07a2r9CDyiBVzS98nsDS 1L1q5UDVAVnb4LZFtH8LhTgJzIFNGpuerRKTuAeeisSz1UGNoWVvurKLS I=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-02.qualcomm.com with ESMTP; 21 Nov 2022 06:01:36 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:01:36 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:35 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , , Dmitry Baryshkov Subject: [PATCH v7 03/20] gunyah: Common types and error codes for Gunyah hypercalls Date: Mon, 21 Nov 2022 05:59:52 -0800 Message-ID: <20221121140009.2353512-4-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Reviewed-by: Dmitry Baryshkov Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + include/linux/gunyah.h | 74 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/MAINTAINERS b/MAINTAINERS index ecca14b28b7c..c0d46ab5b199 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8945,6 +8945,7 @@ L: linux-arm-msm@vger.kernel.org S: Supported F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ +F: include/linux/gunyah.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..824e20a11d27 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_H +#define _GUNYAH_H + +#include +#include + +/* Common Gunyah macros */ +#define GH_CAPID_INVAL U64_MAX +#define GH_VMID_ROOT_VM 0xff + +#define GH_ERROR_OK 0 + +#define GH_ERROR_UNIMPLEMENTED -1 +#define GH_ERROR_RETRY -2 + +#define GH_ERROR_ARG_INVAL 1 +#define GH_ERROR_ARG_SIZE 2 +#define GH_ERROR_ARG_ALIGN 3 + +#define GH_ERROR_NOMEM 10 + +#define GH_ERROR_ADDR_OVFL 20 +#define GH_ERROR_ADDR_UNFL 21 +#define GH_ERROR_ADDR_INVAL 22 + +#define GH_ERROR_DENIED 30 +#define GH_ERROR_BUSY 31 +#define GH_ERROR_IDLE 32 + +#define GH_ERROR_IRQ_BOUND 40 +#define GH_ERROR_IRQ_UNBOUND 41 + +#define GH_ERROR_CSPACE_CAP_NULL 50 +#define GH_ERROR_CSPACE_CAP_REVOKED 51 +#define GH_ERROR_CSPACE_WRONG_OBJ_TYPE 52 +#define GH_ERROR_CSPACE_INSUF_RIGHTS 53 +#define GH_ERROR_CSPACE_FULL 54 + +#define GH_ERROR_MSGQUEUE_EMPTY 60 +#define GH_ERROR_MSGQUEUE_FULL 61 + +static inline int gh_remap_error(int gh_error) +{ + switch (gh_error) { + case GH_ERROR_OK: + return 0; + case GH_ERROR_NOMEM: + return -ENOMEM; + case GH_ERROR_DENIED: + case GH_ERROR_CSPACE_CAP_NULL: + case GH_ERROR_CSPACE_CAP_REVOKED: + case GH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GH_ERROR_CSPACE_INSUF_RIGHTS: + case GH_ERROR_CSPACE_FULL: + return -EACCES; + case GH_ERROR_BUSY: + case GH_ERROR_IDLE: + return -EBUSY; + case GH_ERROR_IRQ_BOUND: + case GH_ERROR_IRQ_UNBOUND: + case GH_ERROR_MSGQUEUE_FULL: + case GH_ERROR_MSGQUEUE_EMPTY: + return -EPERM; + default: + return -EINVAL; + } +} + +#endif From patchwork Mon Nov 21 13:59:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 597D2C4332F for ; Mon, 21 Nov 2022 14:02:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229558AbiKUOCc (ORCPT ); Mon, 21 Nov 2022 09:02:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231666AbiKUOB7 (ORCPT ); Mon, 21 Nov 2022 09:01:59 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B97D238A6; Mon, 21 Nov 2022 06:01:58 -0800 (PST) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALCFqcj012576; Mon, 21 Nov 2022 14:01:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=cWnD/VtzmOTFNtRhKXIobLlELZFzKzUnSZ/Rd7g1umo=; b=UoUTtZWsTlmzGoPQjmC4eMk1292XtvDAas8kUjsV85NEMshaYSBVDibemjIrXVSxdLqL 8uYNYfv+mlpEgaXdhYE/JZj9quETZHLBiJ3sL1/h6D0dzbYHIBn93S72YMdajLraEDJt PfTYJiqMyWVtP72B9e+RfytlbIVej8yME1EKaXO1qAMgZQbPoG61bniZ8q+I+393fpZ9 yhNyiRpKTszdRObLlAezz8gZKGx3hutwu9/MCl3BPj03ko38zSe6LgAoX14PVeizD+2B +jtvOXop0oUutd2idfPmPd61U+WqlqdX6nXr2xRSFoRzJ1dqehXCwVUcBceSwcbAML8h lg== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrbc4n6f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:38 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE1b9m030537 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:37 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:36 -0800 From: Elliot Berman To: Bjorn Andersson , Mark Rutland , Lorenzo Pieralisi , "Sudeep Holla" CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , , , , , , Subject: [PATCH v7 04/20] arm64: smccc: Include alternative-macros.h Date: Mon, 21 Nov 2022 05:59:53 -0800 Message-ID: <20221121140009.2353512-5-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: FuZyxSIjNTHdyHd6fKB7weiepAxSEkDu X-Proofpoint-GUID: FuZyxSIjNTHdyHd6fKB7weiepAxSEkDu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0 spamscore=0 bulkscore=0 adultscore=0 suspectscore=0 malwarescore=0 mlxlogscore=633 priorityscore=1501 clxscore=1015 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Fix build error when CONFIG_ARM64_SVE is selected and asm/alternative-macros.h wasn't implicitly included by another header. In file included from arch/arm64/gunyah/gunyah_hypercall.c:6: arch/arm64/gunyah/gunyah_hypercall.c: In function `gh_hypercall_msgq_send': ./include/linux/arm-smccc.h:387:25: error: expected string literal before `ALTERNATIVE' 387 | #define SMCCC_SVE_CHECK ALTERNATIVE("nop \n", "bl __arm_smccc_sve_check \n", \ Signed-off-by: Elliot Berman --- include/linux/arm-smccc.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 220c8c60e021..6a627cdbbdec 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -383,6 +383,7 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, /* nVHE hypervisor doesn't have a current thread so needs separate checks */ #if defined(CONFIG_ARM64_SVE) && !defined(__KVM_NVHE_HYPERVISOR__) +#include #define SMCCC_SVE_CHECK ALTERNATIVE("nop \n", "bl __arm_smccc_sve_check \n", \ ARM64_SVE) From patchwork Mon Nov 21 13:59:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627278 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92423C433FE for ; Mon, 21 Nov 2022 14:07:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229774AbiKUOH1 (ORCPT ); Mon, 21 Nov 2022 09:07:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231190AbiKUOHD (ORCPT ); Mon, 21 Nov 2022 09:07:03 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 341A012ADA; Mon, 21 Nov 2022 06:03:45 -0800 (PST) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AL8RlZ9009241; Mon, 21 Nov 2022 14:01:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=6Hu/6DLOQBH7Lhfco7bA9G8E/CwYUETD6eYsdppgHM4=; b=b03/SjdLPnIxLVsgWdSBw5fHrANpTAA1LfUTJeC+fe/R0VeBdnLbgNa+3OxzO0/C/tQA IqinZZQkxzYMA6g+o/6+A1/LqlVcKw3T1yjocK3VqZENWW4Vdtb6dxlZsquDkR2NLe9f Z5LqI7sh6rJOF8AkuOUCWJBYxO5scv2GHse9QxB+xHyyJ74iXFsADmHqKS8U426JD3G8 8B1x6k83Dcb6JtSnWU2sfK03RVSFCprBOGtUAGpxLO9KUUsd7jOCroNvD1Ijf3oOGfPu gVFrzAwErTfcHBxafTntq+6bg21t109UJkWzQabZ5vBWGssNPBDvF4opd/JSaTkAl5iS /A== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrf5mptk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:41 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE1exo024244 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:40 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:38 -0800 From: Elliot Berman To: Bjorn Andersson , Catalin Marinas , Will Deacon , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Jassi Brar" , Sudeep Holla , "Mark Rutland" , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 05/20] virt: gunyah: Add hypercalls to identify Gunyah Date: Mon, 21 Nov 2022 05:59:54 -0800 Message-ID: <20221121140009.2353512-6-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: Z8Zxdx3Tdp5dR5gfWFKuOMgVa7L5FvfI X-Proofpoint-ORIG-GUID: Z8Zxdx3Tdp5dR5gfWFKuOMgVa7L5FvfI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Add hypercalls to identify when Linux is running a virtual machine under Gunyah. There are two calls to help identify Gunyah: 1. gh_hypercall_get_uid() returns a UID when running under a Gunyah hypervisor. 2. gh_hypercall_hyp_identify() returns build information and a set of feature flags that are supported by Gunyah. Signed-off-by: Elliot Berman --- MAINTAINERS | 2 + arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 1 + arch/arm64/gunyah/gunyah_hypercall.c | 69 ++++++++++++++++++++++++++++ drivers/virt/Kconfig | 1 + drivers/virt/gunyah/Kconfig | 12 +++++ include/linux/gunyah.h | 25 ++++++++++ 7 files changed, 111 insertions(+) create mode 100644 arch/arm64/gunyah/Makefile create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c create mode 100644 drivers/virt/gunyah/Kconfig diff --git a/MAINTAINERS b/MAINTAINERS index c0d46ab5b199..69c1835b3bb4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8945,6 +8945,8 @@ L: linux-arm-msm@vger.kernel.org S: Supported F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ +F: arch/arm64/gunyah/ +F: drivers/virt/gunyah/ F: include/linux/gunyah.h HABANALABS PCI DRIVER diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild index 5bfbf7d79c99..e4847ba0e3c9 100644 --- a/arch/arm64/Kbuild +++ b/arch/arm64/Kbuild @@ -3,6 +3,7 @@ obj-y += kernel/ mm/ net/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_XEN) += xen/ obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_CRYPTO) += crypto/ # for cleaning diff --git a/arch/arm64/gunyah/Makefile b/arch/arm64/gunyah/Makefile new file mode 100644 index 000000000000..9fbc720b6fb6 --- /dev/null +++ b/arch/arm64/gunyah/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_GUNYAH) += gunyah_hypercall.o diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c new file mode 100644 index 000000000000..0beb3123d650 --- /dev/null +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include + +#define GH_CALL_TYPE_PLATFORM_CALL 0 +#define GH_CALL_TYPE_HYPERCALL 2 +#define GH_CALL_TYPE_SERVICE 3 +#define GH_CALL_TYPE_SHIFT 14 +#define GH_CALL_FUNCTION_NUM_MASK 0x3fff + +#define GH_FN_ID(type, num) ((type) << GH_CALL_TYPE_SHIFT | ((num) & GH_CALL_FUNCTION_NUM_MASK)) + +#define GH_SERVICE(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + GH_FN_ID(GH_CALL_TYPE_SERVICE, fn)) + +#define GH_HYPERCALL_CALL_UID GH_SERVICE(0x3f01) + +#define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + GH_FN_ID(GH_CALL_TYPE_HYPERCALL, fn)) + +#define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000) + +/** + * gh_hypercall_get_uid() - Returns a UID when running under a Gunyah hypervisor + * @uid: An array of 4 u32's (u32 uid[4];) + * + * Caller should compare the resulting UID to a list of known Gunyah UIDs to + * confirm that Linux is running as a guest of Gunyah. + */ +void gh_hypercall_get_uid(u32 uid[4]) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_CALL_UID, &res); + + uid[0] = res.a0; + uid[1] = res.a1; + uid[2] = res.a2; + uid[3] = res.a3; +} +EXPORT_SYMBOL_GPL(gh_hypercall_get_uid); + +/** + * gh_hypercall_hyp_identify() - Returns build information and feature flags + * supported by Gunyah. + * @hyp_identity: filled by the hypercall with the API info and feature flags. + */ +void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_HYP_IDENTIFY, &res); + + hyp_identity->api_info = res.a0; + hyp_identity->flags[0] = res.a1; + hyp_identity->flags[1] = res.a2; + hyp_identity->flags[2] = res.a3; +} +EXPORT_SYMBOL_GPL(gh_hypercall_hyp_identify); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index 87ef258cec64..259dc2be6cad 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -52,4 +52,5 @@ source "drivers/virt/coco/efi_secret/Kconfig" source "drivers/virt/coco/sev-guest/Kconfig" +source "drivers/virt/gunyah/Kconfig" endif diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..127156a678a6 --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + help + The Gunyah drivers are the helper interfaces that runs in a guest VM + such as basic inter-VM IPC and signaling mechanisms, and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y/M here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 824e20a11d27..2765d2b40198 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,6 +6,7 @@ #ifndef _GUNYAH_H #define _GUNYAH_H +#include #include #include @@ -71,4 +72,28 @@ static inline int gh_remap_error(int gh_error) } } +#define GUNYAH_API_V1 1 + +#define GH_API_INFO_API_VERSION_MASK GENMASK_ULL(13, 0) +#define GH_API_INFO_BIG_ENDIAN BIT_ULL(14) +#define GH_API_INFO_IS_64BIT BIT_ULL(15) +#define GH_API_INFO_VARIANT_MASK GENMASK_ULL(63, 56) + +#define GH_IDENTIFY_PARTITION_CSPACE BIT_ULL(0) +#define GH_IDENTIFY_DOORBELL BIT_ULL(1) +#define GH_IDENTIFY_MSGQUEUE BIT_ULL(2) +#define GH_IDENTIFY_VIC BIT_ULL(3) +#define GH_IDENTIFY_VPM BIT_ULL(4) +#define GH_IDENTIFY_VCPU BIT_ULL(5) +#define GH_IDENTIFY_MEMEXTENT BIT_ULL(6) +#define GH_IDENTIFY_TRACE_CTRL BIT_ULL(7) + +struct gh_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +void gh_hypercall_get_uid(u32 uid[4]); +void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity); + #endif From patchwork Mon Nov 21 13:59:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B000C433FE for ; Mon, 21 Nov 2022 14:03:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231478AbiKUODp (ORCPT ); Mon, 21 Nov 2022 09:03:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230198AbiKUOCN (ORCPT ); Mon, 21 Nov 2022 09:02:13 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EFDC81C929; Mon, 21 Nov 2022 06:02:11 -0800 (PST) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALCqsGw013197; Mon, 21 Nov 2022 14:01:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=HF4iBQ+cZRuNQfUCG3QLF2FXAPRplCw3XpMEge6HY8k=; b=UdVUMe9dTK7PS7ucA3sbBLlpQmxWyYTpj5rJZ/9B8A0SGS7dUH6UYxl7h3WMD/ZdqnJP /JmIzZ45aPZl+heAUTYjIdcfboSrdKomZJ8Y2+8Qm+kxBpfzXDCieBJmZEhxo7S+NwGE lWPmeeTHlG5SKQYSLmJsj5JKri8ktFYxVCt7eediSSWHg7tLjwmE5xrFGt3deaOZro86 ZYIWJf2WltmsB6LAr4UiiwGdxAXt6qyZxMyRM0z5iSIKpcNSOlJO2zXGlgAUaHLbOV5Y DvqQ4TpkydFkYSFMVXabL0WMCviE9mWvnDzDyKCh/BXHE/Ixnk7hOgGlvM3xIeRoJV7+ /w== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrbc4n6x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:44 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE1hcL024269 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:43 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:41 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 06/20] virt: gunyah: Identify hypervisor version Date: Mon, 21 Nov 2022 05:59:55 -0800 Message-ID: <20221121140009.2353512-7-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: LOG0Aj_TUCRD5F2KpVHlPdlGGWX-WR7a X-Proofpoint-GUID: LOG0Aj_TUCRD5F2KpVHlPdlGGWX-WR7a X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0 spamscore=0 bulkscore=0 adultscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 priorityscore=1501 clxscore=1015 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Export the version of Gunyah which is reported via the hyp_identify hypercall. Increments of the major API version indicate possibly backwards incompatible changes. Export the hypervisor identity so that Gunyah drivers can act according to the major API version. Signed-off-by: Elliot Berman --- drivers/virt/Makefile | 1 + drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah.c | 46 ++++++++++++++++++++++++++++++++++++ include/linux/gunyah.h | 6 +++++ 4 files changed, 54 insertions(+) create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/gunyah.c diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index 093674e05c40..92b7e5311548 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -11,3 +11,4 @@ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) += acrn/ obj-$(CONFIG_EFI_SECRET) += coco/efi_secret/ obj-$(CONFIG_SEV_GUEST) += coco/sev-guest/ +obj-y += gunyah/ diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile new file mode 100644 index 000000000000..2ac4ee64b89d --- /dev/null +++ b/drivers/virt/gunyah/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/gunyah.c b/drivers/virt/gunyah/gunyah.c new file mode 100644 index 000000000000..c34c9046fc08 --- /dev/null +++ b/drivers/virt/gunyah/gunyah.c @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gunyah: " fmt + +#include +#include +#include +#include + +struct gh_hypercall_hyp_identify_resp gunyah_api; +EXPORT_SYMBOL_GPL(gunyah_api); + +static const uint32_t gunyah_known_uuids[][4] = { + {0x19bd54bd, 0x0b37571b, 0x946f609b, 0x54539de6}, /* QC_HYP (Qualcomm's build) */ + {0x673d5f14, 0x9265ce36, 0xa4535fdb, 0xc1d58fcd}, /* GUNYAH (open source build) */ +}; + +static int __init gunyah_init(void) +{ + u32 uid[4]; + int i; + + gh_hypercall_get_uid(uid); + + for (i = 0; i < ARRAY_SIZE(gunyah_known_uuids); i++) + if (!memcmp(uid, gunyah_known_uuids[i], sizeof(uid))) + break; + + if (i == ARRAY_SIZE(gunyah_known_uuids)) + return -ENODEV; + + gh_hypercall_hyp_identify(&gunyah_api); + + pr_info("Running under Gunyah hypervisor %llx/v%u\n", + FIELD_GET(GH_API_INFO_VARIANT_MASK, gunyah_api.api_info), + gh_api_version()); + + return 0; +} +arch_initcall(gunyah_init); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Driver"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 2765d2b40198..166156f69df9 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -92,6 +92,12 @@ struct gh_hypercall_hyp_identify_resp { u64 api_info; u64 flags[3]; }; +extern struct gh_hypercall_hyp_identify_resp gunyah_api; + +static inline u16 gh_api_version(void) +{ + return FIELD_GET(GH_API_INFO_API_VERSION_MASK, gunyah_api.api_info); +} void gh_hypercall_get_uid(u32 uid[4]); void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity); From patchwork Mon Nov 21 13:59:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5B35C4321E for ; Mon, 21 Nov 2022 14:03:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230442AbiKUODd (ORCPT ); Mon, 21 Nov 2022 09:03:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230353AbiKUOCL (ORCPT ); Mon, 21 Nov 2022 09:02:11 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8852314D1C; Mon, 21 Nov 2022 06:02:09 -0800 (PST) Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALCdY9f027031; Mon, 21 Nov 2022 14:01:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=cWDvb5UzDNi25fkhEnalKmKNFXZVYs7FKqAeVt8ehwg=; b=oYVhok6uMH8kOED+mHZNKpoSWeKnpycscNQXLuRL1M8h97lGLxOqk63uXr7OZ0gv2y6a MgMl8kG81fG9qXBBgfGtD3y3QXTWSQgpn3vomgRvR/MlXu7guoB2GlQq5wj98iCL9Ug9 IQaATv3x7iI4ct8kPklsqOn960bz1mtpPe1qHaIImuosyS/SDo/Ialt4RAE/sOqCvUjE hDnnXaIDAf9Y2DGvU98tkavLfVlUzmYDQ6ZUDkT1Xn3l/6UP+YTUgv+kD/7ZQtCW7BBk vtL/gLchFkDoJJU/hMGj3HjVvD+0dvIlaw9Iwc4gsFQhzhQe8UFK5/beRiYWyz9Wmp9o HA== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrpb4j67-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:47 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE1kar014674 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:46 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:44 -0800 From: Elliot Berman To: Bjorn Andersson , Jassi Brar , Sudeep Holla CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Catalin Marinas , Will Deacon , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 07/20] mailbox: Allow direct registration to a channel Date: Mon, 21 Nov 2022 05:59:56 -0800 Message-ID: <20221121140009.2353512-8-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: V4Pvf6HFqQ0C6Qi4sHWgcokF9zz6TwSD X-Proofpoint-GUID: V4Pvf6HFqQ0C6Qi4sHWgcokF9zz6TwSD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 mlxscore=0 lowpriorityscore=0 suspectscore=0 bulkscore=0 malwarescore=0 priorityscore=1501 mlxlogscore=999 phishscore=0 clxscore=1015 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Support virtual mailbox controllers and clients which are not platform devices or come from the devicetree by allowing them to match client to channel via some other mechanism. Signed-off-by: Elliot Berman --- drivers/mailbox/mailbox.c | 96 ++++++++++++++++++++++++---------- drivers/mailbox/omap-mailbox.c | 18 ++----- drivers/mailbox/pcc.c | 17 ++---- include/linux/mailbox_client.h | 1 + 4 files changed, 76 insertions(+), 56 deletions(-) diff --git a/drivers/mailbox/mailbox.c b/drivers/mailbox/mailbox.c index 4229b9b5da98..adf36c05fa43 100644 --- a/drivers/mailbox/mailbox.c +++ b/drivers/mailbox/mailbox.c @@ -317,6 +317,71 @@ int mbox_flush(struct mbox_chan *chan, unsigned long timeout) } EXPORT_SYMBOL_GPL(mbox_flush); +static int __mbox_bind_client(struct mbox_chan *chan, struct mbox_client *cl) +{ + struct device *dev = cl->dev; + unsigned long flags; + int ret; + + if (chan->cl || !try_module_get(chan->mbox->dev->driver->owner)) { + dev_dbg(dev, "%s: mailbox not free\n", __func__); + return -EBUSY; + } + + spin_lock_irqsave(&chan->lock, flags); + chan->msg_free = 0; + chan->msg_count = 0; + chan->active_req = NULL; + chan->cl = cl; + init_completion(&chan->tx_complete); + + if (chan->txdone_method == TXDONE_BY_POLL && cl->knows_txdone) + chan->txdone_method = TXDONE_BY_ACK; + + spin_unlock_irqrestore(&chan->lock, flags); + + if (chan->mbox->ops->startup) { + ret = chan->mbox->ops->startup(chan); + + if (ret) { + dev_err(dev, "Unable to startup the chan (%d)\n", ret); + mbox_free_channel(chan); + return ret; + } + } + + return 0; +} + +/** + * mbox_bind_client - Request a mailbox channel. + * @chan: The mailbox channel to bind the client to. + * @cl: Identity of the client requesting the channel. + * + * The Client specifies its requirements and capabilities while asking for + * a mailbox channel. It can't be called from atomic context. + * The channel is exclusively allocated and can't be used by another + * client before the owner calls mbox_free_channel. + * After assignment, any packet received on this channel will be + * handed over to the client via the 'rx_callback'. + * The framework holds reference to the client, so the mbox_client + * structure shouldn't be modified until the mbox_free_channel returns. + * + * Return: 0 if the channel was assigned to the client successfully. + * <0 for request failure. + */ +int mbox_bind_client(struct mbox_chan *chan, struct mbox_client *cl) +{ + int ret; + + mutex_lock(&con_mutex); + ret = __mbox_bind_client(chan, cl); + mutex_unlock(&con_mutex); + + return ret; +} +EXPORT_SYMBOL_GPL(mbox_bind_client); + /** * mbox_request_channel - Request a mailbox channel. * @cl: Identity of the client requesting the channel. @@ -340,7 +405,6 @@ struct mbox_chan *mbox_request_channel(struct mbox_client *cl, int index) struct mbox_controller *mbox; struct of_phandle_args spec; struct mbox_chan *chan; - unsigned long flags; int ret; if (!dev || !dev->of_node) { @@ -372,33 +436,9 @@ struct mbox_chan *mbox_request_channel(struct mbox_client *cl, int index) return chan; } - if (chan->cl || !try_module_get(mbox->dev->driver->owner)) { - dev_dbg(dev, "%s: mailbox not free\n", __func__); - mutex_unlock(&con_mutex); - return ERR_PTR(-EBUSY); - } - - spin_lock_irqsave(&chan->lock, flags); - chan->msg_free = 0; - chan->msg_count = 0; - chan->active_req = NULL; - chan->cl = cl; - init_completion(&chan->tx_complete); - - if (chan->txdone_method == TXDONE_BY_POLL && cl->knows_txdone) - chan->txdone_method = TXDONE_BY_ACK; - - spin_unlock_irqrestore(&chan->lock, flags); - - if (chan->mbox->ops->startup) { - ret = chan->mbox->ops->startup(chan); - - if (ret) { - dev_err(dev, "Unable to startup the chan (%d)\n", ret); - mbox_free_channel(chan); - chan = ERR_PTR(ret); - } - } + ret = __mbox_bind_client(chan, cl); + if (ret) + chan = ERR_PTR(ret); mutex_unlock(&con_mutex); return chan; diff --git a/drivers/mailbox/omap-mailbox.c b/drivers/mailbox/omap-mailbox.c index 098c82d87137..711a56ec8592 100644 --- a/drivers/mailbox/omap-mailbox.c +++ b/drivers/mailbox/omap-mailbox.c @@ -441,21 +441,9 @@ struct mbox_chan *omap_mbox_request_channel(struct mbox_client *cl, if (!mbox || !mbox->chan) return ERR_PTR(-ENOENT); - chan = mbox->chan; - spin_lock_irqsave(&chan->lock, flags); - chan->msg_free = 0; - chan->msg_count = 0; - chan->active_req = NULL; - chan->cl = cl; - init_completion(&chan->tx_complete); - spin_unlock_irqrestore(&chan->lock, flags); - - ret = chan->mbox->ops->startup(chan); - if (ret) { - pr_err("Unable to startup the chan (%d)\n", ret); - mbox_free_channel(chan); - chan = ERR_PTR(ret); - } + ret = mbox_bind_client(mbox->chan, cl); + if (ret) + return ERR_PTR(ret); return chan; } diff --git a/drivers/mailbox/pcc.c b/drivers/mailbox/pcc.c index 3c2bc0ca454c..b93fd95559c0 100644 --- a/drivers/mailbox/pcc.c +++ b/drivers/mailbox/pcc.c @@ -284,6 +284,7 @@ pcc_mbox_request_channel(struct mbox_client *cl, int subspace_id) struct mbox_chan *chan; struct device *dev; unsigned long flags; + int rc; if (subspace_id < 0 || subspace_id >= pcc_chan_count) return ERR_PTR(-ENOENT); @@ -296,21 +297,11 @@ pcc_mbox_request_channel(struct mbox_client *cl, int subspace_id) } dev = chan->mbox->dev; - spin_lock_irqsave(&chan->lock, flags); - chan->msg_free = 0; - chan->msg_count = 0; - chan->active_req = NULL; - chan->cl = cl; - init_completion(&chan->tx_complete); - - if (chan->txdone_method == TXDONE_BY_POLL && cl->knows_txdone) - chan->txdone_method = TXDONE_BY_ACK; - - spin_unlock_irqrestore(&chan->lock, flags); + rc = mbox_bind_client(chan, cl); + if (rc) + return ERR_PTR(rc); if (pchan->plat_irq > 0) { - int rc; - rc = devm_request_irq(dev, pchan->plat_irq, pcc_mbox_irq, 0, MBOX_IRQ_NAME, chan); if (unlikely(rc)) { diff --git a/include/linux/mailbox_client.h b/include/linux/mailbox_client.h index 65229a45590f..734694912ef7 100644 --- a/include/linux/mailbox_client.h +++ b/include/linux/mailbox_client.h @@ -37,6 +37,7 @@ struct mbox_client { void (*tx_done)(struct mbox_client *cl, void *mssg, int r); }; +int mbox_bind_client(struct mbox_chan *chan, struct mbox_client *cl); struct mbox_chan *mbox_request_channel_byname(struct mbox_client *cl, const char *name); struct mbox_chan *mbox_request_channel(struct mbox_client *cl, int index); From patchwork Mon Nov 21 13:59:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28D21C4167E for ; Mon, 21 Nov 2022 14:03:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231673AbiKUODf (ORCPT ); Mon, 21 Nov 2022 09:03:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230419AbiKUOCL (ORCPT ); Mon, 21 Nov 2022 09:02:11 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46DEB17043; Mon, 21 Nov 2022 06:02:10 -0800 (PST) Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AL8Iil0023398; Mon, 21 Nov 2022 14:01:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=LaXGD1Ji20/92Kosv5wZDMWkkXiKv3o9E6QOZ2ghcxg=; b=YPDUmggm5uGfhh3EZfUI1TFCQev7XV+AK5ivNvufjVp7bMm+3wmsWZVaClUwB6Yb3gTN VMeoZSu+7J8nrmubk1jpS0JQUkYo0fDZdU3eHF+2HG4dy2Ag5ryTb86q6o/7npBJnYbm HV9X2ERRu41iBb9zY0GXQMr7XgsmLeYpP4YA4hjCLJuWF3Gud+8c7Z8EzwpbdsUIr1TQ Uv//awsZMkHRkUyoH8OY2BNauWg4wLsqHQahaq5xb0Zj6xQ7Y2v8aZjbRJcLkEBsdgXw llk0P2jAFxxM9WiHb2tHQj5cbK1AtVNLgWiD6ADNHEQxtpiUfgkQwexY4tqejH8AmDsD AQ== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxraucq5v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:48 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE1mT8024355 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:48 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:46 -0800 From: Elliot Berman To: Bjorn Andersson , Catalin Marinas , Will Deacon , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Jassi Brar" , Sudeep Holla , "Mark Rutland" , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 08/20] virt: gunyah: msgq: Add hypercalls to send and receive messages Date: Mon, 21 Nov 2022 05:59:57 -0800 Message-ID: <20221121140009.2353512-9-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: su16h1UgvQxX9to2T2Ro0crA6zCQ78HU X-Proofpoint-ORIG-GUID: su16h1UgvQxX9to2T2Ro0crA6zCQ78HU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 adultscore=0 mlxlogscore=999 priorityscore=1501 suspectscore=0 phishscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Add hypercalls to send and receive messages on a Gunyah message queue. Signed-off-by: Elliot Berman --- arch/arm64/gunyah/gunyah_hypercall.c | 33 ++++++++++++++++++++++++++++ include/linux/gunyah.h | 5 +++++ 2 files changed, 38 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index 0beb3123d650..d28600909be4 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -26,6 +26,8 @@ GH_FN_ID(GH_CALL_TYPE_HYPERCALL, fn)) #define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000) +#define GH_HYPERCALL_MSGQ_SEND GH_HYPERCALL(0x001B) +#define GH_HYPERCALL_MSGQ_RECV GH_HYPERCALL(0x001C) /** * gh_hypercall_get_uid() - Returns a UID when running under a Gunyah hypervisor @@ -65,5 +67,36 @@ void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identi } EXPORT_SYMBOL_GPL(gh_hypercall_hyp_identify); +int gh_hypercall_msgq_send(u64 capid, size_t size, uintptr_t buff, int tx_flags, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_MSGQ_SEND, capid, size, buff, tx_flags, 0, &res); + + if (res.a0) + return res.a0; + + *ready = res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gh_hypercall_msgq_send); + +int gh_hypercall_msgq_recv(u64 capid, uintptr_t buff, size_t size, size_t *recv_size, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_MSGQ_RECV, capid, buff, size, 0, &res); + + if (res.a0) + return res.a0; + + *recv_size = res.a1; + *ready = res.a2; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gh_hypercall_msgq_recv); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 166156f69df9..c863cac4a3cf 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -102,4 +102,9 @@ static inline u16 gh_api_version(void) void gh_hypercall_get_uid(u32 uid[4]); void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity); +#define GH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) + +int gh_hypercall_msgq_send(u64 capid, size_t size, uintptr_t buff, int tx_flags, bool *ready); +int gh_hypercall_msgq_recv(u64 capid, uintptr_t buff, size_t size, size_t *recv_size, bool *ready); + #endif From patchwork Mon Nov 21 13:59:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25D6BC43217 for ; Mon, 21 Nov 2022 14:02:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231533AbiKUOCN (ORCPT ); Mon, 21 Nov 2022 09:02:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231623AbiKUOBw (ORCPT ); Mon, 21 Nov 2022 09:01:52 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C6B51C428; Mon, 21 Nov 2022 06:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039311; x=1700575311; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9wqmO7jPqNqSjPOTZene9gcgdP0G34nPdRzWf1dAuV4=; b=NTcLTyEypKCC3VL1cH+BcaSvw/o1s7ma4b98DUJOfqTAoFDZzl8ahG3E Nt9oWsRqQrX044ctbfs40B3DvqB5xX6C5TUEh//S7/tKwC7TASYSgRGve U6lzsEVOB1N7bZe1/FKIy+0aKkH2WMjlpXNLHoZjeXtp3lWr4dJfNIf9a k=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 21 Nov 2022 06:01:51 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:01:51 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:49 -0800 From: Elliot Berman To: Bjorn Andersson , Jassi Brar , Elliot Berman , "Murali Nalajala" , Jonathan Corbet CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Catalin Marinas , Will Deacon , Sudeep Holla , Mark Rutland , "Lorenzo Pieralisi" , , , , , , Subject: [PATCH v7 09/20] mailbox: Add Gunyah message queue mailbox Date: Mon, 21 Nov 2022 05:59:58 -0800 Message-ID: <20221121140009.2353512-10-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Gunyah message queues are a unidirectional inter-VM pipe for messages up to 1024 bytes. This driver supports pairing a receiver message queue and a transmitter message queue to expose a single mailbox channel. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/message-queue.rst | 8 + MAINTAINERS | 1 + drivers/mailbox/Kconfig | 10 + drivers/mailbox/Makefile | 2 + drivers/mailbox/gunyah-msgq.c | 229 ++++++++++++++++++++ include/linux/gunyah.h | 61 +++++- 6 files changed, 309 insertions(+), 2 deletions(-) create mode 100644 drivers/mailbox/gunyah-msgq.c diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst index be4ab289236a..9245dbe6dd8a 100644 --- a/Documentation/virt/gunyah/message-queue.rst +++ b/Documentation/virt/gunyah/message-queue.rst @@ -54,3 +54,11 @@ vIRQ: two TX message queues will have two vIRQs (and two capability IDs). | | | | | | | | | | | | +---------------+ +-----------------+ +---------------+ + +Gunyah message queues are exposed as mailboxes. To create the mailbox, create +a mbox_client and call `gh_msgq_init`. On receipt of the RX_READY interrupt, +all messages in the RX message queue are read and pushed via the `rx_callback` +of the registered mbox_client. + +.. kernel-doc:: drivers/mailbox/gunyah-msgq.c + :identifiers: gh_msgq_init diff --git a/MAINTAINERS b/MAINTAINERS index 69c1835b3bb4..502798197b80 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8946,6 +8946,7 @@ S: Supported F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/gunyah/ +F: drivers/mailbox/gunyah-msgq.c F: drivers/virt/gunyah/ F: include/linux/gunyah.h diff --git a/drivers/mailbox/Kconfig b/drivers/mailbox/Kconfig index 05d6fae800e3..baf9451c5f04 100644 --- a/drivers/mailbox/Kconfig +++ b/drivers/mailbox/Kconfig @@ -41,6 +41,16 @@ config IMX_MBOX help Mailbox implementation for i.MX Messaging Unit (MU). +config GUNYAH_MESSAGE_QUEUES + tristate "Gunyah Message Queue Mailbox" + depends on GUNYAH + help + Mailbox implementation for Gunyah Message Queues. Gunyah message queues + are an IPC mechanism to pass short messages between virtual machines + running under the Gunyah hypervisor. + + Say Y here if you run Linux as a Gunyah virtual machine. + config PLATFORM_MHU tristate "Platform MHU Mailbox" depends on OF diff --git a/drivers/mailbox/Makefile b/drivers/mailbox/Makefile index fc9376117111..5f929bb55e9a 100644 --- a/drivers/mailbox/Makefile +++ b/drivers/mailbox/Makefile @@ -55,6 +55,8 @@ obj-$(CONFIG_MTK_CMDQ_MBOX) += mtk-cmdq-mailbox.o obj-$(CONFIG_ZYNQMP_IPI_MBOX) += zynqmp-ipi-mailbox.o +obj-$(CONFIG_GUNYAH) += gunyah-msgq.o + obj-$(CONFIG_SUN6I_MSGBOX) += sun6i-msgbox.o obj-$(CONFIG_SPRD_MBOX) += sprd-mailbox.o diff --git a/drivers/mailbox/gunyah-msgq.c b/drivers/mailbox/gunyah-msgq.c new file mode 100644 index 000000000000..d5a324e85428 --- /dev/null +++ b/drivers/mailbox/gunyah-msgq.c @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#define mbox_chan_to_msgq(chan) (container_of(chan->mbox, struct gh_msgq, mbox)) + +static inline bool gh_msgq_has_tx(struct gh_msgq *msgq) +{ + return msgq->tx_ghrsc->type == GUNYAH_RESOURCE_TYPE_MSGQ_TX; +} + +static inline bool gh_msgq_has_rx(struct gh_msgq *msgq) +{ + return msgq->rx_ghrsc->type == GUNYAH_RESOURCE_TYPE_MSGQ_RX; +} + +static irqreturn_t gh_msgq_rx_irq_handler(int irq, void *data) +{ + struct gh_msgq *msgq = data; + struct gh_msgq_rx_data rx_data; + unsigned long gh_err; + ssize_t ret; + bool ready = false; + + do { + gh_err = gh_hypercall_msgq_recv(msgq->rx_ghrsc->capid, + (uintptr_t)&rx_data.data, sizeof(rx_data.data), + &rx_data.length, &ready); + if (gh_err == GH_ERROR_OK) { + mbox_chan_received_data(gh_msgq_chan(msgq), &rx_data); + } else if (GH_ERROR_MSGQUEUE_EMPTY) { + break; + } else { + pr_warn("Failed to receive data from msgq for %s: %ld\n", + msgq->mbox.dev ? dev_name(msgq->mbox.dev) : "", ret); + break; + } + } while (ready); + + return IRQ_HANDLED; +} + +static irqreturn_t gh_msgq_tx_irq_handler(int irq, void *data) +{ + struct gh_msgq *msgq = data; + + mbox_chan_txdone(gh_msgq_chan(msgq), 0); + + return IRQ_HANDLED; +} + +static void gh_msgq_txdone_tasklet(struct tasklet_struct *tasklet) +{ + struct gh_msgq *msgq = container_of(tasklet, struct gh_msgq, txdone_tasklet); + + mbox_chan_txdone(gh_msgq_chan(msgq), msgq->last_status); +} + +static int gh_msgq_send_data(struct mbox_chan *chan, void *data) +{ + struct gh_msgq *msgq = mbox_chan_to_msgq(chan); + struct gh_msgq_tx_data *msgq_data = data; + u64 tx_flags = 0; + unsigned long ret; + bool ready; + + if (msgq_data->push) + tx_flags |= GH_HYPERCALL_MSGQ_TX_FLAGS_PUSH; + + ret = gh_hypercall_msgq_send(msgq->tx_ghrsc->capid, msgq_data->length, + (uintptr_t)msgq_data->data, tx_flags, &ready); + + /** + * unlikely because Linux tracks state of msgq and should not try to + * send message when msgq is full. + */ + if (unlikely(ret == GH_ERROR_MSGQUEUE_FULL)) + return -EAGAIN; + + /** + * Propagate all other errors to client. If we return error to mailbox + * framework, then no other messages can be sent and nobody will know + * to retry this message. + */ + msgq->last_status = gh_remap_error(ret); + + /** + * This message was successfully sent, but message queue isn't ready to + * receive more messages because it's now full. Mailbox framework + * requires that we only report that message was transmitted only when + * we're ready to transmit another message. We'll get that in the form + * of tx IRQ once the other side starts to drain the msgq. + */ + if (ret == GH_ERROR_OK && !ready) + return 0; + + /** + * We can send more messages. Mailbox framework requires that tx done + * happens asynchronously to sending the message. Gunyah message queues + * tell us right away on the hypercall return whether we can send more + * messages. To work around this, defer the txdone to a tasklet. + */ + tasklet_schedule(&msgq->txdone_tasklet); + + return 0; +} + +struct mbox_chan_ops gh_msgq_ops = { + .send_data = gh_msgq_send_data, +}; + +/** + * gh_msgq_init() - Initialize a Gunyah message queue with an mbox_client + * @parent: optional, device parent used for the mailbox controller + * @msgq: Pointer to the gh_msgq to initialize + * @cl: A mailbox client to bind to the mailbox channel that the message queue creates + * @tx_ghrsc: optional, the transmission side of the message queue + * @rx_ghrsc: optional, the receiving side of the message queue + * + * At least one of tx_ghrsc and rx_ghrsc should be not NULL. Most message queue use cases come with + * a pair of message queues to facilitiate bidirectional communication. When tx_ghrsc is set, + * the client can send messages with mbox_send_message(gh_msgq_chan(msgq), msg). When rx_ghrsc + * is set, the mbox_client should register an .rx_callback() and the message queue driver will + * push all available messages upon receiving the RX ready interrupt. The messages should be + * consumed or copied by the client right away as the gh_msgq_rx_data will be replaced/destroyed + * after the callback. + * + * Returns - 0 on success, negative otherwise + */ +int gh_msgq_init(struct device *parent, struct gh_msgq *msgq, struct mbox_client *cl, + struct gunyah_resource *tx_ghrsc, struct gunyah_resource *rx_ghrsc) +{ + int ret; + + /* Must have at least a tx_ghrsc or rx_ghrsc and that they are the right device types */ + if ((!tx_ghrsc && !rx_ghrsc) || + (tx_ghrsc && tx_ghrsc->type != GUNYAH_RESOURCE_TYPE_MSGQ_TX) || + (rx_ghrsc && rx_ghrsc->type != GUNYAH_RESOURCE_TYPE_MSGQ_RX)) + return -EINVAL; + + msgq->tx_ghrsc = tx_ghrsc; + msgq->rx_ghrsc = rx_ghrsc; + + msgq->mbox.dev = parent; + msgq->mbox.ops = &gh_msgq_ops; + msgq->mbox.num_chans = 1; + msgq->mbox.chans = kcalloc(msgq->mbox.num_chans, sizeof(*msgq->mbox.chans), GFP_KERNEL); + if (!msgq->mbox.chans) + return -ENOMEM; + msgq->mbox.txdone_irq = true; + + if (gh_msgq_has_tx(msgq)) { + ret = request_irq(msgq->tx_ghrsc->irq, gh_msgq_tx_irq_handler, 0, "gh_msgq_tx", + msgq); + if (ret) + goto err_chans; + } + + if (gh_msgq_has_rx(msgq)) { + ret = request_threaded_irq(msgq->rx_ghrsc->irq, NULL, gh_msgq_rx_irq_handler, + IRQF_ONESHOT, "gh_msgq_rx", msgq); + if (ret) + goto err_tx_irq; + } + + tasklet_setup(&msgq->txdone_tasklet, gh_msgq_txdone_tasklet); + + ret = mbox_controller_register(&msgq->mbox); + if (ret) + goto err_rx_irq; + + ret = mbox_bind_client(gh_msgq_chan(msgq), cl); + if (ret) + goto err_mbox; + + return 0; +err_mbox: + mbox_controller_unregister(&msgq->mbox); +err_rx_irq: + if (gh_msgq_has_rx(msgq)) + free_irq(msgq->rx_ghrsc->irq, msgq); +err_tx_irq: + if (gh_msgq_has_tx(msgq)) + free_irq(msgq->tx_ghrsc->irq, msgq); +err_chans: + kfree(msgq->mbox.chans); + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_init); + +void gh_msgq_remove(struct gh_msgq *msgq) +{ + mbox_controller_unregister(&msgq->mbox); + + if (gh_msgq_has_rx(msgq)) + free_irq(msgq->rx_ghrsc->irq, msgq); + + if (gh_msgq_has_tx(msgq)) + free_irq(msgq->tx_ghrsc->irq, msgq); + + kfree(msgq->mbox.chans); +} +EXPORT_SYMBOL_GPL(gh_msgq_remove); + + +static int __init gh_msgq_init_module(void) +{ + if (gh_api_version() != GUNYAH_API_V1) { + pr_warn("Unrecognized gunyah version: %u. Currently supported: %d\n", + gh_api_version(), GUNYAH_API_V1); + return -ENODEV; + } + + return 0; +} +module_init(gh_msgq_init_module); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Message Queue Driver"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index c863cac4a3cf..e317d7ac938f 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -7,10 +7,67 @@ #define _GUNYAH_H #include -#include #include +#include +#include +#include +#include + +/* Follows resource manager's resource types for VM_GET_HYP_RESOURCES */ +enum gunyah_resource_type { + GUNYAH_RESOURCE_TYPE_BELL_TX = 0, + GUNYAH_RESOURCE_TYPE_BELL_RX = 1, + GUNYAH_RESOURCE_TYPE_MSGQ_TX = 2, + GUNYAH_RESOURCE_TYPE_MSGQ_RX = 3, + GUNYAH_RESOURCE_TYPE_VCPU = 4, +}; + +struct gunyah_resource { + enum gunyah_resource_type type; + u64 capid; + int irq; +}; + +/** + * Gunyah Message Queues + */ + +#define GH_MSGQ_MAX_MSG_SIZE 240 + +struct gh_msgq_tx_data { + size_t length; + bool push; + char data[]; +}; + +struct gh_msgq_rx_data { + size_t length; + char data[GH_MSGQ_MAX_MSG_SIZE]; +}; + +struct gh_msgq { + struct gunyah_resource *tx_ghrsc; + struct gunyah_resource *rx_ghrsc; + + /* msgq private */ + int last_status; + struct mbox_controller mbox; + struct tasklet_struct txdone_tasklet; +}; + + +int gh_msgq_init(struct device *parent, struct gh_msgq *msgq, struct mbox_client *cl, + struct gunyah_resource *tx_ghrsc, struct gunyah_resource *rx_ghrsc); +void gh_msgq_remove(struct gh_msgq *msgq); + +static inline struct mbox_chan *gh_msgq_chan(struct gh_msgq *msgq) +{ + return &msgq->mbox.chans[0]; +} + +/******************************************************************************/ +/* Common arch-independent macros and definitions for Gunyah hypercalls */ -/* Common Gunyah macros */ #define GH_CAPID_INVAL U64_MAX #define GH_VMID_ROOT_VM 0xff From patchwork Mon Nov 21 13:59:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627282 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C2D9C433FE for ; Mon, 21 Nov 2022 14:04:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231135AbiKUOEO (ORCPT ); Mon, 21 Nov 2022 09:04:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231596AbiKUOCT (ORCPT ); Mon, 21 Nov 2022 09:02:19 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD60C9C7A2; Mon, 21 Nov 2022 06:02:17 -0800 (PST) Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALDv8Yr003808; Mon, 21 Nov 2022 14:01:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=WNgw4FajQnRSTHtH7PHEphfGrA1mFv9j2hiO8jbUXvY=; b=KRuLjrn3/vyPkBz9v0QIZyggSO+FJiXyBcTTOXQnGfVYIyT+N5Wclvd9cWsA7xo6lQK6 CTzCP+QNGjzrBqvRYHnwxDYhWMrXfAv6RWfnevZuXQRJkL/Sd4Z58+ciWyLICUicO9Pq JZe4sBYNV0fRYBccrwqOjfvXEXtQ/6GQ8PFM8S4oyX1xegRP8+LIAd30FHO7orQPZTs2 BN3ZY2hwJN1HhOlGdT3tKqi8pBPbPoGtDERT0HhLHl+xSAaimKvMyt0Lv/o+qCGfj4W9 WvGiWtH/r9Ks8C89ahdSEK+y/YbU3GgIq0xDBDyClGwmE4Jky6Gx/wdoAZyjX5u9V668 IQ== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrpb4j6d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:56 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE1sRl014812 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:01:54 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:52 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 10/20] gunyah: rsc_mgr: Add resource manager RPC core Date: Mon, 21 Nov 2022 05:59:59 -0800 Message-ID: <20221121140009.2353512-11-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: RnFrEbPJ5RqcXS-N6iGyIykwt-GOjhbC X-Proofpoint-GUID: RnFrEbPJ5RqcXS-N6iGyIykwt-GOjhbC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 mlxscore=0 lowpriorityscore=0 suspectscore=0 bulkscore=0 malwarescore=0 priorityscore=1501 mlxlogscore=999 phishscore=0 clxscore=1015 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org The resource manager is a special virtual machine which is always running on a Gunyah system. It provides APIs for creating and destroying VMs, secure memory management, sharing/lending of memory between VMs, and setup of inter-VM communication. Calls to the resource manager are made via message queues. This patch implements the basic probing and RPC mechanism to make those API calls. Request/response calls can be made with gh_rm_call. Drivers can also register to notifications pushed by RM via gh_rm_register_notifier Specific API calls that resource manager supports will be implemented in subsequent patches. Signed-off-by: Elliot Berman --- MAINTAINERS | 2 +- drivers/virt/gunyah/Kconfig | 7 + drivers/virt/gunyah/Makefile | 2 + drivers/virt/gunyah/gunyah_rm_rpc.c | 570 +++++++++++++++++++++++++++ drivers/virt/gunyah/gunyah_rsc_mgr.c | 50 +++ drivers/virt/gunyah/rsc_mgr.h | 37 ++ include/linux/gunyah_rsc_mgr.h | 18 + 7 files changed, 685 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/gunyah_rm_rpc.c create mode 100644 drivers/virt/gunyah/gunyah_rsc_mgr.c create mode 100644 drivers/virt/gunyah/rsc_mgr.h create mode 100644 include/linux/gunyah_rsc_mgr.h diff --git a/MAINTAINERS b/MAINTAINERS index 502798197b80..b65f7ff444e5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8948,7 +8948,7 @@ F: Documentation/virt/gunyah/ F: arch/arm64/gunyah/ F: drivers/mailbox/gunyah-msgq.c F: drivers/virt/gunyah/ -F: include/linux/gunyah.h +F: include/linux/gunyah*.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 127156a678a6..0bb497372d4e 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -10,3 +10,10 @@ config GUNYAH Say Y/M here to enable the drivers needed to interact in a Gunyah virtual environment. + +if GUNYAH +config GUNYAH_RESOURCE_MANAGER + tristate + depends on MAILBOX + select GUNYAH_MESSAGE_QUEUES +endif diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 2ac4ee64b89d..b62ac4045621 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1 +1,3 @@ obj-$(CONFIG_GUNYAH) += gunyah.o + +obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER) += gunyah_rsc_mgr.o gunyah_rm_rpc.o diff --git a/drivers/virt/gunyah/gunyah_rm_rpc.c b/drivers/virt/gunyah/gunyah_rm_rpc.c new file mode 100644 index 000000000000..45b1a8691982 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_rm_rpc.c @@ -0,0 +1,570 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rsc_mgr.h" + +/* Resource Manager Header */ +struct gh_rm_rpc_hdr { +#define RM_RPC_HDR_VERSION_ONE 0x1 +#define RM_RPC_API_VERSION_MASK GENMASK(3, 0) + +#define RM_RPC_HDR_WORDS 0x2 +#define RM_RPC_HEADER_WORDS_MASK GENMASK(7, 4) + u8 api; + +#define RM_RPC_TYPE_CONT 0x0 +#define RM_RPC_TYPE_REQ 0x1 +#define RM_RPC_TYPE_RPLY 0x2 +#define RM_RPC_TYPE_NOTIF 0x3 +#define RM_RPC_TYPE_MASK GENMASK(1, 0) + +#define GH_RM_MAX_NUM_FRAGMENTS 62 +#define RM_RPC_FRAGMENTS_MASK GENMASK(7, 2) + u8 type; + __le16 seq; + __le32 msg_id; +} __packed; + +/* Standard reply header */ +struct gh_rm_rpc_reply_hdr { + struct gh_rm_rpc_hdr rpc_hdr; + u32 err_code; +} __packed; + +#define GH_RM_MAX_MSG_SIZE (GH_MSGQ_MAX_MSG_SIZE - sizeof(struct gh_rm_rpc_hdr)) + +/** + * struct gh_rm_connection - Represents a complete message from resource manager + * @payload: Combined payload of all the fragments (msg headers stripped off). + * @size: Size of the payload. + * @ret: Linux return code, set in case there was an error processing connection + * @msg_id: Message ID from the header. + * @type: RM_RPC_TYPE_RPLY or RM_RPC_TYPE_NOTIF. + * @num_fragments: total number of fragments expected to be received. + * @fragments_received: fragments received so far. + * @rm_error: For request/reply sequences with standard replies. + * @seq: Sequence ID for the main message. + * @seq_done: Signals caller that the RM reply has been received + */ +struct gh_rm_connection { + void *payload; + size_t size; + int ret; + u32 msg_id; + u8 type; + + u8 num_fragments; + u8 fragments_received; + + /* only for req/reply sequence */ + u32 rm_error; + u16 seq; + struct completion seq_done; +}; + +struct gh_rm_notif_complete { + struct gh_rm_connection *conn; + struct work_struct work; +}; + +struct gh_rm_rpc { + struct device *dev; + struct gunyah_resource tx_ghrsc, rx_ghrsc; + struct gh_msgq msgq; + struct mbox_client msgq_client; + struct gh_rm_connection *active_rx_connection; + int last_tx_ret; + + struct idr call_idr; + struct mutex call_idr_lock; + + struct mutex send_lock; + + struct work_struct recv_work; +}; + +static struct gh_rm_connection *gh_rm_alloc_connection(u32 msg_id, u8 type) +{ + struct gh_rm_connection *connection; + + connection = kzalloc(sizeof(*connection), GFP_KERNEL); + if (!connection) + return NULL; + + connection->type = type; + connection->msg_id = msg_id; + + return connection; +} + +static int gh_rm_init_connection_payload(struct gh_rm_connection *connection, void *msg, + size_t hdr_size, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t max_buf_size, payload_size; + + if (hdr_size > msg_size) + return -EINVAL; + + payload_size = msg_size - hdr_size; + + connection->num_fragments = FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type); + connection->fragments_received = 0; + connection->type = hdr->type; + + /* There's not going to be any payload, no need to allocate buffer. */ + if (!payload_size && !connection->num_fragments) + return 0; + + /* + * maximum payload size is GH_MSGQ_MAX_MSG_SIZE - hdr_size + * and can received (hdr->fragments + 1) of those + */ + max_buf_size = (GH_MSGQ_MAX_MSG_SIZE - hdr_size) * (connection->num_fragments + 1); + + connection->payload = kzalloc(max_buf_size, GFP_KERNEL); + if (!connection->payload) + return -ENOMEM; + + memcpy(connection->payload, msg + hdr_size, payload_size); + connection->size = payload_size; + return 0; +} + +static void gh_rm_notif_work(struct work_struct *work) +{ + struct gh_rm_notif_complete *notif = container_of(work, struct gh_rm_notif_complete, work); + struct gh_rm_connection *connection = notif->conn; + + /* No users of notifications, yet. */ + + kfree(connection->payload); + kfree(connection); + kfree(notif); +} + +static struct gh_rm_connection *gh_rm_process_notif(struct gh_rm_rpc *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + + connection = gh_rm_alloc_connection(le32_to_cpu(hdr->msg_id), + FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)); + if (!connection) { + dev_err(rsc_mgr->dev, "Failed to alloc connection for notification, dropping.\n"); + return NULL; + } + + if (gh_rm_init_connection_payload(connection, msg, sizeof(*hdr), msg_size)) { + dev_err(rsc_mgr->dev, "Failed to alloc connection buffer for notification, dropping.\n"); + kfree(connection); + return NULL; + } + + return connection; +} + +static struct gh_rm_connection *gh_rm_process_rply(struct gh_rm_rpc *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_reply_hdr *reply_hdr = msg; + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + u16 seq_id = le16_to_cpu(hdr->seq); + + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) + return ERR_PTR(-ERESTARTSYS); + + connection = idr_find(&rsc_mgr->call_idr, seq_id); + mutex_unlock(&rsc_mgr->call_idr_lock); + + if (!connection) { + dev_err(rsc_mgr->dev, "Failed to find connection for sequence %u\n", seq_id); + return NULL; + } + if (connection->msg_id != le32_to_cpu(hdr->msg_id)) { + dev_err(rsc_mgr->dev, "Reply for sequence %u expected msg_id: %x but got %x\n", + seq_id, connection->msg_id, le32_to_cpu(hdr->msg_id)); + /* + * Don't complete connection and error the client, maybe + * resource manager will send us the expected reply sequence soon. + */ + return NULL; + } + + if (gh_rm_init_connection_payload(connection, msg, sizeof(*reply_hdr), msg_size)) { + dev_err(rsc_mgr->dev, "Failed to alloc connection buffer for sequence %d\n", + seq_id); + /* Send connection complete and error the client. */ + connection->ret = -ENOMEM; + complete(&connection->seq_done); + return NULL; + } + + connection->rm_error = reply_hdr->err_code; + return connection; +} + +static void gh_rm_process_cont(struct gh_rm_rpc *rsc_mgr, struct gh_rm_connection *connection, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t payload_size = msg_size - sizeof(*hdr); + + /* + * hdr->fragments and hdr->msg_id preserves the value from first reply + * or notif message. To detect mishandling, check it's still intact. + */ + if (connection->msg_id != le32_to_cpu(hdr->msg_id)) + dev_err(rsc_mgr->dev, "Appending mismatched continuation with id %d to connection with id %d\n", + le32_to_cpu(hdr->msg_id), connection->msg_id); + if (connection->num_fragments != FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type)) + dev_err(rsc_mgr->dev, "Number of fragments mismatch for seq: %d\n", + le16_to_cpu(hdr->seq)); + + memcpy(connection->payload + connection->size, msg + sizeof(*hdr), payload_size); + connection->size += payload_size; + connection->fragments_received++; +} + +static bool gh_rm_complete_connection(struct gh_rm_rpc *rsc_mgr, + struct gh_rm_connection *connection) +{ + struct gh_rm_notif_complete *notif_work; + + if (!connection) + return false; + + if (connection->fragments_received != connection->num_fragments) + return false; + + switch (connection->type) { + case RM_RPC_TYPE_RPLY: + complete(&connection->seq_done); + break; + case RM_RPC_TYPE_NOTIF: + notif_work = kzalloc(sizeof(*notif_work), GFP_KERNEL); + if (notif_work == NULL) + break; + + notif_work->conn = connection; + INIT_WORK(¬if_work->work, gh_rm_notif_work); + + schedule_work(¬if_work->work); + break; + default: + dev_err(rsc_mgr->dev, "Invalid message type (%d) received\n", connection->type); + break; + } + + return true; +} + +static void gh_rm_abort_connection(struct gh_rm_connection *connection) +{ + switch (connection->type) { + case RM_RPC_TYPE_RPLY: + connection->ret = -EIO; + complete(&connection->seq_done); + break; + case RM_RPC_TYPE_NOTIF: + fallthrough; + default: + kfree(connection->payload); + kfree(connection); + } +} + +static void gh_rm_msgq_rx_data(struct mbox_client *cl, void *mssg) +{ + struct gh_rm_rpc *rsc_mgr = container_of(cl, struct gh_rm_rpc, msgq_client); + struct gh_msgq_rx_data *rx_data = mssg; + void *msg = rx_data->data; + size_t msg_size = rx_data->length; + struct gh_rm_rpc_hdr *hdr; + + if (msg_size <= sizeof(struct gh_rm_rpc_hdr)) { + dev_err(rsc_mgr->dev, "Incomplete message size: %ld is too small\n", msg_size); + return; + } + + hdr = msg; + switch (FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)) { + case RM_RPC_TYPE_NOTIF: + if (rsc_mgr->active_rx_connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + dev_err(rsc_mgr->dev, "Received start of new notification without finishing existing message series.\n"); + gh_rm_abort_connection(rsc_mgr->active_rx_connection); + } + rsc_mgr->active_rx_connection = gh_rm_process_notif(rsc_mgr, msg, msg_size); + break; + case RM_RPC_TYPE_RPLY: + if (rsc_mgr->active_rx_connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + dev_err(rsc_mgr->dev, "Received start of new reply without finishing existing message series.\n"); + gh_rm_abort_connection(rsc_mgr->active_rx_connection); + } + rsc_mgr->active_rx_connection = gh_rm_process_rply(rsc_mgr, msg, msg_size); + break; + case RM_RPC_TYPE_CONT: + if (!rsc_mgr->active_rx_connection) { + dev_err(rsc_mgr->dev, "Received a continuation message without receiving initial message\n"); + break; + } + gh_rm_process_cont(rsc_mgr, rsc_mgr->active_rx_connection, msg, msg_size); + break; + default: + dev_err(rsc_mgr->dev, "Invalid message type (%lu) received\n", + FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)); + return; + } + + if (gh_rm_complete_connection(rsc_mgr, rsc_mgr->active_rx_connection)) + rsc_mgr->active_rx_connection = NULL; +} + +static void gh_rm_msgq_tx_done(struct mbox_client *cl, void *mssg, int r) +{ + struct gh_rm_rpc *rsc_mgr = container_of(cl, struct gh_rm_rpc, msgq_client); + + kfree(mssg); + rsc_mgr->last_tx_ret = r; +} + +static int gh_rm_send_request(struct gh_rm_rpc *rsc_mgr, u32 message_id, + const void *req_buff, size_t req_buff_size, + struct gh_rm_connection *connection) +{ + size_t buff_size_remaining = req_buff_size; + const void *req_buff_curr = req_buff; + struct gh_rm_rpc_hdr *hdr; + u32 cont_fragments = 0; + size_t payload_size; + struct gh_msgq_tx_data *msg; + int i, ret; + + if (req_buff_size) + cont_fragments = (req_buff_size - 1) / GH_RM_MAX_MSG_SIZE; + + if (WARN(cont_fragments > GH_RM_MAX_NUM_FRAGMENTS, + "Limit exceeded for the number of fragments: %u\n", cont_fragments)) + return -E2BIG; + + ret = mutex_lock_interruptible(&rsc_mgr->send_lock); + if (ret) + return ret; + + /* Consider also the 'request' packet for the loop count */ + for (i = 0; i <= cont_fragments; i++) { + if (buff_size_remaining > GH_RM_MAX_MSG_SIZE) { + payload_size = GH_RM_MAX_MSG_SIZE; + buff_size_remaining -= payload_size; + } else { + payload_size = buff_size_remaining; + } + + msg = kzalloc(sizeof(*msg) + GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL); + if (!msg) { + ret = -ENOMEM; + goto out; + } + + /* Fill header */ + hdr = (struct gh_rm_rpc_hdr *)msg->data; + hdr->api = FIELD_PREP(RM_RPC_API_VERSION_MASK, RM_RPC_HDR_VERSION_ONE) | + FIELD_PREP(RM_RPC_HEADER_WORDS_MASK, RM_RPC_HDR_WORDS); + hdr->type = FIELD_PREP(RM_RPC_TYPE_MASK, + i == 0 ? RM_RPC_TYPE_REQ : RM_RPC_TYPE_CONT) | + FIELD_PREP(RM_RPC_FRAGMENTS_MASK, cont_fragments); + hdr->seq = cpu_to_le16(connection->seq); + hdr->msg_id = cpu_to_le32(message_id); + + /* Copy payload */ + memcpy(msg->data + sizeof(*hdr), req_buff_curr, payload_size); + req_buff_curr += payload_size; + + /* Force the last fragment to immediately alert the receiver */ + msg->push = i == cont_fragments; + msg->length = sizeof(*hdr) + payload_size; + + ret = mbox_send_message(gh_msgq_chan(&rsc_mgr->msgq), msg); + if (ret < 0) { + kfree(msg); + break; + } + + if (rsc_mgr->last_tx_ret) { + ret = rsc_mgr->last_tx_ret; + break; + } + } + +out: + mutex_unlock(&rsc_mgr->send_lock); + return ret < 0 ? ret : 0; +} + +/** + * gh_rm_call: Achieve request-response type communication with RPC + * @message_id: The RM RPC message-id + * @req_buff: Request buffer that contains the payload + * @req_buff_size: Total size of the payload + * @resp_buf: Pointer to a response buffer + * @resp_buff_size: Size of the response buffer + * + * Make a request to the RM-VM and wait for reply back. For a successful + * response, the function returns the payload. The size of the payload is set in + * resp_buff_size. The resp_buf should be freed by the caller. + * + * Context: Process context. Will sleep waiting for reply. + * Return: >0 is standard reply error from RM. <0 on internal error. + */ +int gh_rm_call(struct gh_rm_rpc *rsc_mgr, u32 message_id, void *req_buff, size_t req_buff_size, + void **resp_buf, size_t *resp_buff_size) +{ + struct gh_rm_connection *connection; + int ret; + + /* messaged_id 0 is reserved */ + if (!message_id) + return -EINVAL; + + if (!rsc_mgr) + return -EPROBE_DEFER; + + connection = gh_rm_alloc_connection(message_id, RM_RPC_TYPE_RPLY); + if (!connection) + return -ENOMEM; + + init_completion(&connection->seq_done); + + /* Allocate a new seq number for this connection */ + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) { + kfree(connection); + return -ERESTARTSYS; + } + connection->seq = idr_alloc_cyclic(&rsc_mgr->call_idr, connection, 0, U16_MAX, GFP_KERNEL); + mutex_unlock(&rsc_mgr->call_idr_lock); + + /* Send the request to the Resource Manager */ + ret = gh_rm_send_request(rsc_mgr, message_id, req_buff, req_buff_size, connection); + if (ret < 0) + goto out; + + /* Wait for response */ + ret = wait_for_completion_interruptible(&connection->seq_done); + if (ret) + goto out; + + if (connection->ret) { + ret = connection->ret; + kfree(connection->payload); + goto out; + } + + if (connection->rm_error) { + ret = connection->rm_error; + kfree(connection->payload); + goto out; + } + + *resp_buf = connection->payload; + *resp_buff_size = connection->size; + +out: + mutex_lock(&rsc_mgr->call_idr_lock); + idr_remove(&rsc_mgr->call_idr, connection->seq); + mutex_unlock(&rsc_mgr->call_idr_lock); + + kfree(connection); + return ret; +} + +static int gh_msgq_platform_probe_direction(struct platform_device *pdev, + u8 gh_type, int idx, struct gunyah_resource *ghrsc) +{ + int ret; + struct device_node *node = pdev->dev.of_node; + + ghrsc->type = gh_type; + + ghrsc->irq = platform_get_irq(pdev, idx); + if (ghrsc->irq < 0) { + dev_err(&pdev->dev, "Failed to get irq%d: %d\n", idx, ghrsc->irq); + return ghrsc->irq; + } + + ret = of_property_read_u64_index(node, "reg", idx, &ghrsc->capid); + if (ret) { + dev_err(&pdev->dev, "Failed to get capid%d: %d\n", idx, ret); + return ret; + } + + return 0; +} + +struct gh_rm_rpc *gh_rm_rpc_init(struct platform_device *pdev) +{ + struct gh_rm_rpc *rsc_mgr; + int ret; + + rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); + if (!rsc_mgr) + return ERR_PTR(-ENOMEM); + + rsc_mgr->dev = &pdev->dev; + + mutex_init(&rsc_mgr->call_idr_lock); + idr_init(&rsc_mgr->call_idr); + mutex_init(&rsc_mgr->send_lock); + + ret = gh_msgq_platform_probe_direction(pdev, GUNYAH_RESOURCE_TYPE_MSGQ_TX, 0, + &rsc_mgr->tx_ghrsc); + if (ret) + return ERR_PTR(ret); + + ret = gh_msgq_platform_probe_direction(pdev, GUNYAH_RESOURCE_TYPE_MSGQ_RX, 1, + &rsc_mgr->rx_ghrsc); + if (ret) + return ERR_PTR(ret); + + rsc_mgr->msgq_client.dev = &pdev->dev; + rsc_mgr->msgq_client.tx_block = true; + rsc_mgr->msgq_client.rx_callback = gh_rm_msgq_rx_data; + rsc_mgr->msgq_client.tx_done = gh_rm_msgq_tx_done; + + ret = gh_msgq_init(&pdev->dev, &rsc_mgr->msgq, &rsc_mgr->msgq_client, + &rsc_mgr->tx_ghrsc, &rsc_mgr->rx_ghrsc); + if (ret) + return ERR_PTR(ret); + + return rsc_mgr; +} +EXPORT_SYMBOL_GPL(gh_rm_rpc_init); + +void gh_rm_rpc_remove(struct gh_rm_rpc *rm) +{ + mbox_free_channel(gh_msgq_chan(&rm->msgq)); + gh_msgq_remove(&rm->msgq); +} +EXPORT_SYMBOL_GPL(gh_rm_rpc_remove); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Resource Manager RPC Driver"); diff --git a/drivers/virt/gunyah/gunyah_rsc_mgr.c b/drivers/virt/gunyah/gunyah_rsc_mgr.c new file mode 100644 index 000000000000..2fae62bbfe49 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_rsc_mgr.c @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include + +#include "rsc_mgr.h" + +static int gh_rm_drv_probe(struct platform_device *pdev) +{ + struct gh_rm_rpc *rsc_mgr; + + rsc_mgr = gh_rm_rpc_init(pdev); + if (IS_ERR(rsc_mgr)) + return PTR_ERR(rsc_mgr); + + platform_set_drvdata(pdev, rsc_mgr); + + return 0; +} + +static int gh_rm_drv_remove(struct platform_device *pdev) +{ + struct gh_rm_rpc *rsc_mgr = platform_get_drvdata(pdev); + + gh_rm_rpc_remove(rsc_mgr); + + return 0; +} + +static const struct of_device_id gh_rm_of_match[] = { + { .compatible = "gunyah-resource-manager" }, + {} +}; +MODULE_DEVICE_TABLE(of, gh_rm_of_match); + +static struct platform_driver gh_rm_driver = { + .probe = gh_rm_drv_probe, + .remove = gh_rm_drv_remove, + .driver = { + .name = "gh_rsc_mgr", + .of_match_table = gh_rm_of_match, + }, +}; +module_platform_driver(gh_rm_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Resource Manager Driver"); diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h new file mode 100644 index 000000000000..01f0af888fad --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __GH_RSC_MGR_PRIV_H +#define __GH_RSC_MGR_PRIV_H + +#include + +/* RM Error codes */ +#define GH_RM_ERROR_OK 0x0 +#define GH_RM_ERROR_UNIMPLEMENTED 0xFFFFFFFF +#define GH_RM_ERROR_NOMEM 0x1 +#define GH_RM_ERROR_NORESOURCE 0x2 +#define GH_RM_ERROR_DENIED 0x3 +#define GH_RM_ERROR_INVALID 0x4 +#define GH_RM_ERROR_BUSY 0x5 +#define GH_RM_ERROR_ARGUMENT_INVALID 0x6 +#define GH_RM_ERROR_HANDLE_INVALID 0x7 +#define GH_RM_ERROR_VALIDATE_FAILED 0x8 +#define GH_RM_ERROR_MAP_FAILED 0x9 +#define GH_RM_ERROR_MEM_INVALID 0xA +#define GH_RM_ERROR_MEM_INUSE 0xB +#define GH_RM_ERROR_MEM_RELEASED 0xC +#define GH_RM_ERROR_VMID_INVALID 0xD +#define GH_RM_ERROR_LOOKUP_FAILED 0xE +#define GH_RM_ERROR_IRQ_INVALID 0xF +#define GH_RM_ERROR_IRQ_INUSE 0x10 +#define GH_RM_ERROR_IRQ_RELEASED 0x11 + +struct gh_rm_rpc; +struct platform_device; + +struct gh_rm_rpc *gh_rm_rpc_init(struct platform_device *pdev); +void gh_rm_rpc_remove(struct gh_rm_rpc *rm); + +#endif diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h new file mode 100644 index 000000000000..b4f55c19954b --- /dev/null +++ b/include/linux/gunyah_rsc_mgr.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_RSC_MGR_H +#define _GUNYAH_RSC_MGR_H + +#include +#include +#include + +#define GH_VMID_INVAL U16_MAX + +/* Gunyah recognizes VMID0 as an alias to the current VM's ID */ +#define GH_VMID_SELF 0 + +#endif From patchwork Mon Nov 21 14:00:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3D09C433FE for ; Mon, 21 Nov 2022 14:02:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231624AbiKUOC3 (ORCPT ); Mon, 21 Nov 2022 09:02:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231659AbiKUOB6 (ORCPT ); Mon, 21 Nov 2022 09:01:58 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44B149CF47; Mon, 21 Nov 2022 06:01:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039317; x=1700575317; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6FRhFwKvkVENg+VPhJgJ86aIuZqgwHwm5Um5GmXyQ8w=; b=G686h3oopT7XF6VuRArh4EinyfLCbDtQLxONL7kETZlQxtAt11VroUW/ FeJNSaa7HnoEGtZxmRbyYHN8Vq4105MaSXTvtsQxMzyVQzTFosQVrKOq9 6a4N26f8Mvy+AJlHndnFHdIZa0b9lJDQsbO40h0sfkk+Yf4UCqDKZu3oI s=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-02.qualcomm.com with ESMTP; 21 Nov 2022 06:01:56 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:01:56 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:55 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 11/20] gunyah: rsc_mgr: Add VM lifecycle RPC Date: Mon, 21 Nov 2022 06:00:00 -0800 Message-ID: <20221121140009.2353512-12-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Add Gunyah Resource Manager RPC to launch an unauthenticated VM. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/gunyah_rm_rpc.c | 217 +++++++++++++++++++++++++++- drivers/virt/gunyah/rsc_mgr.h | 46 ++++++ include/linux/gunyah_rsc_mgr.h | 58 ++++++++ 3 files changed, 319 insertions(+), 2 deletions(-) diff --git a/drivers/virt/gunyah/gunyah_rm_rpc.c b/drivers/virt/gunyah/gunyah_rm_rpc.c index 45b1a8691982..fe6ba476c4dc 100644 --- a/drivers/virt/gunyah/gunyah_rm_rpc.c +++ b/drivers/virt/gunyah/gunyah_rm_rpc.c @@ -436,8 +436,8 @@ static int gh_rm_send_request(struct gh_rm_rpc *rsc_mgr, u32 message_id, * Context: Process context. Will sleep waiting for reply. * Return: >0 is standard reply error from RM. <0 on internal error. */ -int gh_rm_call(struct gh_rm_rpc *rsc_mgr, u32 message_id, void *req_buff, size_t req_buff_size, - void **resp_buf, size_t *resp_buff_size) +static int gh_rm_call(struct gh_rm_rpc *rsc_mgr, u32 message_id, void *req_buff, + size_t req_buff_size, void **resp_buf, size_t *resp_buff_size) { struct gh_rm_connection *connection; int ret; @@ -566,5 +566,218 @@ void gh_rm_rpc_remove(struct gh_rm_rpc *rm) } EXPORT_SYMBOL_GPL(gh_rm_rpc_remove); +/* + * Several RM calls take only a VMID as a parameter and give only standard + * response back. Deduplicate boilerplate code by using this common call. + */ +static int gh_rm_common_vmid_call(struct gh_rm_rpc *rm, u32 message_id, u16 vmid) +{ + void *resp = NULL; + struct gh_vm_common_vmid_req req_payload = { + .vmid = vmid, + }; + size_t resp_size; + int ret; + + ret = gh_rm_call(rm, message_id, &req_payload, sizeof(req_payload), &resp, &resp_size); + if (!ret) + kfree(resp); + + WARN_ON(!ret && resp_size); + + return ret; +} + +/** + * gh_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM identifier. + * @vmid: Use GH_VMID_INVAL to dynamically allocate a VM. A reserved VMID can also be requested + * for a special-purpose platform-defined VM. + * + * Returns - the allocated VMID or negative value on error + */ +int gh_rm_alloc_vmid(struct gh_rm_rpc *rm, u16 vmid) +{ + void *resp; + struct gh_vm_common_vmid_req req_payload = { + .vmid = vmid, + }; + struct gh_vm_common_vmid_req *resp_payload; + size_t resp_size; + int ret; + + if (vmid == GH_VMID_INVAL) + vmid = 0; + + ret = gh_rm_call(rm, GH_RM_RPC_VM_ALLOC_VMID, &req_payload, sizeof(req_payload), &resp, + &resp_size); + if (ret) + return ret; + + if (!vmid) { + if (resp_size != sizeof(*resp_payload)) { + ret = -EINVAL; + } else { + resp_payload = resp; + ret = resp_payload->vmid; + } + } + kfree(resp); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_alloc_vmid); + +/** + * gh_rm_dealloc_vmid() - Dispose the VMID + * @vmid: VM identifier + */ +int gh_rm_dealloc_vmid(struct gh_rm_rpc *rm, u16 vmid) +{ + return gh_rm_common_vmid_call(rm, GH_RM_RPC_VM_DEALLOC_VMID, vmid); +} +EXPORT_SYMBOL_GPL(gh_rm_dealloc_vmid); + +/** + * gh_rm_vm_start() - Move the VM into "ready to run" state + * @vmid: VM identifier + * + * On VMs which use proxy scheduling, vcpu_run is needed to actually run the VM. + * On VMs which use Gunyah's scheduling, the vCPUs start executing in accordance with Gunyah + * scheduling policies. + */ +int gh_rm_vm_start(struct gh_rm_rpc *rm, u16 vmid) +{ + return gh_rm_common_vmid_call(rm, GH_RM_RPC_VM_START, vmid); +} +EXPORT_SYMBOL_GPL(gh_rm_vm_start); + +/** + * gh_rm_vm_stop() - Send a request to Resource Manager VM to stop a VM. + * @vmid: VM identifier + */ +int gh_rm_vm_stop(struct gh_rm_rpc *rm, u16 vmid) +{ + struct gh_vm_stop_req req_payload = { 0 }; + void *resp; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(rm, GH_RM_RPC_VM_STOP, &req_payload, sizeof(req_payload), &resp, &resp_size); + if (ret) + return ret; + kfree(resp); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_vm_stop); + +int gh_rm_vm_configure(struct gh_rm_rpc *rm, u16 vmid, enum gh_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, u64 dtb_offset, u64 dtb_size) +{ + struct gh_vm_config_image_req req_payload = { 0 }; + void *resp; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + req_payload.auth_mech = auth_mechanism; + req_payload.mem_handle = mem_handle; + req_payload.image_offset_low = image_offset; + req_payload.image_offset_high = image_offset >> 32; + req_payload.image_size_low = image_size; + req_payload.image_size_high = image_size >> 32; + req_payload.dtb_offset_low = dtb_offset; + req_payload.dtb_offset_high = dtb_offset >> 32; + req_payload.dtb_size_low = dtb_size; + req_payload.dtb_size_high = dtb_size >> 32; + + ret = gh_rm_call(rm, GH_RM_RPC_VM_CONFIG_IMAGE, &req_payload, sizeof(req_payload), + &resp, &resp_size); + if (ret) + return ret; + kfree(resp); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_vm_configure); + +/** + * gh_rm_vm_init() - Move the VM to initialized state. + * @vmid: VM identifier + * + * RM will allocate needed resources for the VM. After gh_rm_vm_init, gh_rm_get_hyp_resources() + * can be called to learn of the capabilities we can use with the new VM. + */ +int gh_rm_vm_init(struct gh_rm_rpc *rm, u16 vmid) +{ + return gh_rm_common_vmid_call(rm, GH_RM_RPC_VM_INIT, vmid); +} +EXPORT_SYMBOL_GPL(gh_rm_vm_init); + +/** + * gh_rm_get_hyp_resources() - Retrieve hypervisor resources (capabilities) associated with a VM + * @vmid: VMID of the other VM to get the resources of + * @resources: Set by gh_rm_get_hyp_resources and contains the returned hypervisor resources. + * + * Return: >=0 value indicates the number of gh_rm_hyp_resource entries filled into *resources + */ +ssize_t gh_rm_get_hyp_resources(struct gh_rm_rpc *rm, u16 vmid, + struct gh_rm_hyp_resource **resources) +{ + struct gh_vm_get_hyp_resources_resp *resp; + size_t resp_size; + int ret; + struct gh_vm_common_vmid_req req_payload = {0}; + + req_payload.vmid = vmid; + + ret = gh_rm_call(rm, GH_RM_RPC_VM_GET_HYP_RESOURCES, + &req_payload, sizeof(req_payload), + (void **)&resp, &resp_size); + if (ret) + return ret; + + if (resp_size < sizeof(*resp) || + (sizeof(*resp->entries) && (resp->n_entries > U32_MAX / sizeof(*resp->entries))) || + (resp_size != sizeof(*resp) + (resp->n_entries * sizeof(*resp->entries)))) { + ret = -EIO; + goto out; + } + + *resources = kmemdup(resp->entries, (resp->n_entries * sizeof(*resp->entries)), GFP_KERNEL); + ret = resp->n_entries; + +out: + kfree(resp); + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_get_hyp_resources); + +/** + * gh_rm_get_vmid() - Retrieve VMID of this virtual machine + * @vmid: Filled with the VMID of this VM + */ +int gh_rm_get_vmid(struct gh_rm_rpc *rm, u16 *vmid) +{ + void *resp; + size_t resp_size; + int ret; + int payload = 0; + + ret = gh_rm_call(rm, GH_RM_RPC_VM_GET_VMID, &payload, sizeof(payload), &resp, &resp_size); + if (ret) + return ret; + + if (resp_size != sizeof(*vmid)) + return -EIO; + *vmid = *(u16 *)resp; + kfree(resp); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_get_vmid); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Resource Manager RPC Driver"); diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index 01f0af888fad..11a5afa2c79c 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -34,4 +34,50 @@ struct platform_device; struct gh_rm_rpc *gh_rm_rpc_init(struct platform_device *pdev); void gh_rm_rpc_remove(struct gh_rm_rpc *rm); +/* Message IDs: VM Management */ +#define GH_RM_RPC_VM_ALLOC_VMID 0x56000001 +#define GH_RM_RPC_VM_DEALLOC_VMID 0x56000002 +#define GH_RM_RPC_VM_START 0x56000004 +#define GH_RM_RPC_VM_STOP 0x56000005 +#define GH_RM_RPC_VM_CONFIG_IMAGE 0x56000009 +#define GH_RM_RPC_VM_INIT 0x5600000B +#define GH_RM_RPC_VM_GET_HYP_RESOURCES 0x56000020 +#define GH_RM_RPC_VM_GET_VMID 0x56000024 +#define GH_RM_RPC_VM_SET_BOOT_CONTEXT 0x56000031 + +/* Call: CONSOLE_OPEN, CONSOLE_CLOSE, CONSOLE_FLUSH */ +struct gh_vm_common_vmid_req { + u16 vmid; + u16 reserved0; +} __packed; + +/* Call: VM_STOP */ +struct gh_vm_stop_req { + u16 vmid; + u8 flags; + u8 reserved; + u32 stop_reason; +} __packed; + +/* Call: VM_CONFIG_IMAGE */ +struct gh_vm_config_image_req { + u16 vmid; + u16 auth_mech; + u32 mem_handle; + u32 image_offset_low; + u32 image_offset_high; + u32 image_size_low; + u32 image_size_high; + u32 dtb_offset_low; + u32 dtb_offset_high; + u32 dtb_size_low; + u32 dtb_size_high; +} __packed; + +/* Call: GET_HYP_RESOURCES */ +struct gh_vm_get_hyp_resources_resp { + u32 n_entries; + struct gh_rm_hyp_resource entries[]; +} __packed; + #endif diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index b4f55c19954b..a0bd6f39c649 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -15,4 +15,62 @@ /* Gunyah recognizes VMID0 as an alias to the current VM's ID */ #define GH_VMID_SELF 0 +struct gh_rm_rpc; + +enum gh_rm_vm_status { + GH_RM_VM_STATUS_NO_STATE = 0, + GH_RM_VM_STATUS_INIT = 1, + GH_RM_VM_STATUS_READY = 2, + GH_RM_VM_STATUS_RUNNING = 3, + GH_RM_VM_STATUS_PAUSED = 4, + GH_RM_VM_STATUS_LOAD = 5, + GH_RM_VM_STATUS_AUTH = 6, + GH_RM_VM_STATUS_INIT_FAILED = 8, + GH_RM_VM_STATUS_EXITED = 9, + GH_RM_VM_STATUS_RESETTING = 10, + GH_RM_VM_STATUS_RESET = 11, +}; + +/* RPC Calls */ +int gh_rm_alloc_vmid(struct gh_rm_rpc *rm, u16 vmid); +int gh_rm_dealloc_vmid(struct gh_rm_rpc *rm, u16 vmid); +int gh_rm_vm_start(struct gh_rm_rpc *rm, u16 vmid); +int gh_rm_vm_stop(struct gh_rm_rpc *rm, u16 vmid); + +enum gh_rm_vm_auth_mechanism { + GH_RM_VM_AUTH_NONE = 0, + GH_RM_VM_AUTH_QCOM_PIL_ELF = 1, + GH_RM_VM_AUTH_QCOM_ANDROID_PVM = 2, +}; + +int gh_rm_vm_configure(struct gh_rm_rpc *rm, u16 vmid, enum gh_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size); +int gh_rm_vm_init(struct gh_rm_rpc *rm, u16 vmid); + +struct gh_rm_hyp_resource { + u8 type; + u8 reserved; + u16 partner_vmid; + u32 resource_handle; + u32 resource_label; + u32 cap_id_low; + u32 cap_id_high; + u32 virq_handle; + u32 virq; + u32 base_low; + u32 base_high; + u32 size_low; + u32 size_high; +} __packed; + +ssize_t gh_rm_get_hyp_resources(struct gh_rm_rpc *rm, u16 vmid, + struct gh_rm_hyp_resource **resources); +int gh_rm_get_vmid(struct gh_rm_rpc *rm, u16 *vmid); + +#define GH_RM_BOOT_CONTEXT_REG_SET_REGISTERS 0 +#define GH_RM_BOOT_CONTEXT_REG_SET_PC 1 +#define GH_RM_BOOT_CONTEXT_REG_SET_SP_ELx 2 +int gh_rm_set_boot_context(struct gh_rm_rpc *rm, u16 vmid, u8 reg_set, u8 reg_idx, u64 value); + #endif From patchwork Mon Nov 21 14:00:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29C6CC433FE for ; Mon, 21 Nov 2022 14:02:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231645AbiKUOCn (ORCPT ); Mon, 21 Nov 2022 09:02:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229969AbiKUOCC (ORCPT ); Mon, 21 Nov 2022 09:02:02 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB6D4FC6; Mon, 21 Nov 2022 06:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039320; x=1700575320; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=W/AbaL2gOtJsOtw6O608D+VMLPxbCH1NmWF8RQaiB/U=; b=Xz8QJ80nHS9ZC62tbrDvCWTdvghrsI6fYbpzqcIKl+vkh/VFju4TYVcN 00toZuG6ntljhafMLxQIr8y0CXIAdwU4q3q+JsK2USPG1SmdGvvMKga+X mV0qPeD5lwwosw2y2Hy8XTTG0qeKM5zCrejTHfQvjwhonHiR1o7cIm5qQ E=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 21 Nov 2022 06:02:00 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:02:00 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:01:58 -0800 From: Elliot Berman To: Bjorn Andersson , Jonathan Corbet , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 12/20] gunyah: vm_mgr: Introduce basic VM Manager Date: Mon, 21 Nov 2022 06:00:01 -0800 Message-ID: <20221121140009.2353512-13-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Gunyah VM manager is a kernel moduel which exposes an interface to Gunyah userspace to load, run, and interact with other Gunyah virtual machines. The interface is a character device at /dev/gunyah. Add a basic VM manager driver. Upcoming patches will add more ioctls into this driver. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- .../userspace-api/ioctl/ioctl-number.rst | 1 + drivers/virt/gunyah/Kconfig | 10 ++ drivers/virt/gunyah/Makefile | 3 + drivers/virt/gunyah/gunyah_rsc_mgr.c | 32 +++- drivers/virt/gunyah/vm_mgr.c | 155 ++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 31 ++++ include/uapi/linux/gunyah.h | 23 +++ 7 files changed, 249 insertions(+), 6 deletions(-) create mode 100644 drivers/virt/gunyah/vm_mgr.c create mode 100644 drivers/virt/gunyah/vm_mgr.h create mode 100644 include/uapi/linux/gunyah.h diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 5f81e2a24a5c..81456c34c85d 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -136,6 +136,7 @@ Code Seq# Include File Comments 'F' DD video/sstfb.h conflict! 'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! 'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict! +'G' 00-0f linux/gunyah.h conflict! 'H' 00-7F linux/hiddev.h conflict! 'H' 00-0F linux/hidraw.h conflict! 'H' 01 linux/mei.h conflict! diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 0bb497372d4e..5856b7c5157b 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -16,4 +16,14 @@ config GUNYAH_RESOURCE_MANAGER tristate depends on MAILBOX select GUNYAH_MESSAGE_QUEUES + +config GUNYAH_VM_MANAGER + tristate "Gunyah VM Manager" + depends on MAILBOX + select GUNYAH_RESOURCE_MANAGER + help + Gunyah VM manager is a kernel module which exposes an interface to + Gunyah userspace to load, run, and interact with other Gunyah + virtual machines. This module is required to launch other virtual + machines. endif diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b62ac4045621..ae628424719b 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,3 +1,6 @@ obj-$(CONFIG_GUNYAH) += gunyah.o obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER) += gunyah_rsc_mgr.o gunyah_rm_rpc.o + +gunyah_vm_mgr-y += vm_mgr.o +obj-$(CONFIG_GUNYAH_VM_MANAGER) += gunyah_vm_mgr.o diff --git a/drivers/virt/gunyah/gunyah_rsc_mgr.c b/drivers/virt/gunyah/gunyah_rsc_mgr.c index 2fae62bbfe49..47cf107ce894 100644 --- a/drivers/virt/gunyah/gunyah_rsc_mgr.c +++ b/drivers/virt/gunyah/gunyah_rsc_mgr.c @@ -7,25 +7,45 @@ #include #include "rsc_mgr.h" +#include "vm_mgr.h" + +struct gh_rsc_mgr { + struct gh_rm_rpc *rpc; + struct gh_vm_mgr *vm_mgr; +}; static int gh_rm_drv_probe(struct platform_device *pdev) { - struct gh_rm_rpc *rsc_mgr; + struct gh_rsc_mgr *rsc_mgr; - rsc_mgr = gh_rm_rpc_init(pdev); - if (IS_ERR(rsc_mgr)) - return PTR_ERR(rsc_mgr); + rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); + if (!rsc_mgr) + return -ENOMEM; platform_set_drvdata(pdev, rsc_mgr); + rsc_mgr->rpc = gh_rm_rpc_init(pdev); + if (IS_ERR(rsc_mgr->rpc)) + return PTR_ERR(rsc_mgr->rpc); + + rsc_mgr->vm_mgr = gh_vm_mgr_init(&pdev->dev, rsc_mgr->rpc); + if (IS_ERR(rsc_mgr->vm_mgr)) { + dev_warn(&pdev->dev, "Failed to initialize vm manager: %ld\n", + PTR_ERR(rsc_mgr->vm_mgr)); + rsc_mgr->vm_mgr = NULL; + } + return 0; } static int gh_rm_drv_remove(struct platform_device *pdev) { - struct gh_rm_rpc *rsc_mgr = platform_get_drvdata(pdev); + struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev); + + if (rsc_mgr->vm_mgr) + gh_vm_mgr_remove(rsc_mgr->vm_mgr); - gh_rm_rpc_remove(rsc_mgr); + gh_rm_rpc_remove(rsc_mgr->rpc); return 0; } diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c new file mode 100644 index 000000000000..deb8e0bf4a4d --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr.c @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_vm_mgr: " fmt + +#include +#include +#include +#include +#include + +#include + +#include "vm_mgr.h" + +static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_vm_mgr *vm_mgr) +{ + struct gunyah_vm *ghvm; + int ret; + + ret = gh_rm_alloc_vmid(vm_mgr->rm, 0); + if (ret < 0) + return ERR_PTR(ret); + + ghvm = kzalloc(sizeof(*ghvm), GFP_KERNEL); + if (!ghvm) + return ghvm; + + ghvm->vmid = ret; + + return ghvm; +} + +static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) +{ + long r; + + switch (cmd) { + default: + r = -ENOTTY; + break; + } + + return r; +} + +static int gh_vm_release(struct inode *inode, struct file *filp) +{ + struct gunyah_vm *ghvm = filp->private_data; + + kfree(ghvm); + return 0; +} + +static const struct file_operations gh_vm_fops = { + .unlocked_ioctl = gh_vm_ioctl, + .release = gh_vm_release, + .compat_ioctl = compat_ptr_ioctl, + .llseek = noop_llseek, +}; + +static long gh_dev_ioctl_create_vm(struct gh_vm_mgr *vm_mgr, unsigned long arg) +{ + struct gunyah_vm *ghvm; + struct file *file; + int fd, err; + + /* arg reserved for future use. */ + if (arg) + return -EINVAL; + + ghvm = gunyah_vm_alloc(vm_mgr); + if (IS_ERR_OR_NULL(ghvm)) + return PTR_ERR(ghvm) ? : -ENOMEM; + + fd = get_unused_fd_flags(O_CLOEXEC); + if (fd < 0) { + err = fd; + goto err_destroy_vm; + } + + file = anon_inode_getfile("gunyah-vm", &gh_vm_fops, ghvm, O_RDWR); + if (IS_ERR(file)) { + err = PTR_ERR(file); + goto err_put_fd; + } + + fd_install(fd, file); + + return fd; + +err_put_fd: + put_unused_fd(fd); +err_destroy_vm: + kfree(ghvm); + return err; +} + +static long gh_dev_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) +{ + struct miscdevice *miscdev = filp->private_data; + struct gh_vm_mgr *vm_mgr = container_of(miscdev, struct gh_vm_mgr, miscdev); + + switch (cmd) { + case GH_CREATE_VM: + return gh_dev_ioctl_create_vm(vm_mgr, arg); + default: + return -EINVAL; + } +} + +static const struct file_operations gh_dev_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = gh_dev_ioctl, + .compat_ioctl = compat_ptr_ioctl, + .llseek = noop_llseek, +}; + +struct gh_vm_mgr *gh_vm_mgr_init(struct device *dev, struct gh_rm_rpc *rm) +{ + struct gh_vm_mgr *vm_mgr; + int ret; + + vm_mgr = kzalloc(sizeof(*vm_mgr), GFP_KERNEL); + if (!vm_mgr) + return ERR_PTR(-ENOMEM); + + vm_mgr->dev = dev; + vm_mgr->rm = rm; + vm_mgr->miscdev.name = "gunyah"; + vm_mgr->miscdev.minor = MISC_DYNAMIC_MINOR; + vm_mgr->miscdev.fops = &gh_dev_fops; + + ret = misc_register(&vm_mgr->miscdev); + if (ret) { + kfree(vm_mgr); + return ERR_PTR(ret); + } + + return vm_mgr; +} +EXPORT_SYMBOL_GPL(gh_vm_mgr_init); + +void gh_vm_mgr_remove(struct gh_vm_mgr *vm_mgr) +{ + misc_deregister(&vm_mgr->miscdev); + kfree(vm_mgr); +} +EXPORT_SYMBOL_GPL(gh_vm_mgr_remove); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah VM Manager"); + diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h new file mode 100644 index 000000000000..9e9dfb1c0268 --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GH_PRIV_VM_MGR_H +#define _GH_PRIV_VM_MGR_H + +#include + +#include + +struct gh_vm_mgr { + struct device *dev; + struct gh_rm_rpc *rm; + struct miscdevice miscdev; +}; + +#if IS_ENABLED(CONFIG_GUNYAH_VM_MANAGER) +struct gh_vm_mgr *gh_vm_mgr_init(struct device *dev, struct gh_rm_rpc *rm); +void gh_vm_mgr_remove(struct gh_vm_mgr *vm_mgr); +#else +static inline struct gh_vm_mgr *gh_vm_mgr_init(struct gh_rm_rpc *rm) { return NULL; } +static inline void gh_vm_mgr_remove(struct gh_vm_mgr *vm_mgr) { } +#endif + +struct gunyah_vm { + u16 vmid; +}; + +#endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h new file mode 100644 index 000000000000..88a40d6e0b96 --- /dev/null +++ b/include/uapi/linux/gunyah.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _UAPI_LINUX_GUNYAH +#define _UAPI_LINUX_GUNYAH + +/* + * Userspace interface for /dev/gunyah - gunyah based virtual machine + */ + +#include +#include + +#define GH_IOCTL_TYPE 'G' + +/* + * ioctls for /dev/gunyah fds: + */ +#define GH_CREATE_VM _IO(GH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ + +#endif From patchwork Mon Nov 21 14:00:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D1CDC433FE for ; Mon, 21 Nov 2022 14:03:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230324AbiKUODD (ORCPT ); Mon, 21 Nov 2022 09:03:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229894AbiKUOCE (ORCPT ); Mon, 21 Nov 2022 09:02:04 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9EBD5FAE; Mon, 21 Nov 2022 06:02:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039322; x=1700575322; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=l7tmQcGy3NTootmU6QFfSxvXb9cGNIyVov6HfTyhpFQ=; b=fCkFghSvsaPGetDSprnFChe3b8knlvcmeUT1lpoajfw9SG4EMfOAKh61 YUi6OV70GmkSgNobDCy+XidmkXnZGJXIUUK+8V6zOREwV3sopHHM9Bz3P z5tPrloIrF+EUkhAf71CstI5VVG+8CwUcpH7zdFqmPZJ8AMMaiaEoj76E M=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 21 Nov 2022 06:02:02 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:02:02 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:01 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 13/20] gunyah: rsc_mgr: Add RPC for sharing memory Date: Mon, 21 Nov 2022 06:00:02 -0800 Message-ID: <20221121140009.2353512-14-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Gunyah resource manager provides API to manipulate stage 2 page tables. Manipulations are represented as a memory parcel. Memory parcels describe a list of memory regions (intermediate physical address and size), a list of new permissions for VMs, and the memory type (DDR or MMIO). Memory parcels are uniquely identified by a handle allocated by Gunyah. There are a few types of memory parcel sharing which Gunyah supports: - Sharing: the guest and host VM both have access - Lending: only the guest has access; host VM loses access - Donating: Permanently lent (not reclaimed even if guest shuts down) Memory parcels that have been shared or lent can be reclaimed by the host via an additional call. The reclaim operation restores the original access the host VM had to the memory parcel and removes the access to other VM. One point to note that memory parcels don't describe where in the guest VM the memory parcel should reside. The guest VM must accept the memory parcel either explicitly via a "gh_rm_mem_accept" call (not introduced here) or be configured to accept it automatically at boot. As the guest VM accepts the memory parcel, it also mentions the IPA it wants to place memory parcel. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/gunyah_rm_rpc.c | 199 ++++++++++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 44 ++++++ include/linux/gunyah_rsc_mgr.h | 47 +++++++ 3 files changed, 290 insertions(+) diff --git a/drivers/virt/gunyah/gunyah_rm_rpc.c b/drivers/virt/gunyah/gunyah_rm_rpc.c index fe6ba476c4dc..544ec8022104 100644 --- a/drivers/virt/gunyah/gunyah_rm_rpc.c +++ b/drivers/virt/gunyah/gunyah_rm_rpc.c @@ -49,6 +49,8 @@ struct gh_rm_rpc_reply_hdr { #define GH_RM_MAX_MSG_SIZE (GH_MSGQ_MAX_MSG_SIZE - sizeof(struct gh_rm_rpc_hdr)) +#define GH_RM_MAX_MEM_ENTRIES 512 + /** * struct gh_rm_connection - Represents a complete message from resource manager * @payload: Combined payload of all the fragments (msg headers stripped off). @@ -588,6 +590,203 @@ static int gh_rm_common_vmid_call(struct gh_rm_rpc *rm, u32 message_id, u16 vmid return ret; } +static int _gh_rm_mem_append(struct gh_rm_rpc *rm, u32 mem_handle, bool end_append, + struct gh_rm_mem_entry *mem_entries, size_t n_mem_entries) +{ + size_t msg_size = 0; + void *msg; + struct gh_mem_append_req_header *req_header; + struct gh_mem_share_req_mem_section *mem_section; + void *resp; + size_t resp_size; + int ret; + + msg_size += sizeof(struct gh_mem_append_req_header); + msg_size += offsetof(struct gh_mem_share_req_mem_section, entries[n_mem_entries]); + + msg = kzalloc(msg_size, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + req_header = msg; + mem_section = (void *)req_header + sizeof(struct gh_mem_append_req_header); + + req_header->mem_handle = mem_handle; + if (end_append) + req_header->flags |= GH_MEM_APPEND_REQ_FLAGS_END; + + mem_section->n_entries = n_mem_entries; + memcpy(mem_section->entries, mem_entries, sizeof(*mem_entries) * n_mem_entries); + + ret = gh_rm_call(rm, GH_RM_RPC_MEM_APPEND, msg, msg_size, &resp, &resp_size); + if (ret) + return ret; + kfree(resp); + + return ret; +} + +static int gh_rm_mem_append(struct gh_rm_rpc *rm, u32 mem_handle, bool allow_append, + struct gh_rm_mem_entry *mem_entries, size_t n_mem_entries) +{ + bool end_append; + size_t n; + int ret = 0; + + while (n_mem_entries) { + if (n_mem_entries > GH_RM_MAX_MEM_ENTRIES) { + end_append = false; + n = GH_RM_MAX_MEM_ENTRIES; + } else { + end_append = !allow_append; + n = n_mem_entries; + } + + ret = _gh_rm_mem_append(rm, mem_handle, end_append, mem_entries, n); + if (ret) + break; + + mem_entries += n; + n_mem_entries -= n; + } + + return ret; +} + +static int gh_rm_mem_lend_common(struct gh_rm_rpc *rm, u32 message_id, struct gh_rm_mem_parcel *p) +{ + size_t msg_size = 0, initial_n_mem_entries = p->n_mem_entries; + void *msg, *resp; + struct gh_mem_share_req_header *req_header; + struct gh_mem_share_req_acl_section *acl_section; + struct gh_mem_share_req_mem_section *mem_section; + u32 *mem_attr_section; + size_t resp_size; + int ret; + + if (!p->acl_entries || !p->n_acl_entries || !p->mem_entries || !p->n_mem_entries || + p->n_acl_entries > U8_MAX) + return -EINVAL; + + if (initial_n_mem_entries > GH_RM_MAX_MEM_ENTRIES) + initial_n_mem_entries = GH_RM_MAX_MEM_ENTRIES; + + /* The format of the message goes: + * request header + * ACL entries (which VMs get what kind of access to this memory parcel) + * Memory entries (list of memory regions to share) + * Memory attributes (currently unused, we'll hard-code the size to 0) + */ + msg_size += sizeof(struct gh_mem_share_req_header); + msg_size += offsetof(struct gh_mem_share_req_acl_section, entries[p->n_acl_entries]); + msg_size += offsetof(struct gh_mem_share_req_mem_section, entries[initial_n_mem_entries]); + msg_size += sizeof(u32); /* for memory attributes, currently unused */ + + msg = kzalloc(msg_size, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + req_header = msg; + acl_section = (void *)req_header + sizeof(*req_header); + mem_section = (void *)acl_section + offsetof(struct gh_mem_share_req_acl_section, + entries[p->n_acl_entries]); + mem_attr_section = (void *)mem_section + offsetof(struct gh_mem_share_req_mem_section, + entries[initial_n_mem_entries]); + + req_header->mem_type = p->mem_type; + if (initial_n_mem_entries != p->n_mem_entries) + req_header->flags |= GH_MEM_SHARE_REQ_FLAGS_APPEND; + req_header->label = p->label; + + acl_section->n_entries = p->n_acl_entries; + memcpy(acl_section->entries, p->acl_entries, sizeof(*(p->acl_entries)) * p->n_acl_entries); + + mem_section->n_entries = initial_n_mem_entries; + memcpy(mem_section->entries, p->mem_entries, + sizeof(*(p->mem_entries)) * initial_n_mem_entries); + + /* Set n_entries for memory attribute section to 0 */ + *mem_attr_section = 0; + + ret = gh_rm_call(rm, message_id, msg, msg_size, &resp, &resp_size); + if (ret) + return ret; + + if (resp_size != sizeof(u32)) { + ret = -EIO; + goto out; + } + + p->mem_handle = *(u32 *)resp; + + if (initial_n_mem_entries != p->n_mem_entries) { + ret = gh_rm_mem_append(rm, p->mem_handle, false, + &p->mem_entries[initial_n_mem_entries], + p->n_mem_entries - initial_n_mem_entries); + if (ret) + gh_rm_mem_reclaim(rm, p); + } + +out: + kfree(resp); + return ret; +} + +/** + * gh_rm_mem_lend() - Lend memory to other virtual machines. + * @parcel: Package the memory information of the memory to be lent. + * + * Returns - 0 on success; negative value on failure + * + * Lending removes Linux's access to the memory while the memory parcel is lent. + */ +int gh_rm_mem_lend(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *parcel) +{ + return gh_rm_mem_lend_common(rm, GH_RM_RPC_MEM_LEND, parcel); +} +EXPORT_SYMBOL_GPL(gh_rm_mem_lend); + + +/** + * gh_rm_mem_share() - Share memory with other virtual machines. + * @parcel: Package the memory information of the memory to be shared. + * + * Returns - 0 on success; negative value on failure + * + * Sharing keeps Linux's access to the memory while the memory parcel is shared. + */ +int gh_rm_mem_share(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *parcel) +{ + return gh_rm_mem_lend_common(rm, GH_RM_RPC_MEM_SHARE, parcel); +} +EXPORT_SYMBOL_GPL(gh_rm_mem_share); + +/** + * gh_rm_mem_reclaim() - Reclaim a memory parcel + * @parcel: Package the memory information of the memory to be reclaimed. + * + * Returns - 0 on success; negative value on failure + * + * RM maps the associated memory back into the stage-2 page tables of the owner VM. + */ +int gh_rm_mem_reclaim(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *parcel) +{ + struct gh_mem_release_req req = { + .mem_handle = parcel->mem_handle, + }; + size_t resp_size; + void *resp; + int ret; + + ret = gh_rm_call(rm, GH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), &resp, &resp_size); + if (ret) + return ret; + kfree(resp); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_mem_reclaim); + /** * gh_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM identifier. * @vmid: Use GH_VMID_INVAL to dynamically allocate a VM. A reserved VMID can also be requested diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index 11a5afa2c79c..4712405a3486 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -34,6 +34,12 @@ struct platform_device; struct gh_rm_rpc *gh_rm_rpc_init(struct platform_device *pdev); void gh_rm_rpc_remove(struct gh_rm_rpc *rm); +/* Message IDs: Memory Management */ +#define GH_RM_RPC_MEM_LEND 0x51000012 +#define GH_RM_RPC_MEM_SHARE 0x51000013 +#define GH_RM_RPC_MEM_RECLAIM 0x51000015 +#define GH_RM_RPC_MEM_APPEND 0x51000018 + /* Message IDs: VM Management */ #define GH_RM_RPC_VM_ALLOC_VMID 0x56000001 #define GH_RM_RPC_VM_DEALLOC_VMID 0x56000002 @@ -51,6 +57,44 @@ struct gh_vm_common_vmid_req { u16 reserved0; } __packed; +/* Call: MEM_LEND, MEM_SHARE */ +struct gh_mem_share_req_header { + u8 mem_type; + u8 reserved0; +#define GH_MEM_SHARE_REQ_FLAGS_APPEND BIT(1) + u8 flags; + u8 reserved1; + u32 label; +} __packed; + +struct gh_mem_share_req_acl_section { + u32 n_entries; + struct gh_rm_mem_acl_entry entries[]; +}; + +struct gh_mem_share_req_mem_section { + u16 n_entries; + u16 reserved0; + struct gh_rm_mem_entry entries[]; +}; + +/* Call: MEM_RELEASE */ +struct gh_mem_release_req { + u32 mem_handle; + u8 flags; + u16 reserved0; + u8 reserved1; +} __packed; + +/* Call: MEM_APPEND */ +struct gh_mem_append_req_header { + u32 mem_handle; +#define GH_MEM_APPEND_REQ_FLAGS_END BIT(0) + u8 flags; + u16 reserved0; + u8 reserved1; +} __packed; + /* Call: VM_STOP */ struct gh_vm_stop_req { u16 vmid; diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index a0bd6f39c649..c0f5c04c64dc 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -11,6 +11,7 @@ #include #define GH_VMID_INVAL U16_MAX +#define GH_MEM_HANDLE_INVAL U32_MAX /* Gunyah recognizes VMID0 as an alias to the current VM's ID */ #define GH_VMID_SELF 0 @@ -31,7 +32,53 @@ enum gh_rm_vm_status { GH_RM_VM_STATUS_RESET = 11, }; +struct gh_rm_mem_acl_entry { + u16 vmid; +#define GH_RM_ACL_X BIT(0) +#define GH_RM_ACL_W BIT(1) +#define GH_RM_ACL_R BIT(2) + u8 perms; + u8 reserved; +} __packed; + +struct gh_rm_mem_entry { + u64 ipa_base; + u64 size; +} __packed; + +enum gh_rm_mem_type { + GH_RM_MEM_TYPE_NORMAL = 0, + GH_RM_MEM_TYPE_IO = 1, +}; + +/* + * struct gh_rm_mem_parcel - Package info about memory to be lent/shared/donated/reclaimed + * @mem_type: The type of memory: normal (DDR) or IO + * @label: An client-specified identifier which can be used by the other VMs to identify the purpose + * of the memory parcel. + * @acl_entries: An array of access control entries. Each entry specifies a VM and what access + * is allowed for the memory parcel. + * @n_acl_entries: Count of the number of entries in the `acl_entries` array. + * @mem_entries: An list of regions to be associated with the memory parcel. Addresses should be + * (intermediate) physical addresses from Linux's perspective. + * @n_mem_entries: Count of the number of entries in the `mem_entries` array. + * @mem_handle: On success, filled with memory handle that RM allocates for this memory parcel + */ +struct gh_rm_mem_parcel { + enum gh_rm_mem_type mem_type; + u32 label; + size_t n_acl_entries; + struct gh_rm_mem_acl_entry *acl_entries; + size_t n_mem_entries; + struct gh_rm_mem_entry *mem_entries; + u32 mem_handle; +}; + /* RPC Calls */ +int gh_rm_mem_lend(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *parcel); +int gh_rm_mem_share(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *parcel); +int gh_rm_mem_reclaim(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *parcel); + int gh_rm_alloc_vmid(struct gh_rm_rpc *rm, u16 vmid); int gh_rm_dealloc_vmid(struct gh_rm_rpc *rm, u16 vmid); int gh_rm_vm_start(struct gh_rm_rpc *rm, u16 vmid); From patchwork Mon Nov 21 14:00:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEC48C4332F for ; Mon, 21 Nov 2022 14:04:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231605AbiKUOEb (ORCPT ); Mon, 21 Nov 2022 09:04:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231636AbiKUOCd (ORCPT ); Mon, 21 Nov 2022 09:02:33 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5875EAFE5F; Mon, 21 Nov 2022 06:02:24 -0800 (PST) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALCtBOs015905; Mon, 21 Nov 2022 14:02:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=eosVzzXka/PG9KmCHLIvQWIgURqMW0U5oCIVuUxg/00=; b=X9dp07YK/8jEzCTuGhRZAOyD7JDDo8xx77AgOtb68shGRJdzZX3Z+oForSnLjwKpUbuo eHm4+CF7R2DC8Tc8cOrGwVSTmPGmut/crOmat+61EABtxKUkk1QrZlUWATT8NHaK+YLL WRLtVt4MqSJD+kiwk61MGfpzZFfKsy0eOwRRNXbHSohBA6opeBKYrt2uXYN84YtpKkyg LvTYrqbEODjR83k36i4Yi/GfdLMytyWZY7DLgCSn/hCPbM4lGfKJRSJu60klaScSCo8P r4PSY2YVuOCzzCNoPF7I4WPUwoUhUgVoSKg2HIAS+9H0dOqfF1+m7R819vSWwWEtrtgL SA== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrf0vqh0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:06 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE25rf025027 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:05 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:03 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 14/20] gunyah: vm_mgr: Add/remove user memory regions Date: Mon, 21 Nov 2022 06:00:03 -0800 Message-ID: <20221121140009.2353512-15-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: zwN6eQZ1fQ1lF84GqboeN0QzdROiMau7 X-Proofpoint-ORIG-GUID: zwN6eQZ1fQ1lF84GqboeN0QzdROiMau7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 priorityscore=1501 spamscore=0 suspectscore=0 phishscore=0 clxscore=1015 impostorscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org When launching a virtual machine, Gunyah userspace allocates memory for the guest and informs Gunyah about these memory regions through SET_USER_MEMORY_REGION ioctl. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/vm_mgr.c | 46 +++++++ drivers/virt/gunyah/vm_mgr.h | 28 ++++ drivers/virt/gunyah/vm_mgr_mm.c | 222 ++++++++++++++++++++++++++++++++ include/uapi/linux/gunyah.h | 22 ++++ 5 files changed, 319 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/vm_mgr_mm.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index ae628424719b..82fd18d518d1 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -2,5 +2,5 @@ obj-$(CONFIG_GUNYAH) += gunyah.o obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER) += gunyah_rsc_mgr.o gunyah_rm_rpc.o -gunyah_vm_mgr-y += vm_mgr.o +gunyah_vm_mgr-y += vm_mgr.o vm_mgr_mm.o obj-$(CONFIG_GUNYAH_VM_MANAGER) += gunyah_vm_mgr.o diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index deb8e0bf4a4d..c2760e87f074 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -29,15 +29,57 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_vm_mgr *vm_mgr) return ghvm; ghvm->vmid = ret; + ghvm->vm_mgr = vm_mgr; + + mutex_init(&ghvm->mm_lock); + INIT_LIST_HEAD(&ghvm->memory_mappings); return ghvm; } static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { + struct gunyah_vm *ghvm = filp->private_data; + void __user *argp = (void __user *)arg; long r; switch (cmd) { + case GH_VM_SET_USER_MEM_REGION: { + struct gunyah_vm_memory_mapping *mapping; + struct gh_userspace_memory_region region; + + r = -EFAULT; + if (copy_from_user(®ion, argp, sizeof(region))) + break; + + r = -EINVAL; + /* All other flag bits are reserved for future use */ + if (region.flags & ~(GH_MEM_ALLOW_READ | GH_MEM_ALLOW_WRITE | GH_MEM_ALLOW_EXEC | + GH_MEM_LENT)) + break; + + + if (region.memory_size) { + r = 0; + mapping = gh_vm_mem_mapping_alloc(ghvm, ®ion); + if (IS_ERR(mapping)) { + r = PTR_ERR(mapping); + break; + } + } else { + mapping = gh_vm_mem_mapping_find(ghvm, region.label); + if (IS_ERR(mapping)) { + r = PTR_ERR(mapping); + break; + } + r = 0; + if (!mapping) + break; + gh_vm_mem_mapping_reclaim(ghvm, mapping); + kfree(mapping); + } + break; + } default: r = -ENOTTY; break; @@ -49,7 +91,11 @@ static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) static int gh_vm_release(struct inode *inode, struct file *filp) { struct gunyah_vm *ghvm = filp->private_data; + struct gunyah_vm_memory_mapping *mapping, *tmp; + list_for_each_entry_safe(mapping, tmp, &ghvm->memory_mappings, list) { + gh_vm_mem_mapping_reclaim(ghvm, mapping); + } kfree(ghvm); return 0; } diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 9e9dfb1c0268..f4a246b3403e 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -7,6 +7,9 @@ #define _GH_PRIV_VM_MGR_H #include +#include +#include +#include #include @@ -24,8 +27,33 @@ static inline struct gh_vm_mgr *gh_vm_mgr_init(struct gh_rm_rpc *rm) { return NU static inline void gh_vm_mgr_remove(struct gh_vm_mgr *vm_mgr) { } #endif +enum gunyah_vm_mem_share_type { + VM_MEM_SHARE, + VM_MEM_LEND, +}; + +struct gunyah_vm_memory_mapping { + struct list_head list; + enum gunyah_vm_mem_share_type share_type; + struct gh_rm_mem_parcel parcel; + + __u64 guest_phys_addr; + __u32 mem_size; + struct page **pages; + unsigned long npages; +}; + struct gunyah_vm { u16 vmid; + struct gh_vm_mgr *vm_mgr; + + struct mutex mm_lock; + struct list_head memory_mappings; }; +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_alloc(struct gunyah_vm *ghvm, + struct gh_userspace_memory_region *region); +void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_mapping *mapping); +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label); + #endif diff --git a/drivers/virt/gunyah/vm_mgr_mm.c b/drivers/virt/gunyah/vm_mgr_mm.c new file mode 100644 index 000000000000..104e5ba26ace --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr_mm.c @@ -0,0 +1,222 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_vm_mgr: " fmt + +#include +#include + +#include + +#include "vm_mgr.h" + +static inline bool page_contiguous(phys_addr_t p, phys_addr_t t) +{ + return t - p == PAGE_SIZE; +} + +static struct gunyah_vm_memory_mapping *__gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label) +{ + struct gunyah_vm_memory_mapping *mapping; + + + list_for_each_entry(mapping, &ghvm->memory_mappings, list) + if (mapping->parcel.label == label) + return mapping; + + return NULL; +} + +void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_mapping *mapping) +{ + int i, ret = 0; + + if (mapping->parcel.mem_handle != GH_MEM_HANDLE_INVAL) { + ret = gh_rm_mem_reclaim(ghvm->vm_mgr->rm, &mapping->parcel); + if (ret) + dev_warn(ghvm->vm_mgr->dev, + "Failed to reclaim memory parcel for label %d: %d\n", + mapping->parcel.label, ret); + } + + if (!ret) + for (i = 0; i < mapping->npages; i++) + unpin_user_page(mapping->pages[i]); + + kfree(mapping->pages); + kfree(mapping->parcel.acl_entries); + kfree(mapping->parcel.mem_entries); + + mutex_lock(&ghvm->mm_lock); + list_del(&mapping->list); + mutex_unlock(&ghvm->mm_lock); +} + +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label) +{ + struct gunyah_vm_memory_mapping *mapping; + int ret; + + ret = mutex_lock_interruptible(&ghvm->mm_lock); + if (ret) + return ERR_PTR(ret); + mapping = __gh_vm_mem_mapping_find(ghvm, label); + mutex_unlock(&ghvm->mm_lock); + return mapping; +} + +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_alloc(struct gunyah_vm *ghvm, + struct gh_userspace_memory_region *region) +{ + phys_addr_t curr_page, prev_page; + struct gunyah_vm_memory_mapping *mapping, *tmp_mapping; + struct gh_rm_mem_entry *mem_entries; + int i, j, pinned, ret = 0; + struct gh_rm_mem_parcel *parcel; + + if (!region->memory_size || !PAGE_ALIGNED(region->memory_size) || + !PAGE_ALIGNED(region->userspace_addr)) + return ERR_PTR(-EINVAL); + + ret = mutex_lock_interruptible(&ghvm->mm_lock); + if (ret) + return ERR_PTR(ret); + mapping = __gh_vm_mem_mapping_find(ghvm, region->label); + if (mapping) { + mapping = ERR_PTR(-EEXIST); + goto unlock; + } + + mapping = kzalloc(sizeof(*mapping), GFP_KERNEL); + if (!mapping) { + mapping = ERR_PTR(-ENOMEM); + goto unlock; + } + + mapping->parcel.label = region->label; + mapping->guest_phys_addr = region->guest_phys_addr; + mapping->npages = region->memory_size >> PAGE_SHIFT; + parcel = &mapping->parcel; + parcel->mem_handle = GH_MEM_HANDLE_INVAL; /* to be filled later by mem_share/mem_lend */ + parcel->mem_type = GH_RM_MEM_TYPE_NORMAL; + + /* Check for overlap */ + list_for_each_entry(tmp_mapping, &ghvm->memory_mappings, list) { + if (!((mapping->guest_phys_addr + (mapping->npages << PAGE_SHIFT) <= + tmp_mapping->guest_phys_addr) || + (mapping->guest_phys_addr >= + tmp_mapping->guest_phys_addr + (tmp_mapping->npages << PAGE_SHIFT)))) { + ret = -EEXIST; + goto unlock; + } + } + + list_add(&mapping->list, &ghvm->memory_mappings); +unlock: + mutex_unlock(&ghvm->mm_lock); + if (ret) + goto free_mapping; + + mapping->pages = kcalloc(mapping->npages, sizeof(*mapping->pages), GFP_KERNEL); + if (!mapping->pages) { + ret = -ENOMEM; + goto reclaim; + } + + pinned = pin_user_pages_fast(region->userspace_addr, mapping->npages, + FOLL_WRITE | FOLL_LONGTERM, mapping->pages); + if (pinned < 0) { + ret = pinned; + goto reclaim; + } else if (pinned != mapping->npages) { + ret = -EFAULT; + mapping->npages = pinned; /* update npages for reclaim */ + goto reclaim; + } + + if (region->flags & GH_MEM_LENT) { + parcel->n_acl_entries = 1; + mapping->share_type = VM_MEM_LEND; + } else { + parcel->n_acl_entries = 2; + mapping->share_type = VM_MEM_SHARE; + } + parcel->acl_entries = kcalloc(parcel->n_acl_entries, + sizeof(*parcel->acl_entries), + GFP_KERNEL); + if (!parcel->acl_entries) { + ret = -ENOMEM; + goto reclaim; + } + + parcel->acl_entries[0].vmid = ghvm->vmid; + if (region->flags & GH_MEM_ALLOW_READ) + parcel->acl_entries[0].perms |= GH_RM_ACL_R; + if (region->flags & GH_MEM_ALLOW_WRITE) + parcel->acl_entries[0].perms |= GH_RM_ACL_W; + if (region->flags & GH_MEM_ALLOW_EXEC) + parcel->acl_entries[0].perms |= GH_RM_ACL_X; + + if (mapping->share_type == VM_MEM_SHARE) { + ret = gh_rm_get_vmid(ghvm->vm_mgr->rm, &parcel->acl_entries[1].vmid); + if (ret) + goto reclaim; + /* Host assumed to have all these permissions. Gunyah will not + * grant new permissions if host actually had less than RWX + */ + parcel->acl_entries[1].perms |= GH_RM_ACL_R | GH_RM_ACL_W | GH_RM_ACL_X; + } + + mem_entries = kcalloc(mapping->npages, sizeof(*mem_entries), GFP_KERNEL); + if (!mem_entries) { + ret = -ENOMEM; + goto reclaim; + } + + // reduce number of entries by combining contiguous pages into single memory entry + prev_page = mem_entries[0].ipa_base = page_to_phys(mapping->pages[0]); + mem_entries[0].size = PAGE_SIZE; + for (i = 1, j = 0; i < mapping->npages; i++) { + curr_page = page_to_phys(mapping->pages[i]); + if (page_contiguous(prev_page, curr_page)) { + mem_entries[j].size += PAGE_SIZE; + } else { + j++; + mem_entries[j].ipa_base = curr_page; + mem_entries[j].size = PAGE_SIZE; + } + + prev_page = curr_page; + } + + parcel->n_mem_entries = j + 1; + parcel->mem_entries = kmemdup(mem_entries, sizeof(*mem_entries) * parcel->n_mem_entries, + GFP_KERNEL); + kfree(mem_entries); + if (!parcel->mem_entries) { + ret = -ENOMEM; + goto reclaim; + } + + switch (mapping->share_type) { + case VM_MEM_LEND: + ret = gh_rm_mem_lend(ghvm->vm_mgr->rm, parcel); + break; + case VM_MEM_SHARE: + ret = gh_rm_mem_share(ghvm->vm_mgr->rm, parcel); + break; + } + if (ret > 0) + ret = -EINVAL; + if (ret) + goto reclaim; + + return mapping; +reclaim: + gh_vm_mem_mapping_reclaim(ghvm, mapping); +free_mapping: + kfree(mapping); + return ERR_PTR(ret); +} diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 88a40d6e0b96..574f33b198d0 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -20,4 +20,26 @@ */ #define GH_CREATE_VM _IO(GH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ +/* + * ioctls for VM fds + */ +struct gh_userspace_memory_region { + __u32 label; +#define GH_MEM_ALLOW_READ (1UL << 0) +#define GH_MEM_ALLOW_WRITE (1UL << 1) +#define GH_MEM_ALLOW_EXEC (1UL << 2) +/* + * The guest will be lent the memory instead of shared. + * In other words, the guest has exclusive access to the memory region and the host loses access. + */ +#define GH_MEM_LENT (1UL << 3) + __u32 flags; + __u64 guest_phys_addr; + __u64 memory_size; + __u64 userspace_addr; +}; + +#define GH_VM_SET_USER_MEM_REGION _IOW(GH_IOCTL_TYPE, 0x1, \ + struct gh_userspace_memory_region) + #endif From patchwork Mon Nov 21 14:00:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B10BAC433FE for ; Mon, 21 Nov 2022 14:04:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230451AbiKUOEo (ORCPT ); Mon, 21 Nov 2022 09:04:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231649AbiKUOCn (ORCPT ); Mon, 21 Nov 2022 09:02:43 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 262E1113E; Mon, 21 Nov 2022 06:02:27 -0800 (PST) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALD3vTY011160; Mon, 21 Nov 2022 14:02:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=wJ/vyLrgdS7/uf1RzWm6ZDczkSB8KOThUUbpR86FCHw=; b=MjN8YDP8CmQBLoQEsT3cANDo0flH70TlA7D71XNBXIEB2Ki1WJOWWh6De8Ef4x2T5D78 NW6F0eIkQCIMJn7yScOqfCTexwO+RkiFFs0XNEFEjnBJbDYWZ7yx5P1/j+4u8p5X9B9q 5BOkgbzQjAUmVwXiS8m+jU6poMTXVCRTnuPQtPGEzzBDBOovHtbzfd9kUUZvQogqCqiH NwhXI3NEclZyD5o0uMHwwJEL/rGRj/lYR0r5Vkppl2EOWEcAq6iUiT/mpyP5m925oOmn /HWXKqZoxxxnn9wcedtPPV0dMzF6s7QpKfZrCY7/UFW1dg+ZHzDIdgs5MAU6yUUjmqDu Kw== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrf5mpuw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:09 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE28ZR015434 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:08 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:06 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 15/20] gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot Date: Mon, 21 Nov 2022 06:00:04 -0800 Message-ID: <20221121140009.2353512-16-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 8b6byQ41OznX4puruFFqErbSxskAEnNn X-Proofpoint-ORIG-GUID: 8b6byQ41OznX4puruFFqErbSxskAEnNn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Add remaining ioctls to support non-proxy VM boot: - Gunyah Resource Manager uses the VM's devicetree to configure the virtual machine. The location of the devicetree in the guest's virtual memory can be declared via the SET_DTB_CONFIG ioctl. - Trigger start of the virtual machine with VM_START ioctl. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 92 ++++++++++++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 9 ++++ drivers/virt/gunyah/vm_mgr_mm.c | 24 +++++++++ include/uapi/linux/gunyah.h | 8 +++ 4 files changed, 132 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index c2760e87f074..200adc024e67 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -8,7 +8,7 @@ #include #include #include -#include +#include #include #include @@ -33,10 +33,79 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_vm_mgr *vm_mgr) mutex_init(&ghvm->mm_lock); INIT_LIST_HEAD(&ghvm->memory_mappings); + init_rwsem(&ghvm->status_lock); return ghvm; } +static int gh_vm_start(struct gunyah_vm *ghvm) +{ + struct gunyah_vm_memory_mapping *mapping; + u64 dtb_offset; + u32 mem_handle; + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE) { + up_write(&ghvm->status_lock); + return 0; + } + + mapping = gh_vm_mem_mapping_find_mapping(ghvm, + ghvm->dtb_config.gpa, ghvm->dtb_config.size); + if (!mapping) { + dev_warn(ghvm->vm_mgr->dev, "Failed to find the memory_handle for DTB\n"); + ret = -EINVAL; + goto err; + } + + mem_handle = mapping->parcel.mem_handle; + dtb_offset = ghvm->dtb_config.gpa - mapping->guest_phys_addr; + + ret = gh_rm_vm_configure(ghvm->vm_mgr->rm, ghvm->vmid, ghvm->auth, mem_handle, + 0, 0, dtb_offset, ghvm->dtb_config.size); + if (ret) { + dev_warn(ghvm->vm_mgr->dev, "Failed to configure VM: %d\n", ret); + goto err; + } + + ret = gh_rm_vm_init(ghvm->vm_mgr->rm, ghvm->vmid); + if (ret) { + dev_warn(ghvm->vm_mgr->dev, "Failed to initialize VM: %d\n", ret); + goto err; + } + + ret = gh_rm_vm_start(ghvm->vm_mgr->rm, ghvm->vmid); + if (ret) { + dev_warn(ghvm->vm_mgr->dev, "Failed to start VM: %d\n", ret); + goto err; + } + + ghvm->vm_status = GH_RM_VM_STATUS_READY; + + up_write(&ghvm->status_lock); + return ret; +err: + ghvm->vm_status = GH_RM_VM_STATUS_INIT_FAILED; + up_write(&ghvm->status_lock); + return ret; +} + +static void gh_vm_stop(struct gunyah_vm *ghvm) +{ + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status == GH_RM_VM_STATUS_READY) { + ret = gh_rm_vm_stop(ghvm->vm_mgr->rm, ghvm->vmid); + if (ret) + dev_warn(ghvm->vm_mgr->dev, "Failed to stop VM: %d\n", ret); + } + + ghvm->vm_status = GH_RM_VM_STATUS_EXITED; + up_write(&ghvm->status_lock); +} + static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { struct gunyah_vm *ghvm = filp->private_data; @@ -80,6 +149,25 @@ static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) } break; } + case GH_VM_SET_DTB_CONFIG: { + struct gh_vm_dtb_config dtb_config; + + r = -EFAULT; + if (copy_from_user(&dtb_config, argp, sizeof(dtb_config))) + break; + + dtb_config.size = PAGE_ALIGN(dtb_config.size); + ghvm->dtb_config = dtb_config; + + r = 0; + break; + } + case GH_VM_START: { + r = gh_vm_start(ghvm); + if (r) + r = -EINVAL; + break; + } default: r = -ENOTTY; break; @@ -93,6 +181,8 @@ static int gh_vm_release(struct inode *inode, struct file *filp) struct gunyah_vm *ghvm = filp->private_data; struct gunyah_vm_memory_mapping *mapping, *tmp; + gh_vm_stop(ghvm); + list_for_each_entry_safe(mapping, tmp, &ghvm->memory_mappings, list) { gh_vm_mem_mapping_reclaim(ghvm, mapping); } diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index f4a246b3403e..d5ee446d4c04 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -10,6 +10,7 @@ #include #include #include +#include #include @@ -47,6 +48,12 @@ struct gunyah_vm { u16 vmid; struct gh_vm_mgr *vm_mgr; + enum gh_rm_vm_auth_mechanism auth; + struct gh_vm_dtb_config dtb_config; + + enum gh_rm_vm_status vm_status; + struct rw_semaphore status_lock; + struct mutex mm_lock; struct list_head memory_mappings; }; @@ -55,5 +62,7 @@ struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_alloc(struct gunyah_vm *ghvm, struct gh_userspace_memory_region *region); void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_mapping *mapping); struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label); +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find_mapping(struct gunyah_vm *ghvm, + u64 gpa, u32 size); #endif diff --git a/drivers/virt/gunyah/vm_mgr_mm.c b/drivers/virt/gunyah/vm_mgr_mm.c index 104e5ba26ace..451135521ee5 100644 --- a/drivers/virt/gunyah/vm_mgr_mm.c +++ b/drivers/virt/gunyah/vm_mgr_mm.c @@ -54,6 +54,30 @@ void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_m mutex_unlock(&ghvm->mm_lock); } +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find_mapping(struct gunyah_vm *ghvm, + u64 gpa, u32 size) +{ + struct gunyah_vm_memory_mapping *mapping = NULL; + int ret; + + ret = mutex_lock_interruptible(&ghvm->mm_lock); + if (ret) + return ERR_PTR(ret); + + list_for_each_entry(mapping, &ghvm->memory_mappings, list) { + if (gpa >= mapping->guest_phys_addr && + (gpa + size <= mapping->guest_phys_addr + + (mapping->npages << PAGE_SHIFT))) { + goto unlock; + } + } + + mapping = NULL; +unlock: + mutex_unlock(&ghvm->mm_lock); + return mapping; +} + struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label) { struct gunyah_vm_memory_mapping *mapping; diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 574f33b198d0..36359ad2175e 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -42,4 +42,12 @@ struct gh_userspace_memory_region { #define GH_VM_SET_USER_MEM_REGION _IOW(GH_IOCTL_TYPE, 0x1, \ struct gh_userspace_memory_region) +struct gh_vm_dtb_config { + __u64 gpa; + __u64 size; +}; +#define GH_VM_SET_DTB_CONFIG _IOW(GH_IOCTL_TYPE, 0x2, struct gh_vm_dtb_config) + +#define GH_VM_START _IO(GH_IOCTL_TYPE, 0x3) + #endif From patchwork Mon Nov 21 14:00:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F942C43217 for ; Mon, 21 Nov 2022 14:05:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229969AbiKUOFB (ORCPT ); Mon, 21 Nov 2022 09:05:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231668AbiKUODJ (ORCPT ); Mon, 21 Nov 2022 09:03:09 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BBEA38B6; Mon, 21 Nov 2022 06:02:30 -0800 (PST) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AL8OdTT000720; Mon, 21 Nov 2022 14:02:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=gzKrfexqZKPBpWA56YB1YQ8qQNrE07Z0wCn3d46rYAw=; b=nXm25ub/rJZKSmfVqfJFDQi3iyGINBCweSUqAeIMqTSMRsrJnWwv37AQxGqpPtIcuYwP ZAVL8Bkn6jCWbrbTt+kDmXKS/zAqe3C7/c+nJ/gf628mbB4Jxvofxfud2cpMiUCpug93 SOhxUf1ZtSMxd31NiQS1fMiDl7bpUdbgKDnQvVKOZN2JEB3DCG7A5z6j4JRqiEQHqJTO 0LniHgyKydwkJcSYy4m1gaj4QT82v+95ekdUy0VpmDZMlRsWcXrn0tOzxjjLk8eeHR8M H9dmlTwMMZy8PY4HduT+hVOLfQmUmate0S9Z2TOfsk7vPQx2fisWTajNXkCWs23vCjbY CQ== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrf5mpv1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:12 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE2Ba4016000 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:11 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:09 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 16/20] samples: Add sample userspace Gunyah VM Manager Date: Mon, 21 Nov 2022 06:00:05 -0800 Message-ID: <20221121140009.2353512-17-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 22Ls7JAY2zPRYVjs89E7eBrDPLopD2mn X-Proofpoint-ORIG-GUID: 22Ls7JAY2zPRYVjs89E7eBrDPLopD2mn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Add a sample Gunyah VMM capable of launching a non-proxy scheduled VM. Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + samples/Kconfig | 10 ++ samples/Makefile | 1 + samples/gunyah/.gitignore | 2 + samples/gunyah/Makefile | 6 + samples/gunyah/gunyah_vmm.c | 270 +++++++++++++++++++++++++++++++++++ samples/gunyah/sample_vm.dts | 69 +++++++++ 7 files changed, 359 insertions(+) create mode 100644 samples/gunyah/.gitignore create mode 100644 samples/gunyah/Makefile create mode 100644 samples/gunyah/gunyah_vmm.c create mode 100644 samples/gunyah/sample_vm.dts diff --git a/MAINTAINERS b/MAINTAINERS index b65f7ff444e5..e54c28859827 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8949,6 +8949,7 @@ F: arch/arm64/gunyah/ F: drivers/mailbox/gunyah-msgq.c F: drivers/virt/gunyah/ F: include/linux/gunyah*.h +F: samples/gunyah/ HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/samples/Kconfig b/samples/Kconfig index 0d81c00289ee..197c8a3610e7 100644 --- a/samples/Kconfig +++ b/samples/Kconfig @@ -263,6 +263,16 @@ config SAMPLE_CORESIGHT_SYSCFG This demonstrates how a user may create their own CoreSight configurations and easily load them into the system at runtime. +config SAMPLE_GUNYAH + bool "Build example Gunyah Virtual Machine Manager" + depends on CC_CAN_LINK && HEADERS_INSTALL + depends on GUNYAH_VM_MANAGER + help + Build an example Gunyah VMM userspace program capable of launching + a basic virtual machine under the Gunyah hypervisor. + This demonstrates how to create a virtual machine under the Gunyah + hypervisor. + source "samples/rust/Kconfig" endif # SAMPLES diff --git a/samples/Makefile b/samples/Makefile index 9832ef3f8fcb..2600bd4b82f8 100644 --- a/samples/Makefile +++ b/samples/Makefile @@ -36,3 +36,4 @@ obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak/ obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG) += coresight/ obj-$(CONFIG_SAMPLE_FPROBE) += fprobe/ obj-$(CONFIG_SAMPLES_RUST) += rust/ +obj-$(CONFIG_SAMPLE_GUNYAH) += gunyah/ diff --git a/samples/gunyah/.gitignore b/samples/gunyah/.gitignore new file mode 100644 index 000000000000..adc7d1589fde --- /dev/null +++ b/samples/gunyah/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +/gunyah_vmm diff --git a/samples/gunyah/Makefile b/samples/gunyah/Makefile new file mode 100644 index 000000000000..faf14f9bb337 --- /dev/null +++ b/samples/gunyah/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-only + +userprogs-always-y += gunyah_vmm +dtb-y += sample_vm.dtb + +userccflags += -I usr/include diff --git a/samples/gunyah/gunyah_vmm.c b/samples/gunyah/gunyah_vmm.c new file mode 100644 index 000000000000..ec74393c864e --- /dev/null +++ b/samples/gunyah/gunyah_vmm.c @@ -0,0 +1,270 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#define __USE_GNU +#include + +#include + +struct vm_config { + int image_fd; + int dtb_fd; + int ramdisk_fd; + + uint64_t guest_base; + uint64_t guest_size; + + uint64_t image_offset; + off_t image_size; + uint64_t dtb_offset; + off_t dtb_size; + uint64_t ramdisk_offset; + off_t ramdisk_size; +}; + +static struct option options[] = { + { "help", no_argument, NULL, 'h' }, + { "image", required_argument, NULL, 'i' }, + { "dtb", required_argument, NULL, 'd' }, + { "ramdisk", optional_argument, NULL, 'r' }, + { "base", optional_argument, NULL, 'B' }, + { "size", optional_argument, NULL, 'S' }, + { "image_offset", optional_argument, NULL, 'I' }, + { "dtb_offset", optional_argument, NULL, 'D' }, + { "ramdisk_offset", optional_argument, NULL, 'R' }, + { } +}; + +static void print_help(char *cmd) +{ + printf("gunyah_vmm, a sample tool to launch Gunyah VMs\n" + "Usage: %s \n" + " --help, -h this menu\n" + " --image, -i VM image file to load (e.g. a kernel Image) [Required]\n" + " --dtb, -d Devicetree to load [Required]\n" + " --ramdisk, -r Ramdisk to load\n" + " --base, -B
Set the base address of guest's memory [Default: 0x80000000]\n" + " --size, -S The number of bytes large to make the guest's memory [Default: 0x6400000 (100 MB)]\n" + " --image_offset, -I Offset into guest memory to load the VM image file [Default: 0x10000]\n" + " --dtb_offset, -D Offset into guest memory to load the DTB [Default: 0]\n" + " --ramdisk_offset, -R Offset into guest memory to load a ramdisk [Default: 0x4600000]\n" + , cmd); +} + +int main(int argc, char **argv) +{ + int gunyah_fd, vm_fd, guest_fd; + struct gh_userspace_memory_region guest_mem_desc = { 0 }; + struct gh_vm_dtb_config dtb_config = { 0 }; + char *guest_mem; + struct vm_config config = { + /* Defaults good enough to boot static kernel and a basic ramdisk */ + .ramdisk_fd = -1, + .guest_base = 0x80000000, + .guest_size = 104857600, /* 100 MB */ + .image_offset = 0, + .dtb_offset = 0x45f0000, + .ramdisk_offset = 0x4600000, /* put at +70MB (30MB for ramdisk) */ + }; + struct stat st; + int opt, optidx, ret = 0; + long int l; + + while ((opt = getopt_long(argc, argv, "hi:d:r:B:S:I:D:R:c:", options, &optidx)) != -1) { + switch (opt) { + case 'i': + config.image_fd = open(optarg, O_RDONLY | O_CLOEXEC); + if (config.image_fd < 0) { + perror("Failed to open image"); + return -1; + } + if (stat(optarg, &st) < 0) { + perror("Failed to stat image"); + return -1; + } + config.image_size = st.st_size; + break; + case 'd': + config.dtb_fd = open(optarg, O_RDONLY | O_CLOEXEC); + if (config.dtb_fd < 0) { + perror("Failed to open dtb"); + return -1; + } + if (stat(optarg, &st) < 0) { + perror("Failed to stat dtb"); + return -1; + } + config.dtb_size = st.st_size; + break; + case 'r': + config.ramdisk_fd = open(optarg, O_RDONLY | O_CLOEXEC); + if (config.ramdisk_fd < 0) { + perror("Failed to open ramdisk"); + return -1; + } + if (stat(optarg, &st) < 0) { + perror("Failed to stat ramdisk"); + return -1; + } + config.ramdisk_size = st.st_size; + break; + case 'B': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse base address"); + return -1; + } + config.guest_base = l; + break; + case 'S': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse memory size"); + return -1; + } + config.guest_size = l; + break; + case 'I': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse image offset"); + return -1; + } + config.image_offset = l; + break; + case 'D': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse dtb offset"); + return -1; + } + config.dtb_offset = l; + break; + case 'R': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse ramdisk offset"); + return -1; + } + config.ramdisk_offset = l; + break; + case 'h': + print_help(argv[0]); + return 0; + default: + print_help(argv[0]); + return -1; + } + } + + if (!config.image_fd || !config.dtb_fd) { + print_help(argv[0]); + return -1; + } + + if (config.image_offset + config.image_size > config.guest_size) { + fprintf(stderr, "Image offset and size puts it outside guest memory. Make image smaller or increase guest memory size.\n"); + return -1; + } + + if (config.dtb_offset + config.dtb_size > config.guest_size) { + fprintf(stderr, "DTB offset and size puts it outside guest memory. Make dtb smaller or increase guest memory size.\n"); + return -1; + } + + if (config.ramdisk_fd == -1 && + config.ramdisk_offset + config.ramdisk_size > config.guest_size) { + fprintf(stderr, "Ramdisk offset and size puts it outside guest memory. Make ramdisk smaller or increase guest memory size.\n"); + return -1; + } + + gunyah_fd = open("/dev/gunyah", O_RDWR | O_CLOEXEC); + if (gunyah_fd < 0) { + perror("Failed to open /dev/gunyah"); + return -1; + } + + vm_fd = ioctl(gunyah_fd, GH_CREATE_VM, 0); + if (vm_fd < 0) { + perror("Failed to create vm"); + return -1; + } + + guest_fd = memfd_create("guest_memory", MFD_CLOEXEC); + if (guest_fd < 0) { + perror("Failed to create guest memfd"); + return -1; + } + + if (ftruncate(guest_fd, config.guest_size) < 0) { + perror("Failed to grow guest memory"); + return -1; + } + + guest_mem = mmap(NULL, config.guest_size, PROT_READ | PROT_WRITE, MAP_SHARED, guest_fd, 0); + if (guest_mem == MAP_FAILED) { + perror("Not enough memory"); + return -1; + } + + if (read(config.image_fd, guest_mem + config.image_offset, config.image_size) < 0) { + perror("Failed to read image into guest memory"); + return -1; + } + + if (read(config.dtb_fd, guest_mem + config.dtb_offset, config.dtb_size) < 0) { + perror("Failed to read dtb into guest memory"); + return -1; + } + + if (config.ramdisk_fd > 0 && + read(config.ramdisk_fd, guest_mem + config.ramdisk_offset, + config.ramdisk_size) < 0) { + perror("Failed to read ramdisk into guest memory"); + return -1; + } + + guest_mem_desc.label = 0; + guest_mem_desc.flags = GH_MEM_ALLOW_READ | GH_MEM_ALLOW_WRITE | GH_MEM_ALLOW_EXEC; + guest_mem_desc.guest_phys_addr = config.guest_base; + guest_mem_desc.memory_size = config.guest_size; + guest_mem_desc.userspace_addr = (__u64)guest_mem; + + if (ioctl(vm_fd, GH_VM_SET_USER_MEM_REGION, &guest_mem_desc) < 0) { + perror("Failed to register guest memory with VM"); + return -1; + } + + dtb_config.gpa = config.guest_base + config.dtb_offset; + dtb_config.size = config.dtb_size; + if (ioctl(vm_fd, GH_VM_SET_DTB_CONFIG, &dtb_config) < 0) { + perror("Failed to set DTB configuration for VM"); + return -1; + } + + ret = ioctl(vm_fd, GH_VM_START); + if (ret) { + perror("GH_VM_START failed"); + return -1; + } + + while (1) + sleep(10); + + return 0; +} diff --git a/samples/gunyah/sample_vm.dts b/samples/gunyah/sample_vm.dts new file mode 100644 index 000000000000..1c410d58c298 --- /dev/null +++ b/samples/gunyah/sample_vm.dts @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: BSD-3-Clause +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +/dts-v1/; + +/ { + #address-cells = <2>; + #size-cells = <2>; + interrupt-parent = <&intc>; + + chosen { + bootargs = "nokaslr"; + }; + + cpus { + #address-cells = <0x2>; + #size-cells = <0>; + + cpu@0 { + device_type = "cpu"; + compatible = "arm,armv8"; + reg = <0 0>; + }; + }; + + intc: interrupt-controller@3FFF0000 { + compatible = "arm,gic-v3"; + #interrupt-cells = <3>; + #address-cells = <2>; + #size-cells = <2>; + interrupt-controller; + reg = <0 0x3FFF0000 0 0x10000>, + <0 0x3FFD0000 0 0x20000>; + }; + + timer { + compatible = "arm,armv8-timer"; + always-on; + interrupts = <1 13 0x108>, + <1 14 0x108>, + <1 11 0x108>, + <1 10 0x108>; + clock-frequency = <19200000>; + }; + + gunyah-vm-config { + image-name = "linux_vm_0"; + os-type = "linux"; + + memory { + #address-cells = <2>; + #size-cells = <2>; + + base-address = <0 0x80000000>; + }; + + interrupts { + config = <&intc>; + }; + + vcpus { + affinity-map = < 0 >; + sched-priority = < (-1) >; + sched-timeslice = < 2000 >; + }; + }; +}; From patchwork Mon Nov 21 14:00:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BE14C4332F for ; Mon, 21 Nov 2022 14:04:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231547AbiKUOEL (ORCPT ); Mon, 21 Nov 2022 09:04:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231578AbiKUOCR (ORCPT ); Mon, 21 Nov 2022 09:02:17 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 360242EF53; Mon, 21 Nov 2022 06:02:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039335; x=1700575335; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qy6zLlRt58JzjnAnjZx0E210Sky1kDaCzk1olERIjGA=; b=GtK3PqbbD3BmCUPb550lB/G6QPxm4Da7BWe0G6mllY3emKjUbyrMfD9t yx2jxRk89w0+ebdoFQG3dO7ZSQyFuo/XcbZzGOv3Inl35B7aZSDWpbJP4 l510ooPXiKVJGQ1ChGvIQ5m1s+33g0hx7Rv1y+WyMZlaE9ou2veFPS3SW M=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-01.qualcomm.com with ESMTP; 21 Nov 2022 06:02:14 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:02:14 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:12 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 17/20] gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim Date: Mon, 21 Nov 2022 06:00:06 -0800 Message-ID: <20221121140009.2353512-18-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org On Qualcomm platforms, there is a firmware entity which controls access to physical pages. In order to share memory with another VM, this entity needs to be informed that the guest VM should have access to the memory. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Kconfig | 4 ++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_platform_hooks.c | 63 +++++++++++++++++++++ drivers/virt/gunyah/gunyah_rm_rpc.c | 7 ++- drivers/virt/gunyah/rsc_mgr.h | 3 + include/linux/gunyah_rsc_mgr.h | 14 +++++ 6 files changed, 91 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/gunyah_platform_hooks.c diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 5856b7c5157b..41e619128b3d 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -11,11 +11,15 @@ config GUNYAH Say Y/M here to enable the drivers needed to interact in a Gunyah virtual environment. +config GUNYAH_PLATFORM_HOOKS + tristate + if GUNYAH config GUNYAH_RESOURCE_MANAGER tristate depends on MAILBOX select GUNYAH_MESSAGE_QUEUES + select GUNYAH_PLATFORM_HOOKS config GUNYAH_VM_MANAGER tristate "Gunyah VM Manager" diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 82fd18d518d1..90d589ce1f44 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,5 @@ obj-$(CONFIG_GUNYAH) += gunyah.o +obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER) += gunyah_rsc_mgr.o gunyah_rm_rpc.o diff --git a/drivers/virt/gunyah/gunyah_platform_hooks.c b/drivers/virt/gunyah/gunyah_platform_hooks.c new file mode 100644 index 000000000000..36cc02c75963 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_platform_hooks.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include + +#include "rsc_mgr.h" + +static struct gunyah_rm_platform_ops *rm_platform_ops; +static DECLARE_RWSEM(rm_platform_ops_lock); + +int gh_rm_platform_pre_mem_share(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_mem_share) + ret = rm_platform_ops->pre_mem_share(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_platform_pre_mem_share); + +int gh_rm_platform_post_mem_reclaim(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->post_mem_reclaim) + ret = rm_platform_ops->post_mem_reclaim(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_platform_post_mem_reclaim); + +int gh_rm_register_platform_ops(struct gunyah_rm_platform_ops *platform_ops) +{ + int ret = 0; + + down_write(&rm_platform_ops_lock); + if (!rm_platform_ops) + rm_platform_ops = platform_ops; + else + ret = -EEXIST; + up_write(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_register_platform_ops); + +void gh_rm_unregister_platform_ops(struct gunyah_rm_platform_ops *platform_ops) +{ + down_write(&rm_platform_ops_lock); + if (rm_platform_ops == platform_ops) + rm_platform_ops = NULL; + up_write(&rm_platform_ops_lock); +} +EXPORT_SYMBOL_GPL(gh_rm_unregister_platform_ops); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Platform Hooks"); diff --git a/drivers/virt/gunyah/gunyah_rm_rpc.c b/drivers/virt/gunyah/gunyah_rm_rpc.c index 544ec8022104..f81c7b55e7d7 100644 --- a/drivers/virt/gunyah/gunyah_rm_rpc.c +++ b/drivers/virt/gunyah/gunyah_rm_rpc.c @@ -671,6 +671,10 @@ static int gh_rm_mem_lend_common(struct gh_rm_rpc *rm, u32 message_id, struct gh if (initial_n_mem_entries > GH_RM_MAX_MEM_ENTRIES) initial_n_mem_entries = GH_RM_MAX_MEM_ENTRIES; + ret = gh_rm_platform_pre_mem_share(rm, p); + if (ret) + return ret; + /* The format of the message goes: * request header * ACL entries (which VMs get what kind of access to this memory parcel) @@ -714,6 +718,7 @@ static int gh_rm_mem_lend_common(struct gh_rm_rpc *rm, u32 message_id, struct gh if (resp_size != sizeof(u32)) { ret = -EIO; + gh_rm_platform_post_mem_reclaim(rm, p); goto out; } @@ -783,7 +788,7 @@ int gh_rm_mem_reclaim(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *parcel) return ret; kfree(resp); - return ret; + return gh_rm_platform_post_mem_reclaim(rm, parcel); } EXPORT_SYMBOL_GPL(gh_rm_mem_reclaim); diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index 4712405a3486..3a7f171a7416 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -34,6 +34,9 @@ struct platform_device; struct gh_rm_rpc *gh_rm_rpc_init(struct platform_device *pdev); void gh_rm_rpc_remove(struct gh_rm_rpc *rm); +int gh_rm_platform_pre_mem_share(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel); +int gh_rm_platform_post_mem_reclaim(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel); + /* Message IDs: Memory Management */ #define GH_RM_RPC_MEM_LEND 0x51000012 #define GH_RM_RPC_MEM_SHARE 0x51000013 diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index c0f5c04c64dc..7cdb736f7054 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -120,4 +120,18 @@ int gh_rm_get_vmid(struct gh_rm_rpc *rm, u16 *vmid); #define GH_RM_BOOT_CONTEXT_REG_SET_SP_ELx 2 int gh_rm_set_boot_context(struct gh_rm_rpc *rm, u16 vmid, u8 reg_set, u8 reg_idx, u64 value); +struct gunyah_rm_platform_ops { + int (*pre_mem_share)(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel); + int (*post_mem_reclaim)(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel); +}; + +#if IS_ENABLED(CONFIG_GUNYAH) +int gh_rm_register_platform_ops(struct gunyah_rm_platform_ops *platform_ops); +void gh_rm_unregister_platform_ops(struct gunyah_rm_platform_ops *platform_ops); +#else +static inline int gh_rm_register_platform_ops(struct gunyah_rm_platform_ops *platform_ops) + { return 0; } +static inline void gh_rm_unregister_platform_ops(struct gunyah_rm_platform_ops *platform_ops) { } +#endif + #endif From patchwork Mon Nov 21 14:00:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBB99C4167D for ; Mon, 21 Nov 2022 14:04:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229749AbiKUOEW (ORCPT ); Mon, 21 Nov 2022 09:04:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231605AbiKUOCU (ORCPT ); Mon, 21 Nov 2022 09:02:20 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F33B686A63; Mon, 21 Nov 2022 06:02:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1669039340; x=1700575340; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7FbkFUUwW8f1lkbeYVGTNASH9BSWg8uSE/T9xxdn1m8=; b=po4MhVFNoVYDDKELfqhwa63yF6eYA8n1IT6Zjo8kd133Tg3RsidnC7PX 6npulL0M7PFMyyFqRSMULEjXOZ3qvrseDjTotQ4oaDrqBuFK0TTaj3cgL v5N6ABEJTQU+cr0h1A+2jNHg6QstCXkczf7UAmboZqp4g0UfXz5Q5789+ s=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 21 Nov 2022 06:02:19 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Nov 2022 06:02:18 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:15 -0800 From: Elliot Berman To: Bjorn Andersson , Andy Gross , Bjorn Andersson , Konrad Dybcio , Srinivas Kandagatla , Amol Maheshwari , Arnd Bergmann , Greg Kroah-Hartman , Kalle Valo , "David S. Miller" , Eric Dumazet , "Jakub Kicinski" , Paolo Abeni , Mathieu Poirier CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Prakruthi Deepak Heragu , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , "Lorenzo Pieralisi" , , , , , , , Neil Armstrong , , , , Subject: [PATCH v7 18/20] firmware: qcom_scm: Use fixed width src vm bitmap Date: Mon, 21 Nov 2022 06:00:07 -0800 Message-ID: <20221121140009.2353512-19-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org The maximum VMID for assign_mem is 63. Use a u64 to represent this bitmap instead of architecture-dependent "unsigned int" which varies in size on 32-bit and 64-bit platforms. Signed-off-by: Elliot Berman --- Note this will have build conflicts with https://lore.kernel.org/all/20221114-narmstrong-sm8550-upstream-mpss_dsm-v2-2-f7c65d6f0e55@linaro.org/ which would also need an "unsigned int" -> "u64" in struct qcom_mpss_dsm_mem:perms. drivers/firmware/qcom_scm.c | 12 +++++++----- drivers/misc/fastrpc.c | 6 ++++-- drivers/net/wireless/ath/ath10k/qmi.c | 4 ++-- drivers/remoteproc/qcom_q6v5_mss.c | 8 ++++---- drivers/soc/qcom/rmtfs_mem.c | 2 +- include/linux/qcom_scm.h | 2 +- 6 files changed, 19 insertions(+), 15 deletions(-) diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c index cdbfe54c8146..92763dce6477 100644 --- a/drivers/firmware/qcom_scm.c +++ b/drivers/firmware/qcom_scm.c @@ -898,7 +898,7 @@ static int __qcom_scm_assign_mem(struct device *dev, phys_addr_t mem_region, * Return negative errno on failure or 0 on success with @srcvm updated. */ int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, - unsigned int *srcvm, + u64 *srcvm, const struct qcom_scm_vmperm *newvm, unsigned int dest_cnt) { @@ -915,9 +915,9 @@ int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, __le32 *src; void *ptr; int ret, i, b; - unsigned long srcvm_bits = *srcvm; + u64 srcvm_bits = *srcvm; - src_sz = hweight_long(srcvm_bits) * sizeof(*src); + src_sz = hweight64(srcvm_bits) * sizeof(*src); mem_to_map_sz = sizeof(*mem_to_map); dest_sz = dest_cnt * sizeof(*destvm); ptr_sz = ALIGN(src_sz, SZ_64) + ALIGN(mem_to_map_sz, SZ_64) + @@ -930,8 +930,10 @@ int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, /* Fill source vmid detail */ src = ptr; i = 0; - for_each_set_bit(b, &srcvm_bits, BITS_PER_LONG) - src[i++] = cpu_to_le32(b); + for (b = 0; b < BITS_PER_TYPE(u64); b++) { + if (srcvm_bits & BIT(b)) + src[i++] = cpu_to_le32(b); + } /* Fill details of mem buff to map */ mem_to_map = ptr + ALIGN(src_sz, SZ_64); diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 7ff0b63c25e3..2ad388f99fe1 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -299,11 +299,13 @@ static void fastrpc_free_map(struct kref *ref) if (map->attr & FASTRPC_ATTR_SECUREMAP) { struct qcom_scm_vmperm perm; int err = 0; + u64 src; + src = BIT(map->fl->cctx->vmperms[0].vmid); perm.vmid = QCOM_SCM_VMID_HLOS; perm.perm = QCOM_SCM_PERM_RWX; err = qcom_scm_assign_mem(map->phys, map->size, - &(map->fl->cctx->vmperms[0].vmid), &perm, 1); + &src, &perm, 1); if (err) { dev_err(map->fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d", map->phys, map->size, err); @@ -744,7 +746,7 @@ static int fastrpc_map_create(struct fastrpc_user *fl, int fd, * If subsystem VMIDs are defined in DTSI, then do * hyp_assign from HLOS to those VM(s) */ - unsigned int perms = BIT(QCOM_SCM_VMID_HLOS); + u64 perms = BIT(QCOM_SCM_VMID_HLOS); map->attr = attr; err = qcom_scm_assign_mem(map->phys, (u64)map->size, &perms, diff --git a/drivers/net/wireless/ath/ath10k/qmi.c b/drivers/net/wireless/ath/ath10k/qmi.c index 66cb7a1e628a..6d1d87e1cdde 100644 --- a/drivers/net/wireless/ath/ath10k/qmi.c +++ b/drivers/net/wireless/ath/ath10k/qmi.c @@ -28,7 +28,7 @@ static int ath10k_qmi_map_msa_permission(struct ath10k_qmi *qmi, { struct qcom_scm_vmperm dst_perms[3]; struct ath10k *ar = qmi->ar; - unsigned int src_perms; + u64 src_perms; u32 perm_count; int ret; @@ -60,7 +60,7 @@ static int ath10k_qmi_unmap_msa_permission(struct ath10k_qmi *qmi, { struct qcom_scm_vmperm dst_perms; struct ath10k *ar = qmi->ar; - unsigned int src_perms; + u64 src_perms; int ret; src_perms = BIT(QCOM_SCM_VMID_MSS_MSA) | BIT(QCOM_SCM_VMID_WLAN); diff --git a/drivers/remoteproc/qcom_q6v5_mss.c b/drivers/remoteproc/qcom_q6v5_mss.c index fddb63cffee0..9e8bde7a7ec4 100644 --- a/drivers/remoteproc/qcom_q6v5_mss.c +++ b/drivers/remoteproc/qcom_q6v5_mss.c @@ -227,8 +227,8 @@ struct q6v5 { bool has_qaccept_regs; bool has_ext_cntl_regs; bool has_vq6; - int mpss_perm; - int mba_perm; + u64 mpss_perm; + u64 mba_perm; const char *hexagon_mdt_image; int version; }; @@ -404,7 +404,7 @@ static void q6v5_pds_disable(struct q6v5 *qproc, struct device **pds, } } -static int q6v5_xfer_mem_ownership(struct q6v5 *qproc, int *current_perm, +static int q6v5_xfer_mem_ownership(struct q6v5 *qproc, u64 *current_perm, bool local, bool remote, phys_addr_t addr, size_t size) { @@ -939,7 +939,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw, struct page *page; dma_addr_t phys; void *metadata; - int mdata_perm; + u64 mdata_perm; int xferop_ret; size_t size; void *vaddr; diff --git a/drivers/soc/qcom/rmtfs_mem.c b/drivers/soc/qcom/rmtfs_mem.c index 0feaae357821..69991e47aa23 100644 --- a/drivers/soc/qcom/rmtfs_mem.c +++ b/drivers/soc/qcom/rmtfs_mem.c @@ -30,7 +30,7 @@ struct qcom_rmtfs_mem { unsigned int client_id; - unsigned int perms; + u64 perms; }; static ssize_t qcom_rmtfs_mem_show(struct device *dev, diff --git a/include/linux/qcom_scm.h b/include/linux/qcom_scm.h index f8335644a01a..77f7b5837216 100644 --- a/include/linux/qcom_scm.h +++ b/include/linux/qcom_scm.h @@ -96,7 +96,7 @@ extern int qcom_scm_mem_protect_video_var(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start, u32 cp_nonpixel_size); extern int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, - unsigned int *src, + u64 *src, const struct qcom_scm_vmperm *newvm, unsigned int dest_cnt); From patchwork Mon Nov 21 14:00:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30BF2C43219 for ; Mon, 21 Nov 2022 14:06:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230207AbiKUOGN (ORCPT ); Mon, 21 Nov 2022 09:06:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40584 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230194AbiKUOFr (ORCPT ); Mon, 21 Nov 2022 09:05:47 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF05915A28; Mon, 21 Nov 2022 06:02:52 -0800 (PST) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALDpuuw002757; Mon, 21 Nov 2022 14:02:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=mXb2fL6LiNsISTP1XVQxJOuaLu5jYesWFp3pLxmAg70=; b=PaYdMu9/9MHoniJlEBDclScdHeHtgg4k1Q28H3iAa4ibHCPu4SuNbOvjHtdpL5UXWugF 7HdfOk18EFV7sJsY7e5a6v7sgfHMjS2YtxRTScCongvaBG1Xu4Fkbww8JtCm6piv7nFI 0DTWYqguGCBOB7q6NbGwi1LR6CttGmujrETdaP/k1zsT3F3bzDSS5hkFJVXkfpiiZoUI O6jYQnBeKhgWucqTUy0oBpRUTCc5D+BpDSBu+G66+0lJkCx+MRbsxi3FKW0M40T7l6wQ LQ1dnQGr1Ka9YNV4K1SJ4wXPlMiP7d7rhYlAtho+Dn9mGQyj7cgLBdpDScv07aYKiLv0 vw== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrkv4j7u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:22 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE2LHm031134 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:21 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:19 -0800 From: Elliot Berman To: Bjorn Andersson , Andy Gross , Bjorn Andersson , Konrad Dybcio CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , "Lorenzo Pieralisi" , , , , , , Subject: [PATCH v7 19/20] firmware: qcom_scm: Register Gunyah platform ops Date: Mon, 21 Nov 2022 06:00:08 -0800 Message-ID: <20221121140009.2353512-20-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: F-OM38BBb85ligj1Izx4ohLwH3UxOV7Q X-Proofpoint-ORIG-GUID: F-OM38BBb85ligj1Izx4ohLwH3UxOV7Q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 spamscore=0 mlxlogscore=999 malwarescore=0 impostorscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 adultscore=0 suspectscore=0 bulkscore=0 clxscore=1011 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Qualcomm platforms have a firmware entity which performs access control to physical pages. Dynamically started Gunyah virtual machines use the QCOM_SCM_RM_MANAGED_VMID for access. Linux thus needs to assign access to the memory used by guest VMs. Gunyah doesn't do this operation for us since it is the current VM (typically VMID_HLOS) delegating the access and not Gunyah itself. Use the Gunyah platform ops to achieve this so that only Qualcomm platforms attempt to make the needed SCM calls. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/firmware/Kconfig | 2 + drivers/firmware/qcom_scm.c | 95 +++++++++++++++++++++++++++++++++++++ 2 files changed, 97 insertions(+) diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index b59e3041fd62..b888068ff6f2 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -214,6 +214,8 @@ config MTK_ADSP_IPC config QCOM_SCM tristate + select VIRT_DRIVERS + select GUNYAH_PLATFORM_HOOKS config QCOM_SCM_DOWNLOAD_MODE_DEFAULT bool "Qualcomm download mode enabled by default" diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c index 92763dce6477..2dde5cd9701b 100644 --- a/drivers/firmware/qcom_scm.c +++ b/drivers/firmware/qcom_scm.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "qcom_scm.h" @@ -27,6 +28,9 @@ module_param(download_mode, bool, 0); #define SCM_HAS_IFACE_CLK BIT(1) #define SCM_HAS_BUS_CLK BIT(2) +#define QCOM_SCM_RM_MANAGED_VMID 0x3A +#define QCOM_SCM_MAX_MANAGED_VMID 0x3F + struct qcom_scm { struct device *dev; struct clk *core_clk; @@ -1292,6 +1296,94 @@ int qcom_scm_lmh_dcvsh(u32 payload_fn, u32 payload_reg, u32 payload_val, } EXPORT_SYMBOL(qcom_scm_lmh_dcvsh); +static int qcom_scm_gh_rm_pre_mem_share(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm *new_perms; + u64 src, src_cpy; + int ret, i, n; + + new_perms = kcalloc(mem_parcel->n_acl_entries, sizeof(*new_perms), GFP_KERNEL); + if (!new_perms) + return -ENOMEM; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + if (mem_parcel->acl_entries[n].vmid <= QCOM_SCM_MAX_MANAGED_VMID) + new_perms[n].vmid = mem_parcel->acl_entries[n].vmid; + else + new_perms[n].vmid = QCOM_SCM_RM_MANAGED_VMID; + if (mem_parcel->acl_entries[n].perms & GH_RM_ACL_X) + new_perms[n].perm |= QCOM_SCM_PERM_EXEC; + if (mem_parcel->acl_entries[n].perms & GH_RM_ACL_W) + new_perms[n].perm |= QCOM_SCM_PERM_WRITE; + if (mem_parcel->acl_entries[n].perms & GH_RM_ACL_R) + new_perms[n].perm |= QCOM_SCM_PERM_READ; + } + + src = (1ull << QCOM_SCM_VMID_HLOS); + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy = src; + ret = qcom_scm_assign_mem(mem_parcel->mem_entries[i].ipa_base, + mem_parcel->mem_entries[i].size, + &src_cpy, new_perms, mem_parcel->n_acl_entries); + if (ret) { + src = 0; + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + if (mem_parcel->acl_entries[n].vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= (1ull << mem_parcel->acl_entries[n].vmid); + else + src |= (1ull << QCOM_SCM_RM_MANAGED_VMID); + } + + new_perms[0].vmid = QCOM_SCM_VMID_HLOS; + + for (i--; i >= 0; i--) { + src_cpy = src; + ret = qcom_scm_assign_mem(mem_parcel->mem_entries[i].ipa_base, + mem_parcel->mem_entries[i].size, + &src_cpy, new_perms, 1); + WARN_ON_ONCE(ret); + } + break; + } + } + + kfree(new_perms); + return ret; +} + +static int qcom_scm_gh_rm_post_mem_reclaim(struct gh_rm_rpc *rm, + struct gh_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm new_perms; + u64 src = 0; + int ret, i, n; + + new_perms.vmid = QCOM_SCM_VMID_HLOS; + new_perms.perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | QCOM_SCM_PERM_READ; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + if (mem_parcel->acl_entries[n].vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= (1ull << mem_parcel->acl_entries[n].vmid); + else + src |= (1ull << QCOM_SCM_RM_MANAGED_VMID); + } + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + ret = qcom_scm_assign_mem(mem_parcel->mem_entries[i].ipa_base, + mem_parcel->mem_entries[i].size, + &src, &new_perms, 1); + WARN_ON_ONCE(ret); + } + + return ret; +} + +static struct gunyah_rm_platform_ops qcom_scm_gh_rm_platform_ops = { + .pre_mem_share = qcom_scm_gh_rm_pre_mem_share, + .post_mem_reclaim = qcom_scm_gh_rm_post_mem_reclaim, +}; + static int qcom_scm_find_dload_address(struct device *dev, u64 *addr) { struct device_node *tcsr; @@ -1414,6 +1506,9 @@ static int qcom_scm_probe(struct platform_device *pdev) if (download_mode) qcom_scm_set_download_mode(true); + if (gh_rm_register_platform_ops(&qcom_scm_gh_rm_platform_ops)) + dev_warn(__scm->dev, "Gunyah RM platform ops were already registered\n"); + return 0; } From patchwork Mon Nov 21 14:00:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 627279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95F84C4332F for ; Mon, 21 Nov 2022 14:06:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230214AbiKUOGG (ORCPT ); Mon, 21 Nov 2022 09:06:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229879AbiKUOFp (ORCPT ); Mon, 21 Nov 2022 09:05:45 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F6A81570A; Mon, 21 Nov 2022 06:02:51 -0800 (PST) Received: from pps.filterd (m0279868.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ALDS0EE002790; Mon, 21 Nov 2022 14:02:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=Iu8gzi/8SDZ9jKDVjZrgx67m0CCgK3Dg5o8FVtuo6Vg=; b=MWF45FLXCl0RbX8iCOLvmV2AusDeY0xzn/cmp1EcD38gKsE4mLshQtjMAwcZwh23naHI JOgue/SYS8mUhnvcl0zVs5OrhBXruSRKhANqfvCcdj1qHYtcQ7A/PrxMPjAKUuHwLl86 /FsyIqcIV+6T+VwCDo2MmHz5J5+s83cjpHTojrDM5X8gReLQ2kcuEVpO8qfa5Ri1GxhR 3C6XGirCLXftdni5/SVvaS8wuH2rMRUeD6vvTs2gRSJegM0uDvb5V2RZQnQPmHZ5kV0A QXfksUOlgjxmR5hTkIkMY//y+Jy4VJmxs6iNyyWJ6aZmjCVe6jfREyZ4JbiZy67/NUV0 Dg== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3kxrtqcfk0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:24 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2ALE2NCF025232 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 21 Nov 2022 14:02:23 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 21 Nov 2022 06:02:21 -0800 From: Elliot Berman To: Bjorn Andersson , Jonathan Corbet , Bagas Sanjaya , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , , , , , , Subject: [PATCH v7 20/20] docs: gunyah: Document Gunyah VM Manager Date: Mon, 21 Nov 2022 06:00:09 -0800 Message-ID: <20221121140009.2353512-21-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121140009.2353512-1-quic_eberman@quicinc.com> References: <20221121140009.2353512-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: Byp0FmToY85YvAkIubm7FfoecHUI5HaK X-Proofpoint-GUID: Byp0FmToY85YvAkIubm7FfoecHUI5HaK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-21_13,2022-11-18_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 malwarescore=0 bulkscore=0 phishscore=0 priorityscore=1501 impostorscore=0 suspectscore=0 mlxlogscore=999 lowpriorityscore=0 adultscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211210109 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Document the ioctls and usage of Gunyah VM Manager driver. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 1 + Documentation/virt/gunyah/vm-manager.rst | 91 ++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100644 Documentation/virt/gunyah/vm-manager.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst index fbadbdd24da7..9019a03b6f3e 100644 --- a/Documentation/virt/gunyah/index.rst +++ b/Documentation/virt/gunyah/index.rst @@ -7,6 +7,7 @@ Gunyah Hypervisor .. toctree:: :maxdepth: 1 + vm-manager message-queue Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst new file mode 100644 index 000000000000..16fb2b98c371 --- /dev/null +++ b/Documentation/virt/gunyah/vm-manager.rst @@ -0,0 +1,91 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================= +Virtual Machine Manager +======================= + +The Gunyah Virtual Machine Manager is a Linux driver to support launching +virtual machines using Gunyah. It presently supports launching non-proxy +scheduled Linux-like virtual machines. + +Sample Userspace VMM +==================== + +A sample userspace VMM is included in samples/gunyah/ along with a minimal +devicetree that can be used to launch a VM. To build this sample, enable +CONFIG_SAMPLE_GUNYAH. + +IOCTLs and userspace VMM flows +============================== + +The kernel exposes a char device interface at /dev/gunyah. + +To create a VM, use the GH_CREATE_VM ioctl. A successful call will return a +"Gunyah VM" file descriptor. + +/dev/gunyah API Descriptions +---------------------------- + +GH_CREATE_VM +~~~~~~~~~~~~ + +Creates a Gunyah VM. The argument is reserved for future use and must be 0. + +Gunyah VM API Descriptions +-------------------------- + +GH_VM_SET_USER_MEM_REGION +~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + struct gh_userspace_memory_region { + __u32 label; + __u32 flags; + __u64 guest_phys_addr; + __u64 memory_size; + __u64 userspace_addr; + }; + +This ioctl allows the user to create or delete a memory parcel for a guest +virtual machine. Each memory region is uniquely identified by a label; +attempting to create two regions with the same label is not allowed. + +While VMM is guest-agnostic and allows runtime addition of memory regions, +Linux guest virtual machines do not support accepting memory regions at runtime. +Thus, memory regions should be provided before starting the VM and the VM must +be configured to accept these at boot-up. + +The guest physical address is used by Linux kernel to check that the requested +user regions do not overlap and to help find the corresponding memory region +for calls like GH_VM_SET_DTB_CONFIG. + +To delete a memory region, call GH_VM_SET_USER_MEM_REGION with label set to the +desired region and memory_size set to 0. + +The flags field of gh_userspace_memory_region accepts the following bits. All +other bits must be 0 and are reserved for future use. The ioctl will return +-EINVAL if an unsupported bit is detected. + + - GH_MEM_ALLOW_READ/GH_MEM_ALLOW_WRITE/GH_MEM_ALLOW_EXEC sets read/write/exec + permissions for the guest, respectively. + - GH_MEM_LENT means that the memory will be unmapped from the host and be + unaccessible by the host while the guest has the region. + +GH_VM_SET_DTB_CONFIG +~~~~~~~~~~~~~~~~~~~~ + +:: + + struct gh_vm_dtb_config { + __u64 gpa; + __u64 size; + }; + +This ioctl sets the location of the VM's devicetree blob and is used by Gunyah +Resource Manager to allocate resources. + +GH_VM_START +~~~~~~~~~~~ + +This ioctl starts the VM.