From patchwork Mon Aug 1 21:12:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66669C19F2D for ; Mon, 1 Aug 2022 21:14:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234730AbiHAVN7 (ORCPT ); Mon, 1 Aug 2022 17:13:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232288AbiHAVN6 (ORCPT ); Mon, 1 Aug 2022 17:13:58 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93863186DE; Mon, 1 Aug 2022 14:13:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388437; x=1690924437; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9xD/zs9qtNWomxU7JBnnn4p050BQ9Xldp5Az/0aV2IA=; b=t7XFc1vkkh4T0030CmDl64hkgMU9anAJjsayGCyNX5opk2VtNLJspl5q AMknbaH82cw3ij3tRdqLNH6A7P1PIh/2gl5PvDtWkGnCOpn6KZdSa7S32 egWZ3w2zuJ+JAr+G6U+Sff+tVPpeZZuyVXgy2vyNOlqeVFIWDvA8DvFQq o=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:56 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:56 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:55 -0700 From: Elliot Berman To: Bjorn Andersson , , Jonathan Corbet CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Will Deacon , Catalin Marinas , , Subject: [PATCH v2 01/11] docs: gunyah: Introduce Gunyah Hypervisor Date: Mon, 1 Aug 2022 14:12:30 -0700 Message-ID: <20220801211240.597859-2-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It does not depend on any lower-privileged OS/kernel code for its core functionality. This increases its security and can support a smaller trusted computing based when compared to Type-2 hypervisors. Add documentation describing the Gunyah hypervisor and the main components of the Gunyah hypervisor which are of interest to Linux virtualization development. Signed-off-by: Elliot Berman Reported-by: kernel test robot --- Documentation/virt/gunyah/index.rst | 92 +++++++++++++++++++++ Documentation/virt/gunyah/message-queue.rst | 52 ++++++++++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 7 ++ 4 files changed, 152 insertions(+) create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst new file mode 100644 index 000000000000..e7bb2b14543e --- /dev/null +++ b/Documentation/virt/gunyah/index.rst @@ -0,0 +1,92 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Gunyah Hypervisor +================= + +.. toctree:: + :maxdepth: 1 + + message-queue + +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in +a higher CPU privilege level. It does not depend on any lower-privileged operating system +for its core functionality. This increases its security and can support a much smaller +trusted computing base than a Type-2 hypervisor. + +Gunyah is an open source hypervisor. The source repo is available at +https://github.com/quic/gunyah-hypervisor. + +Gunyah provides these following features. + +- Scheduling: + A scheduler for virtual CPUs (VCPUs) on physical CPUs and enables time-sharing + of the CPUs. +- Memory Management: + APIs handling memory, abstracted as objects, limiting direct use of physical + addresses. Memory ownership and usage tracking of all memory under its control. + Memory partitioning between VMs is a fundamental security feature. +- Interrupt Virtualization: + Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled + in the hypervisor and routed to the assigned VM. +- Inter-VM Communication: + There are several different mechanisms provided for communicating between VMs. +- Virtual platform: + Architectural devices such as interrupt controllers and CPU timers are directly provided + by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI. +- Device Virtualization: + Para-virtualization of devices is supported using inter-VM communication. + +Architectures supported +======================= +AArch64 with a GIC + +Resources and Capabilities +========================== + +Some services or resources provided by the Gunyah hypervisor are described by capability IDs. +For instance, inter-VM communication is performed with doorbells and message queues. The specific +instance of a doorbell is described by a capability ID. These devices are described in Linux as a +struct gunyah_device. + +High level management of these resources is performed by the resource manager VM. RM informs a +guest VM about resources it can access through either the device tree or via guest-initiated RPC. + +Resource Manager +================ + +The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor. +It provides policy enforcement aspects of the virtualization system. The resource manager can +be treated as an extension of the Hypervisor but is separated to its own partition to ensure +that the hypervisor layer itself remains small and secure and to maintain a separation of policy +and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines. + +Communication with the resource manager from each guest VM happens with message-queue.rst. Details +about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c + +:: + +-------+ +--------+ +--------+ + | RM | | VM_A | | VM_B | + +-.-.-.-+ +---.----+ +---.----+ + | | | | + +-.-.-----------.------------.----+ + | | \==========/ | | + | \========================/ | + | Gunyah | + +---------------------------------+ + +The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager. + +The resource manager provides the following features: + +- Generate device-tree overlay +- VM creation and deletion +- VM device-tree management +- VM access control policy +- Interrupt routing configuration + +When booting a virtual machine which uses a devicetree, resource manager overlays a +/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM, +how to communicate with resource manager, and basic description and capabilities of +this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description +of this node. diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst new file mode 100644 index 000000000000..afd405f3a5e1 --- /dev/null +++ b/Documentation/virt/gunyah/message-queue.rst @@ -0,0 +1,52 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Message Queues +============== +Message queue is a simple low-capacity IPC channel between two VMs. It is +intended for sending small control and configuration messages. Each message +queue object is unidirectional, so a full-duplex IPC channel requires a pair of +objects. + +Messages can be up to 240 bytes in length. Longer messages require a further +protocol on top of the message queue messages themselves. For instance, communication +with the resource manager adds a header field for sending longer messages via multiple +message fragments. + +The diagram below shows how message queue works. A typical configuration involves +2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message +queue 2 allows VM_B to send messages to VM_A. + +1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall + with the message to inform the hypervisor to add the message to + message queue 1's queue. +2. Gunyah raises the corresponding interrupt for VM_B when any of these happens: + a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case. + b. Explicility with gh_msgq_push command from VM_A. + c. Message queue has reached a threshold depth. +3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer. + +For VM_B to send a message to VM_A, the process is identical, except that hypercalls +reference message queue 2's capability ID. + +:: + + +---------------+ +-----------------+ +---------------+ + | VM_A | |Gunyah hypervisor| | VM_B | + | | | | | | + | | | | | | + | | Tx | | | | + | |-------->| | Rx vIRQ | | + |gh_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gh_msgq_recv() | + | |<------- | | | | + | | | | | | + | Message Queue | | | | Message Queue | + | driver | | | | driver | + | | | | | | + | | | | | | + | | | | Tx | | + | | Rx vIRQ | |<--------| | + |gh_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gh_msgq_send() | + | | | |-------->| | + | | | | | | + | | | | | | + +---------------+ +-----------------+ +---------------+ diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index 492f0920b988..dd4e8ef284eb 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -14,6 +14,7 @@ Linux Virtualization Support ne_overview acrn/index coco/sev-guest + gunyah/index .. only:: html and subproject diff --git a/MAINTAINERS b/MAINTAINERS index 04ec80ee7352..18fb034526e1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8736,6 +8736,13 @@ L: linux-efi@vger.kernel.org S: Maintained F: block/partitions/efi.* +GUNYAH HYPERVISOR DRIVER +M: Elliot Berman +M: Murali Nalajala +L: linux-arm-msm@vger.kernel.org +S: Maintained +F: Documentation/virt/gunyah/ + HABANALABS PCI DRIVER M: Oded Gabbay S: Supported From patchwork Mon Aug 1 21:12:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42265C25B0D for ; Mon, 1 Aug 2022 21:14:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234930AbiHAVOA (ORCPT ); Mon, 1 Aug 2022 17:14:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234719AbiHAVN7 (ORCPT ); Mon, 1 Aug 2022 17:13:59 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA0BF17E09; Mon, 1 Aug 2022 14:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388438; x=1690924438; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vIfmWBafylqnlRoEuinALPR3jmyroFOxK4aJ7M4ecys=; b=cIn3RmqYxCKUVcyE8Km+7ed9DGJPXolWux/VXATmMPBlmd+UeToO9JN/ QOLbooFgFJmAgDEjD0TsvNqBHKsO2xPxF3xtVMUBRmSi8tjCPmGILlprz ZGZ2zK0d+57cbA+C50y0zMtdtwZmB3audsUAwJg9FTEO3Lu1vVNacmIx0 k=; Received: from unknown (HELO ironmsg02-sd.qualcomm.com) ([10.53.140.142]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:57 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg02-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:56 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:56 -0700 From: Elliot Berman To: Bjorn Andersson , Rob Herring , Krzysztof Kozlowski , CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Jonathan Corbet , Will Deacon , Catalin Marinas , , Subject: [PATCH v2 02/11] dt-bindings: Add binding for gunyah hypervisor Date: Mon, 1 Aug 2022 14:12:31 -0700 Message-ID: <20220801211240.597859-3-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org When Linux is booted as a guest under the Gunyah hypervisor, Gunyah applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Signed-off-by: Elliot Berman --- .../bindings/firmware/gunyah-hypervisor.yaml | 84 +++++++++++++++++++ MAINTAINERS | 1 + 2 files changed, 85 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 000000000000..e50d932e768c --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,84 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Murali Nalajala + - Elliot Berman + +description: |+ + On systems which support devicetree, Gunyah generates and overlays a deviceetree overlay which + describes the basic configuration of the hypervisor. Virtual machines use this information for + initial discovery that they are running as a Gunyah guest VM. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + oneOf: + - items: + - const: gunyah-hypervisor-1.0 + - const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + oneOf: + - items: + - const: gunyah-resource-manager-1-0 + - const: gunyah-resource-manager + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + additionalProperties: false + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "gunyah-hypervisor-1.0", "gunyah-hypervisor"; + + gunyah-resource-mgr@1 { + compatible = "gunyah-resource-manager-1-0", "gunyah-resource-manager"; + interrupts = , /* TX full IRQ */ + ; /* RX empty IRQ */ + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; + /* TX, RX cap ids */ + }; + }; diff --git a/MAINTAINERS b/MAINTAINERS index 18fb034526e1..0cd12ea6c11c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8741,6 +8741,7 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Maintained +F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ HABANALABS PCI DRIVER From patchwork Mon Aug 1 21:12:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A9F1C19F2C for ; Mon, 1 Aug 2022 21:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234951AbiHAVOA (ORCPT ); Mon, 1 Aug 2022 17:14:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232288AbiHAVOA (ORCPT ); Mon, 1 Aug 2022 17:14:00 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B10CF19008; Mon, 1 Aug 2022 14:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388438; x=1690924438; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2wwXDsSDy8neRX8+Ed9SYEvVYy0LIc3XuXU/Jlw0R6s=; b=tgr4LxBa118J6DWD8goSlOwTRpVS7oVuyDqsM94xPFcWNdfG5qD0FqAY oHI/vtPTIx3mouoO/QsmQFEp335+G7fVNgoL/CgnsuSwmp/0pIBhUce83 eVD9w23amelnaMUNIxre5O/mm8H7zNPG2IugX/Gy6h5s09XgZgTGOepBl 0=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:57 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:57 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:56 -0700 From: Elliot Berman To: Bjorn Andersson , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Date: Mon, 1 Aug 2022 14:12:32 -0700 Message-ID: <20220801211240.597859-4-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add initial support to perform Gunyah hypercalls. The arm64 ABI for Gunyah hypercalls generally follows the SMC Calling Convention. Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + arch/arm64/include/asm/gunyah.h | 134 ++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 arch/arm64/include/asm/gunyah.h diff --git a/MAINTAINERS b/MAINTAINERS index 0cd12ea6c11c..02f97ac90cdf 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8743,6 +8743,7 @@ L: linux-arm-msm@vger.kernel.org S: Maintained F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ +F: arch/arm64/include/asm/gunyah.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h new file mode 100644 index 000000000000..4820e9389f40 --- /dev/null +++ b/arch/arm64/include/asm/gunyah.h @@ -0,0 +1,134 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __ASM_GUNYAH_H +#define __ASM_GUNYAH_H + +#include +#include + +#define GH_CALL_TYPE_PLATFORM_CALL 0 +#define GH_CALL_TYPE_HYPERCALL 2 +#define GH_CALL_TYPE_SERVICE 3 +#define GH_CALL_TYPE_SHIFT 14 +#define GH_CALL_FUNCTION_NUM_MASK 0x3fff + +#define GH_SERVICE(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + (GH_CALL_TYPE_SERVICE << GH_CALL_TYPE_SHIFT) \ + | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) + +#define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + (GH_CALL_TYPE_HYPERCALL << GH_CALL_TYPE_SHIFT) \ + | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) + +#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x + +#define __gh_count_args(...) \ + ___gh_count_args(_, ## __VA_ARGS__, 8, 7, 6, 5, 4, 3, 2, 1, 0) + +#define __gh_skip_0(...) __VA_ARGS__ +#define __gh_skip_1(a, ...) __VA_ARGS__ +#define __gh_skip_2(a, b, ...) __VA_ARGS__ +#define __gh_skip_3(a, b, c, ...) __VA_ARGS__ +#define __gh_skip_4(a, b, c, d, ...) __VA_ARGS__ +#define __gh_skip_5(a, b, c, d, e, ...) __VA_ARGS__ +#define __gh_skip_6(a, b, c, d, e, f, ...) __VA_ARGS__ +#define __gh_skip_7(a, b, c, d, e, f, g, ...) __VA_ARGS__ +#define __gh_skip_8(a, b, c, d, e, f, g, h, ...) __VA_ARGS__ +#define __gh_to_res(nargs, ...) __gh_skip_ ## nargs (__VA_ARGS__) + +#define __gh_declare_arg_0(...) + +#define __gh_declare_arg_1(arg1, ...) \ + .a1 = (arg1) + +#define __gh_declare_arg_2(arg1, arg2, ...) \ + __gh_declare_arg_1(arg1), \ + .a2 = (arg2) + +#define __gh_declare_arg_3(arg1, arg2, arg3, ...) \ + __gh_declare_arg_2(arg1, arg2), \ + .a3 = (arg3) + +#define __gh_declare_arg_4(arg1, arg2, arg3, arg4, ...) \ + __gh_declare_arg_3(arg1, arg2, arg3), \ + .a4 = (arg4) + +#define __gh_declare_arg_5(arg1, arg2, arg3, arg4, arg5, ...) \ + __gh_declare_arg_4(arg1, arg2, arg3, arg4), \ + .a5 = (arg5) + +#define __gh_declare_arg_6(arg1, arg2, arg3, arg4, arg5, arg6, ...) \ + __gh_declare_arg_5(arg1, arg2, arg3, arg4, arg5), \ + .a6 = (arg6) + +#define __gh_declare_arg_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7, ...) \ + __gh_declare_arg_6(arg1, arg2, arg3, arg4, arg5, arg6), \ + .a7 = (arg7) + +#define __gh_declare_arg_8(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, ...) \ + __gh_declare_arg_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7), \ + .a8 = (arg8) + +#define ___gh_declare_args(nargs) __gh_declare_arg_ ## nargs +#define __gh_declare_args(nargs) ___gh_declare_args(nargs) +#define _gh_declare_args(nargs, ...) __gh_declare_args(nargs)(__VA_ARGS__) + +#define __gh_assign_res_0(...) + +#define __gh_assign_res_1(r1) \ + r1 = __res.a0 + +#define __gh_assign_res_2(r1, r2) \ + __gh_assign_res_1(r1); \ + r2 = __res.a1 + +#define __gh_assign_res_3(r1, r2, r3) \ + __gh_assign_res_2(r1, r2); \ + r3 = __res.a2 + +#define __gh_assign_res_4(r1, r2, r3, r4) \ + __gh_assign_res_3(r1, r2, r3); \ + r4 = __res.a3 + +#define __gh_assign_res_5(r1, r2, r3, r4, r5) \ + __gh_assign_res_4(r1, r2, r3, r4); \ + r5 = __res.a4 + +#define __gh_assign_res_6(r1, r2, r3, r4, r5, r6) \ + __gh_assign_res_5(r1, r2, r3, r4, r5); \ + r6 = __res.a5 + +#define __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7) \ + __gh_assign_res_6(r1, r2, r3, r4, r5, r6); \ + r7 = __res.a6 + +#define __gh_assign_res_8(r1, r2, r3, r4, r5, r6, r7, r8) \ + __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7); \ + r8 = __res.a7 + +#define ___gh_assign_res(nargs) __gh_assign_res_ ## nargs +#define __gh_assign_res(nargs) ___gh_assign_res(nargs) +#define _gh_assign_res(...) __gh_assign_res(__gh_count_args(__VA_ARGS__))(__VA_ARGS__) + +/** + * arch_gh_hypercall() - Performs an AArch64-specific call into hypervisor using Gunyah ABI + * @hcall_num: Hypercall function ID to invoke + * @nargs: Number of input arguments + * @...: First nargs are the input arguments. Remaining arguments are output variables. + */ +#define arch_gh_hypercall(hcall_num, nargs, ...) \ + do { \ + struct arm_smccc_1_2_regs __res; \ + struct arm_smccc_1_2_regs __args = { \ + .a0 = hcall_num, \ + _gh_declare_args(nargs, __VA_ARGS__) \ + }; \ + arm_smccc_1_2_hvc(&__args, &__res); \ + _gh_assign_res(__gh_to_res(nargs, __VA_ARGS__)); \ + } while (0) + +#endif From patchwork Mon Aug 1 21:12:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594918 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 857D8C25B0E for ; Mon, 1 Aug 2022 21:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235007AbiHAVOB (ORCPT ); Mon, 1 Aug 2022 17:14:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234487AbiHAVOA (ORCPT ); Mon, 1 Aug 2022 17:14:00 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 887573C150; Mon, 1 Aug 2022 14:13:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388439; x=1690924439; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8Zu1w/0V0h7dug7aLTMZQuEKcNOe7CoCHzakPAKWRP0=; b=tAw8LVSOdTCRaKgM3QipjQk3tmpjQgAP3SPTxPRaG61M1ci4Z6q3Vbs2 S5/H57SjIL+Ke/bwBCK2LN5D3TXMZLhnr/eCD1h+oGL+9WGOaTfdH0uua +0aT6xA7rtJbsnrbdQ0yKtaoH/ej91lIt3i+HcxsjpwYVymeQPTYW4rFd 4=; Received: from unknown (HELO ironmsg05-sd.qualcomm.com) ([10.53.140.145]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:58 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg05-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:58 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:57 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 04/11] gunyah: Common types and error codes for Gunyah hypercalls Date: Mon, 1 Aug 2022 14:12:33 -0700 Message-ID: <20220801211240.597859-5-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + include/linux/gunyah.h | 75 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/MAINTAINERS b/MAINTAINERS index 02f97ac90cdf..2e4f1d9ed47b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8744,6 +8744,7 @@ S: Maintained F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah.h +F: include/linux/gunyah.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..69931a0f5736 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,75 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_H +#define _GUNYAH_H + +#include +#include +#include + +typedef u64 gh_capid_t; + +/* Common Gunyah macros */ +#define GH_CAPID_INVAL U64_MAX + +#define GH_ERROR_OK 0 +#define GH_ERROR_UNIMPLEMENTED -1 +#define GH_ERROR_RETRY -2 + +#define GH_ERROR_ARG_INVAL 1 +#define GH_ERROR_ARG_SIZE 2 +#define GH_ERROR_ARG_ALIGN 3 + +#define GH_ERROR_NOMEM 10 + +#define GH_ERROR_ADDR_OVFL 20 +#define GH_ERROR_ADDR_UNFL 21 +#define GH_ERROR_ADDR_INVAL 22 + +#define GH_ERROR_DENIED 30 +#define GH_ERROR_BUSY 31 +#define GH_ERROR_IDLE 32 + +#define GH_ERROR_IRQ_BOUND 40 +#define GH_ERROR_IRQ_UNBOUND 41 + +#define GH_ERROR_CSPACE_CAP_NULL 50 +#define GH_ERROR_CSPACE_CAP_REVOKED 51 +#define GH_ERROR_CSPACE_WRONG_OBJ_TYPE 52 +#define GH_ERROR_CSPACE_INSUF_RIGHTS 53 +#define GH_ERROR_CSPACE_FULL 54 + +#define GH_ERROR_MSGQUEUE_EMPTY 60 +#define GH_ERROR_MSGQUEUE_FULL 61 + +static inline int gh_remap_error(int gh_error) +{ + switch (gh_error) { + case GH_ERROR_OK: + return 0; + case GH_ERROR_NOMEM: + return -ENOMEM; + case GH_ERROR_DENIED: + case GH_ERROR_CSPACE_CAP_NULL: + case GH_ERROR_CSPACE_CAP_REVOKED: + case GH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GH_ERROR_CSPACE_INSUF_RIGHTS: + case GH_ERROR_CSPACE_FULL: + return -EACCES; + case GH_ERROR_BUSY: + case GH_ERROR_IDLE: + return -EBUSY; + case GH_ERROR_IRQ_BOUND: + case GH_ERROR_IRQ_UNBOUND: + case GH_ERROR_MSGQUEUE_FULL: + case GH_ERROR_MSGQUEUE_EMPTY: + return -EPERM; + default: + return -EINVAL; + } +} + +#endif From patchwork Mon Aug 1 21:12:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594917 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D986C25B08 for ; Mon, 1 Aug 2022 21:14:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234487AbiHAVOC (ORCPT ); Mon, 1 Aug 2022 17:14:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231152AbiHAVOA (ORCPT ); Mon, 1 Aug 2022 17:14:00 -0400 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11A0A39BA0; Mon, 1 Aug 2022 14:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388439; x=1690924439; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J9519SgLqABIUbX23vSTKPiAzykXv5PmWxk67sQPv0w=; b=y7Qm6VPecl0if6lnJhERAp5lmIpxHgdmJX7ZR7qf1ss8k3aTRbGPVoTK kYHuLUbgFl7x8FncFA8XfQloxpStMtqlLphqhOVh1e4CoCCI9NDH4zA7t iZrb+GUPNigJQdeUJK3LuIPBgvYTFlPgR9r0TgKAAgpqPbb8HRr1li4dE 0=; Received: from unknown (HELO ironmsg01-sd.qualcomm.com) ([10.53.140.141]) by alexa-out-sd-02.qualcomm.com with ESMTP; 01 Aug 2022 14:13:58 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg01-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:58 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:57 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 05/11] virt: gunyah: Add sysfs nodes Date: Mon, 1 Aug 2022 14:12:34 -0700 Message-ID: <20220801211240.597859-6-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add /sys/hypervisor support when detecting that Linux is running in a Gunyah environment. Export the version of Gunyah which is reported via the hyp_identify hypercall. Signed-off-by: Elliot Berman --- .../ABI/testing/sysfs-hypervisor-gunyah | 37 +++++ MAINTAINERS | 2 + arch/arm64/include/asm/gunyah.h | 4 + drivers/virt/Kconfig | 1 + drivers/virt/Makefile | 1 + drivers/virt/gunyah/Kconfig | 13 ++ drivers/virt/gunyah/Makefile | 4 + drivers/virt/gunyah/sysfs.c | 139 ++++++++++++++++++ 8 files changed, 201 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-hypervisor-gunyah create mode 100644 drivers/virt/gunyah/Kconfig create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/sysfs.c diff --git a/Documentation/ABI/testing/sysfs-hypervisor-gunyah b/Documentation/ABI/testing/sysfs-hypervisor-gunyah new file mode 100644 index 000000000000..41bce1965c91 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-hypervisor-gunyah @@ -0,0 +1,37 @@ +What: /sys/hypervisor/type +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + Type of hypervisor: + "gunyah": Gunyah hypervisor + +What: /sys/hypervisor/features +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + Space separated list of features supported by Linux and Gunyah: + "cspace": Gunyah devices + "doorbell": Sending/receiving virtual interrupts via Gunyah doorbells + "message-queue": Sending/receiving messages via Gunyah message queues + "vic": Interrupt lending + "vpm": Virtual platform management + "vcpu": Virtual CPU management + "memextent": Memory lending/management + "trace": Gunyah hypervisor tracing + + +What: /sys/hypervisor/version/api +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + The Gunyah API version. + +What: /sys/hypervisor/version/variant +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + The Gunyah variant (build) version. \ No newline at end of file diff --git a/MAINTAINERS b/MAINTAINERS index 2e4f1d9ed47b..e63c51ee1a2a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8741,9 +8741,11 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Maintained +F: Documentation/ABI/testing/sysfs-hypervisor-gunyah F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah.h +F: drivers/virt/gunyah/ F: include/linux/gunyah.h HABANALABS PCI DRIVER diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h index 4820e9389f40..3aee35009910 100644 --- a/arch/arm64/include/asm/gunyah.h +++ b/arch/arm64/include/asm/gunyah.h @@ -19,11 +19,15 @@ (GH_CALL_TYPE_SERVICE << GH_CALL_TYPE_SHIFT) \ | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) +#define GH_HYPERCALL_CALL_UID GH_SERVICE(0x3f01) + #define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ ARM_SMCCC_OWNER_VENDOR_HYP, \ (GH_CALL_TYPE_HYPERCALL << GH_CALL_TYPE_SHIFT) \ | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) +#define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000) + #define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x #define __gh_count_args(...) \ diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index 87ef258cec64..259dc2be6cad 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -52,4 +52,5 @@ source "drivers/virt/coco/efi_secret/Kconfig" source "drivers/virt/coco/sev-guest/Kconfig" +source "drivers/virt/gunyah/Kconfig" endif diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index 093674e05c40..ec4fdfa8eef1 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -6,6 +6,7 @@ obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o obj-$(CONFIG_VMGENID) += vmgenid.o obj-y += vboxguest/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) += acrn/ diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..1e493017885e --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + select SYS_HYPERVISOR + help + The Gunyah drivers are the helper interfaces that runs in a guest VM + such as basic inter-VM IPC and signaling mechanisms and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile new file mode 100644 index 000000000000..0aa086f9149f --- /dev/null +++ b/drivers/virt/gunyah/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only + +gunyah-y += sysfs.o +obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c new file mode 100644 index 000000000000..253433a939cf --- /dev/null +++ b/drivers/virt/gunyah/sysfs.c @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gunyah: " fmt + +#include +#include +#include +#include +#include +#include + +#define QC_HYP_UID0 0x19bd54bd +#define QC_HYP_UID1 0x0b37571b +#define QC_HYP_UID2 0x946f609b +#define QC_HYP_UID3 0x54539de6 + +#define GUNYAH_UID0 0x673d5f14 +#define GUNYAH_UID1 0x9265ce36 +#define GUNYAH_UID2 0xa4535fdb +#define GUNYAH_UID3 0xc1d58fcd + +#define gh_uid_matches(prefix, uid) \ + ((uid)[0] == prefix ## _UID0 && (uid)[1] == prefix ## _UID1 && \ + (uid)[2] == prefix ## _UID2 && (uid)[3] == prefix ## _UID3) + +#define GH_API_INFO_API_VERSION(x) (((x) >> 0) & 0x3fff) +#define GH_API_INFO_BIG_ENDIAN(x) (((x) >> 14) & 1) +#define GH_API_INFO_IS_64BIT(x) (((x) >> 15) & 1) +#define GH_API_INFO_VARIANT(x) (((x) >> 56) & 0xff) + +#define GH_IDENTIFY_PARTITION_CSPACE(flags) (((flags)[0] >> 0) & 1) +#define GH_IDENTIFY_DOORBELL(flags) (((flags)[0] >> 1) & 1) +#define GH_IDENTIFY_MSGQUEUE(flags) (((flags)[0] >> 2) & 1) +#define GH_IDENTIFY_VIC(flags) (((flags)[0] >> 3) & 1) +#define GH_IDENTIFY_VPM(flags) (((flags)[0] >> 4) & 1) +#define GH_IDENTIFY_VCPU(flags) (((flags)[0] >> 5) & 1) +#define GH_IDENTIFY_MEMEXTENT(flags) (((flags)[0] >> 6) & 1) +#define GH_IDENTIFY_TRACE_CTRL(flags) (((flags)[0] >> 7) & 1) + +struct gh_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +static struct gh_hypercall_hyp_identify_resp gunyah_api; + +static ssize_t type_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "gunyah\n"); +} +static struct kobj_attribute type_attr = __ATTR_RO(type); + +static ssize_t api_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_API_VERSION(gunyah_api.api_info)); +} +static struct kobj_attribute api_attr = __ATTR_RO(api); + +static ssize_t variant_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_VARIANT(gunyah_api.api_info)); +} +static struct kobj_attribute variant_attr = __ATTR_RO(variant); + +static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "\n"); +} +static struct kobj_attribute features_attr = __ATTR_RO(features); + +static struct attribute *version_attrs[] = { + &api_attr.attr, + &variant_attr.attr, + NULL +}; + +static const struct attribute_group version_group = { + .name = "version", + .attrs = version_attrs, +}; + +static int __init gh_sysfs_register(void) +{ + int ret; + + ret = sysfs_create_file(hypervisor_kobj, &type_attr.attr); + if (ret) + return ret; + + ret = sysfs_create_group(hypervisor_kobj, &version_group); + if (ret) + return ret; + + return sysfs_create_file(hypervisor_kobj, &features_attr.attr); +} + +static void gh_sysfs_unregister(void) +{ + sysfs_remove_file(hypervisor_kobj, &type_attr.attr); + sysfs_remove_group(hypervisor_kobj, &version_group); +} + +static int __init gunyah_init(void) +{ + unsigned long uid[4]; + + arch_gh_hypercall(GH_HYPERCALL_CALL_UID, 0, uid[0], uid[1], uid[2], uid[3]); + + if (!(gh_uid_matches(GUNYAH, uid) || gh_uid_matches(QC_HYP, uid))) + return 0; + + arch_gh_hypercall(GH_HYPERCALL_HYP_IDENTIFY, 0, gunyah_api.api_info, + gunyah_api.flags[0], gunyah_api.flags[1], gunyah_api.flags[2]); + + if (GH_API_INFO_API_VERSION(gunyah_api.api_info) != 1) { + pr_warn("Unrecognized gunyah version: %llu. Currently supported: 1\n", + GH_API_INFO_API_VERSION(gunyah_api.api_info)); + return 0; + } + + pr_notice("Running under Gunyah hypervisor v%lld/%llx\n", + GH_API_INFO_API_VERSION(gunyah_api.api_info), + GH_API_INFO_VARIANT(gunyah_api.api_info)); + + return gh_sysfs_register(); +} +module_init(gunyah_init); + +static void __exit gunyah_exit(void) +{ + gh_sysfs_unregister(); +} +module_exit(gunyah_exit); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Driver"); From patchwork Mon Aug 1 21:12:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EB64C19F2D for ; Mon, 1 Aug 2022 21:14:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235074AbiHAVOC (ORCPT ); Mon, 1 Aug 2022 17:14:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232288AbiHAVOB (ORCPT ); Mon, 1 Aug 2022 17:14:01 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3734417E09; Mon, 1 Aug 2022 14:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388440; x=1690924440; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wckTDe8qexURvwCtE2aZc4qm1X8M+6zM/y6Z0jnbyIo=; b=KFlAlfllooZ5fD+YLt95YXbLEMajZLw8JAn6RdHR5sv0lRngWeMZz0fx /jCRObeNY6lBmuUW7XkHbkBwVKkewwz6QgKzmtRIaMtVm4cbluwL8yPFV L0ZknvNadzHhq28mMXyho+Bvbb7fHyaV65Lyt23cGHq875kKSeBQh1i81 o=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:59 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:59 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:58 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 06/11] virt: gunyah: Add capabilities bus and devices Date: Mon, 1 Aug 2022 14:12:35 -0700 Message-ID: <20220801211240.597859-7-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Some resources provided by the Gunyah hypervisor are described as objects. The objects are identified with a capability ID. For instance, Inter-VM communication is performed with doorbells and message queues. Each doorbell and message queue endpoint can be described consisely as a Linux device. These resources are discovered either on the devicetree or reported by the Resource Manager. Devices on the Gunyah bus are matched with drivers according to the type ID reported by resource manager. Most resources will be discovered directly from the resource manager, so matching directly on type ID seems like sensible design. Each resource may also optionally have an interrupt associated with it and a known partner VM (e.g. which VM is the receiver of a message queue). Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/device.c | 108 +++++++++++++++++++++++++++ drivers/virt/gunyah/gunyah_private.h | 12 +++ drivers/virt/gunyah/sysfs.c | 25 ++++++- include/linux/gunyah.h | 45 +++++++++++ 5 files changed, 189 insertions(+), 3 deletions(-) create mode 100644 drivers/virt/gunyah/device.c create mode 100644 drivers/virt/gunyah/gunyah_private.h diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 0aa086f9149f..3869fb7371df 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only -gunyah-y += sysfs.o +gunyah-y += sysfs.o device.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/device.c b/drivers/virt/gunyah/device.c new file mode 100644 index 000000000000..93595f9a65b9 --- /dev/null +++ b/drivers/virt/gunyah/device.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "ghdev: " fmt + +#include +#include +#include +#include +#include + +#include "gunyah_private.h" + +static int gunyah_match(struct device *dev, struct device_driver *drv) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(drv); + + return ghdev->type == ghdrv->type; +} + +static int gunyah_probe(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver); + + return ghdrv->probe ? ghdrv->probe(ghdev) : 0; +} + +static void gunyah_remove(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver); + + if (ghdrv->remove) + ghdrv->remove(ghdev); +} + +static struct bus_type gunyah_bus = { + .name = "gunyah", + .match = gunyah_match, + .probe = gunyah_probe, + .remove = gunyah_remove, +}; + +int gunyah_register_driver(struct gunyah_driver *ghdrv) +{ + ghdrv->driver.bus = &gunyah_bus; + return driver_register(&ghdrv->driver); +} + +void gunyah_unregister_driver(struct gunyah_driver *ghdrv) +{ + driver_unregister(&ghdrv->driver); +} + +static void gunyah_device_release(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + + kfree(ghdev); +} + +struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type) +{ + struct gunyah_device *ghdev; + + ghdev = kzalloc(sizeof(*ghdev), GFP_KERNEL); + if (!ghdev) + return NULL; + + ghdev->capid = capid; + ghdev->type = type; + ghdev->irq = IRQ_NOTCONNECTED; + ghdev->dev.parent = parent; + ghdev->dev.release = gunyah_device_release; + ghdev->dev.bus = &gunyah_bus; + device_initialize(&ghdev->dev); + return ghdev; +} + +int gunyah_device_add(struct gunyah_device *ghdev) +{ + int ret; + + ret = dev_set_name(&ghdev->dev, "%u.%08llx", ghdev->type, ghdev->capid); + if (ret) + return ret; + + return device_add(&ghdev->dev); +} + +void gunyah_device_remove(struct gunyah_device *ghdev) +{ + device_unregister(&ghdev->dev); +} + +int __init gunyah_bus_init(void) +{ + return bus_register(&gunyah_bus); +} + +void gunyah_bus_exit(void) +{ + bus_unregister(&gunyah_bus); +} diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h new file mode 100644 index 000000000000..5f3832608020 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_private.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_PRIVATE_H +#define _GUNYAH_PRIVATE_H + +int __init gunyah_bus_init(void); +void gunyah_bus_exit(void); + +#endif diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 253433a939cf..220560cb3b1c 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -12,6 +12,8 @@ #include #include +#include "gunyah_private.h" + #define QC_HYP_UID0 0x19bd54bd #define QC_HYP_UID1 0x0b37571b #define QC_HYP_UID2 0x946f609b @@ -67,7 +69,13 @@ static struct kobj_attribute variant_attr = __ATTR_RO(variant); static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) { - return sysfs_emit(buffer, "\n"); + int len = 0; + + if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags)) + len += sysfs_emit_at(buffer, len, "cspace "); + + len += sysfs_emit_at(buffer, len, "\n"); + return len; } static struct kobj_attribute features_attr = __ATTR_RO(features); @@ -105,6 +113,7 @@ static void gh_sysfs_unregister(void) static int __init gunyah_init(void) { + int ret; unsigned long uid[4]; arch_gh_hypercall(GH_HYPERCALL_CALL_UID, 0, uid[0], uid[1], uid[2], uid[3]); @@ -125,12 +134,24 @@ static int __init gunyah_init(void) GH_API_INFO_API_VERSION(gunyah_api.api_info), GH_API_INFO_VARIANT(gunyah_api.api_info)); - return gh_sysfs_register(); + ret = gh_sysfs_register(); + if (ret) + return ret; + + ret = gunyah_bus_init(); + if (ret) + goto err_sysfs; + + return ret; +err_sysfs: + gh_sysfs_unregister(); + return ret; } module_init(gunyah_init); static void __exit gunyah_exit(void) { + gunyah_bus_exit(); gh_sysfs_unregister(); } module_exit(gunyah_exit); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 69931a0f5736..ce35f4491773 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,6 +6,7 @@ #ifndef _GUNYAH_H #define _GUNYAH_H +#include #include #include #include @@ -72,4 +73,48 @@ static inline int gh_remap_error(int gh_error) } } +/* Follows resource manager's resource types for VM_GET_HYP_RESOURCES */ +#define GUNYAH_DEVICE_TYPE_BELL_TX 0 +#define GUNYAH_DEVICE_TYPE_BELL_RX 1 +#define GUNYAH_DEVICE_TYPE_MSGQ_TX 2 +#define GUNYAH_DEVICE_TYPE_MSGQ_RX 3 +#define GUNYAH_DEVICE_TYPE_VCPU 4 + +struct gunyah_device { + u8 type; + gh_capid_t capid; + int irq; + + struct device dev; +}; + +#define to_gunyah_device(dev) container_of(dev, struct gunyah_device, dev) + +static inline void *ghdev_get_drvdata(const struct gunyah_device *ghdev) +{ + return dev_get_drvdata(&ghdev->dev); +} + +static inline void ghdev_set_drvdata(struct gunyah_device *ghdev, void *data) +{ + dev_set_drvdata(&ghdev->dev, data); +} + +struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type); + +int gunyah_device_add(struct gunyah_device *ghdev); +void gunyah_device_remove(struct gunyah_device *ghdev); + +struct gunyah_driver { + struct device_driver driver; + u8 type; + int (*probe)(struct gunyah_device *ghdev); + int (*remove)(struct gunyah_device *ghdev); +}; + +#define to_gunyah_driver(drv) container_of(drv, struct gunyah_driver, driver) + +int gunyah_register_driver(struct gunyah_driver *ghdrv); +void gunyah_unregister_driver(struct gunyah_driver *ghdrv); + #endif From patchwork Mon Aug 1 21:12:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594916 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C616C3F6B0 for ; Mon, 1 Aug 2022 21:14:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235063AbiHAVOE (ORCPT ); Mon, 1 Aug 2022 17:14:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234994AbiHAVOB (ORCPT ); Mon, 1 Aug 2022 17:14:01 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41C853C161; Mon, 1 Aug 2022 14:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388440; x=1690924440; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SWnaPKeSYzf43pvBC3iC3Et/H+kvep4Pqvwdek+EBCk=; b=tmc8mXPW6v1FlPhY2cwwmULwXrlhz7VwyBUybtlnmqItf6pf6NH130yF lwnj4pmrcPHTzl9TcCt3TzGQoXesDDGJ1s22HZrmYMSc5odXnPdKxxuXQ JytBwSLmwbQudikNzQhtTtHwY/lvzmQ9rOoNp7zlly1uPU4tNdR5fmEVv Q=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:00 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:59 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:58 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 07/11] gunyah: msgq: Add Gunyah message queues Date: Mon, 1 Aug 2022 14:12:36 -0700 Message-ID: <20220801211240.597859-8-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah message queues are unidirectional pipelines to communicate between 2 virtual machines, but are typically paired to allow bidirectional communication. The intended use case is for small control messages between 2 VMs, as they support a maximum of 240 bytes. Message queues can be discovered either by resource manager or on the devicetree. To support discovery on the devicetree, client drivers can use gh_msgq_platform_host_attach to allocate the tx and rx message queues according to Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml. Signed-off-by: Elliot Berman --- arch/arm64/include/asm/gunyah.h | 4 + drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/gunyah_private.h | 3 + drivers/virt/gunyah/msgq.c | 223 +++++++++++++++++++++++++++ drivers/virt/gunyah/sysfs.c | 9 ++ include/linux/gunyah.h | 13 ++ 6 files changed, 253 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/msgq.c diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h index 3aee35009910..ba7398bd851b 100644 --- a/arch/arm64/include/asm/gunyah.h +++ b/arch/arm64/include/asm/gunyah.h @@ -27,6 +27,10 @@ | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) #define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000) +#define GH_HYPERCALL_MSGQ_SEND GH_HYPERCALL(0x001B) +#define GH_HYPERCALL_MSGQ_RECV GH_HYPERCALL(0x001C) + +#define GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH BIT(0) #define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 3869fb7371df..94dc8e738911 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only -gunyah-y += sysfs.o device.o +gunyah-y += sysfs.o device.o msgq.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h index 5f3832608020..2ade32bd9bdf 100644 --- a/drivers/virt/gunyah/gunyah_private.h +++ b/drivers/virt/gunyah/gunyah_private.h @@ -9,4 +9,7 @@ int __init gunyah_bus_init(void); void gunyah_bus_exit(void); +int __init gh_msgq_init(void); +void gh_msgq_exit(void); + #endif diff --git a/drivers/virt/gunyah/msgq.c b/drivers/virt/gunyah/msgq.c new file mode 100644 index 000000000000..afc2572d3e7d --- /dev/null +++ b/drivers/virt/gunyah/msgq.c @@ -0,0 +1,223 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "gunyah_private.h" + +struct gh_msgq { + bool ready; + wait_queue_head_t wq; + spinlock_t lock; +}; + +static irqreturn_t gh_msgq_irq_handler(int irq, void *dev) +{ + struct gh_msgq *msgq = dev; + + spin_lock(&msgq->lock); + msgq->ready = true; + spin_unlock(&msgq->lock); + wake_up_interruptible_all(&msgq->wq); + + return IRQ_HANDLED; +} + +static int __gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, u64 tx_flags) +{ + unsigned long flags, gh_error; + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + bool ready; + + spin_lock_irqsave(&msgq->lock, flags); + arch_gh_hypercall(GH_HYPERCALL_MSGQ_SEND, 5, + ghdev->capid, size, (uintptr_t)buff, tx_flags, 0, + gh_error, ready); + switch (gh_error) { + case GH_ERROR_OK: + ret = 0; + msgq->ready = ready; + break; + case GH_ERROR_MSGQUEUE_FULL: + ret = -EAGAIN; + msgq->ready = false; + break; + default: + ret = gh_remap_error(gh_error); + break; + } + spin_unlock_irqrestore(&msgq->lock, flags); + + return ret; +} + +/** + * gh_msgq_send() - Send a message to the client running on a different VM + * @client: The client descriptor that was obtained via gh_msgq_register() + * @buff: Pointer to the buffer where the received data must be placed + * @buff_size: The size of the buffer space available + * @flags: Optional flags to pass to receive the data. For the list of flags, + * see linux/gunyah/gh_msgq.h + * + * Returns: The number of bytes copied to buff. <0 if there was an error. + * + * Note: this function may sleep and should not be called from interrupt context + */ +ssize_t gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags) +{ + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + u64 tx_flags = 0; + + if (flags & GH_MSGQ_TX_PUSH) + tx_flags |= GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH; + + do { + ret = __gh_msgq_send(ghdev, buff, size, tx_flags); + + if (ret == -EAGAIN) { + if (flags & GH_MSGQ_NONBLOCK) + goto out; + if (wait_event_interruptible(msgq->wq, msgq->ready)) + ret = -ERESTARTSYS; + } + } while (ret == -EAGAIN); + +out: + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_send); + +static ssize_t __gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size) +{ + unsigned long flags, gh_error; + size_t recv_size; + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + bool ready; + + spin_lock_irqsave(&msgq->lock, flags); + + arch_gh_hypercall(GH_HYPERCALL_MSGQ_RECV, 4, + ghdev->capid, (uintptr_t)buff, size, 0, + gh_error, recv_size, ready); + switch (gh_error) { + case GH_ERROR_OK: + ret = recv_size; + msgq->ready = ready; + break; + case GH_ERROR_MSGQUEUE_EMPTY: + ret = -EAGAIN; + msgq->ready = false; + break; + default: + ret = gh_remap_error(gh_error); + break; + } + spin_unlock_irqrestore(&msgq->lock, flags); + + return ret; +} + +/** + * gh_msgq_recv() - Receive a message from the client running on a different VM + * @client: The client descriptor that was obtained via gh_msgq_register() + * @buff: Pointer to the buffer where the received data must be placed + * @buff_size: The size of the buffer space available + * @flags: Optional flags to pass to receive the data. For the list of flags, + * see linux/gunyah/gh_msgq.h + * + * Returns: The number of bytes copied to buff. <0 if there was an error. + * + * Note: this function may sleep and should not be called from interrupt context + */ +ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags) +{ + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + + do { + ret = __gh_msgq_recv(ghdev, buff, size); + + if (ret == -EAGAIN) { + if (flags & GH_MSGQ_NONBLOCK) + goto out; + if (wait_event_interruptible(msgq->wq, msgq->ready)) + ret = -ERESTARTSYS; + } + } while (ret == -EAGAIN); + +out: + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_recv); + +static int gh_msgq_probe(struct gunyah_device *ghdev) +{ + struct gh_msgq *msgq; + + msgq = devm_kzalloc(&ghdev->dev, sizeof(*msgq), GFP_KERNEL); + if (!msgq) + return -ENOMEM; + ghdev_set_drvdata(ghdev, msgq); + + msgq->ready = true; /* Assume we can use the message queue right away */ + init_waitqueue_head(&msgq->wq); + spin_lock_init(&msgq->lock); + + return devm_request_irq(&ghdev->dev, ghdev->irq, gh_msgq_irq_handler, 0, + dev_name(&ghdev->dev), msgq); +} + +static struct gunyah_driver gh_msgq_tx_driver = { + .driver = { + .name = "gh_msgq_tx", + .owner = THIS_MODULE, + }, + .type = GUNYAH_DEVICE_TYPE_MSGQ_TX, + .probe = gh_msgq_probe, +}; + +static struct gunyah_driver gh_msgq_rx_driver = { + .driver = { + .name = "gh_msgq_rx", + .owner = THIS_MODULE, + }, + .type = GUNYAH_DEVICE_TYPE_MSGQ_RX, + .probe = gh_msgq_probe, +}; + +int __init gh_msgq_init(void) +{ + int ret; + + ret = gunyah_register_driver(&gh_msgq_tx_driver); + if (ret) + return ret; + + ret = gunyah_register_driver(&gh_msgq_rx_driver); + if (ret) + goto err_rx; + + return ret; +err_rx: + gunyah_unregister_driver(&gh_msgq_tx_driver); + return ret; +} + +void gh_msgq_exit(void) +{ + gunyah_unregister_driver(&gh_msgq_rx_driver); + gunyah_unregister_driver(&gh_msgq_tx_driver); +} diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 220560cb3b1c..7589689e5e92 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -73,6 +73,8 @@ static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags)) len += sysfs_emit_at(buffer, len, "cspace "); + if (GH_IDENTIFY_MSGQUEUE(gunyah_api.flags)) + len += sysfs_emit_at(buffer, len, "message-queue "); len += sysfs_emit_at(buffer, len, "\n"); return len; @@ -142,7 +144,13 @@ static int __init gunyah_init(void) if (ret) goto err_sysfs; + ret = gh_msgq_init(); + if (ret) + goto err_bus; + return ret; +err_bus: + gunyah_bus_exit(); err_sysfs: gh_sysfs_unregister(); return ret; @@ -151,6 +159,7 @@ module_init(gunyah_init); static void __exit gunyah_exit(void) { + gh_msgq_exit(); gunyah_bus_exit(); gh_sysfs_unregister(); } diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index ce35f4491773..099224f9d6d1 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,6 +6,7 @@ #ifndef _GUNYAH_H #define _GUNYAH_H +#include #include #include #include @@ -117,4 +118,16 @@ struct gunyah_driver { int gunyah_register_driver(struct gunyah_driver *ghdrv); void gunyah_unregister_driver(struct gunyah_driver *ghdrv); +#define GH_MSGQ_MAX_MSG_SIZE 1024 + +/* Possible flags to pass for Tx or Rx */ +#define GH_MSGQ_TX_PUSH BIT(0) +#define GH_MSGQ_NONBLOCK BIT(32) + +ssize_t gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags); +ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags); + + #endif From patchwork Mon Aug 1 21:12:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E96EC25B07 for ; Mon, 1 Aug 2022 21:14:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234994AbiHAVOF (ORCPT ); Mon, 1 Aug 2022 17:14:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231152AbiHAVOD (ORCPT ); Mon, 1 Aug 2022 17:14:03 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE79C3C163; Mon, 1 Aug 2022 14:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388440; x=1690924440; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5/aSZM4f3z/9XJTcxshtcyVWDSuHSE81mwr8fNk8a3s=; b=gfxlZ4hs42XaViJiiBLQmO75kLdJG4/+LwdKw6LzUNyA/xLssWimnTCu TBH2LZOiiYs7xsX7oO33bEc10OGRutEEdq1dEtA3pQ/mI2UFo3dq1YAYW 53W1Uq+iKT8MQCXEGfcs+QsDjLrI7W6skuwmhNSB9YhcPKCHGoaK+cbk0 E=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:00 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:00 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:59 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 08/11] gunyah: rsc_mgr: Add resource manager RPC core Date: Mon, 1 Aug 2022 14:12:37 -0700 Message-ID: <20220801211240.597859-9-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org The resource manager is a special virtual machine which is always running on a Gunyah system. It provides APIs for creating and destroying VMs, secure memory management, sharing/lending of memory between VMs, and setup of inter-VM communication. Calls to the resource manager are made via message queues. This patch implements the basic probing and RPC mechanism to make those API calls. Request/response calls can be made with gh_rm_call. Drivers can also register to notifications pushed by RM via gh_rm_register_notifier Specific API calls that resource manager supports will be implemented in subsequent patches. Signed-off-by: Elliot Berman --- MAINTAINERS | 2 +- drivers/virt/gunyah/Kconfig | 1 + drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_private.h | 3 + drivers/virt/gunyah/rsc_mgr.c | 623 +++++++++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 34 ++ drivers/virt/gunyah/sysfs.c | 7 + include/linux/gunyah_rsc_mgr.h | 29 ++ 8 files changed, 699 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/rsc_mgr.c create mode 100644 drivers/virt/gunyah/rsc_mgr.h create mode 100644 include/linux/gunyah_rsc_mgr.h diff --git a/MAINTAINERS b/MAINTAINERS index e63c51ee1a2a..dcd961d6623c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8746,7 +8746,7 @@ F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah.h F: drivers/virt/gunyah/ -F: include/linux/gunyah.h +F: include/linux/gunyah*.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 1e493017885e..0adb6efd4848 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -4,6 +4,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 select SYS_HYPERVISOR + select AUXILIARY_BUS help The Gunyah drivers are the helper interfaces that runs in a guest VM such as basic inter-VM IPC and signaling mechanisms and higher level diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 94dc8e738911..86655bca8944 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only gunyah-y += sysfs.o device.o msgq.o +gunyah-y += rsc_mgr.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h index 2ade32bd9bdf..6483ffa8c15d 100644 --- a/drivers/virt/gunyah/gunyah_private.h +++ b/drivers/virt/gunyah/gunyah_private.h @@ -12,4 +12,7 @@ void gunyah_bus_exit(void); int __init gh_msgq_init(void); void gh_msgq_exit(void); +int __init gh_rsc_mgr_init(void); +void gh_rsc_mgr_exit(void); + #endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c new file mode 100644 index 000000000000..b8268ee02fab --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -0,0 +1,623 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "gunyah_private.h" +#include "rsc_mgr.h" + +/* Resource Manager Header */ +struct gh_rm_rpc_hdr { + u8 version:4, + hdr_words:4; + u8 type:2, + fragments:6; + u16 seq; + u32 msg_id; +} __packed; + +/* Standard reply header */ +struct gh_rm_rpc_reply_hdr { + struct gh_rm_rpc_hdr rpc_hdr; + u32 err_code; +} __packed; + +/* RPC Header versions */ +#define GH_RM_RPC_HDR_VERSION_ONE 0x1 + +/* RPC Header words */ +#define GH_RM_RPC_HDR_WORDS 0x2 + +/* RPC Message types */ +#define GH_RM_RPC_TYPE_CONT 0x0 +#define GH_RM_RPC_TYPE_REQ 0x1 +#define GH_RM_RPC_TYPE_RPLY 0x2 +#define GH_RM_RPC_TYPE_NOTIF 0x3 + +#define GH_RM_MAX_NUM_FRAGMENTS 62 + +#define GH_RM_MAX_MSG_SIZE (GH_MSGQ_MAX_MSG_SIZE - sizeof(struct gh_rm_rpc_hdr)) + +/** + * struct gh_rm_connection - Represents a complete message from resource manager + * @payload: Combined payload of all the fragments (i.e. msg headers stripped off). + * @size: Size of the payload. + * @ret: Linux return code, set in case there was an error processing the connection. + * @msg_id: Message ID from the header. + * @type: GH_RM_RPC_TYPE_RPLY or GH_RM_RPC_TYPE_NOTIF. + * @num_fragments: total number of fragments expected to be received for this connection. + * @fragments_recieved: fragments received so far. + * @rm_error: For request/reply sequences with standard replies. + * @seq: Sequence ID for the main message. + */ +struct gh_rm_connection { + void *payload; + size_t size; + int ret; + u32 msg_id; + u8 type; + + u8 num_fragments; + u8 fragments_received; + + /* only for req/reply sequence */ + u32 rm_error; + u16 seq; + struct completion seq_done; +}; + +struct gh_rm_notif_complete { + struct gh_rm_connection *conn; + struct work_struct work; +}; + +struct gh_rsc_mgr { + struct task_struct *recv_task; + struct gunyah_device *msgq_tx, *msgq_rx; + + struct idr call_idr; + struct mutex call_idr_lock; + + struct mutex send_lock; +}; + +static struct gh_rsc_mgr *__rsc_mgr; +SRCU_NOTIFIER_HEAD_STATIC(gh_rm_notifier); + +static struct gh_rm_connection *gh_rm_alloc_connection(u32 msg_id, u8 type) +{ + struct gh_rm_connection *connection; + + connection = kzalloc(sizeof(*connection), GFP_KERNEL); + if (!connection) + return NULL; + + connection->type = type; + connection->msg_id = msg_id; + + return connection; +} + +/** + * gh_rm_init_connection_payload() - Fills the first message for a connection. + */ +static int gh_rm_init_connection_payload(struct gh_rm_connection *connection, void *msg, + size_t hdr_size, size_t payload_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t max_buf_size; + + connection->num_fragments = hdr->fragments; + connection->fragments_received = 0; + connection->type = hdr->type; + + /* There's not going to be any payload, no need to allocate buffer. */ + if (!payload_size && !connection->num_fragments) + return 0; + + /* + * maximum payload size is GH_MSGQ_MAX_MSG_SIZE - hdr_size + * and can received (hdr->fragments + 1) of those + */ + max_buf_size = (GH_MSGQ_MAX_MSG_SIZE - hdr_size) * (hdr->fragments + 1); + + connection->payload = kzalloc(max_buf_size, GFP_KERNEL); + if (!connection->payload) + return -ENOMEM; + + memcpy(connection->payload, msg + hdr_size, payload_size); + connection->size = payload_size; + return 0; +} + +static void gh_rm_notif_work(struct work_struct *work) +{ + struct gh_rm_notif_complete *notif = container_of(work, struct gh_rm_notif_complete, work); + struct gh_rm_connection *connection = notif->conn; + u32 notif_id = connection->msg_id; + struct gh_rm_notification notification = { + .buff = connection->payload, + .size = connection->size, + }; + + srcu_notifier_call_chain(&gh_rm_notifier, notif_id, ¬ification); + + kfree(connection->payload); + kfree(connection); + kfree(notif); +} + +static struct gh_rm_connection *gh_rm_process_notif(struct gh_rsc_mgr *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + + connection = gh_rm_alloc_connection(hdr->msg_id, hdr->type); + if (!connection) { + pr_err("Failed to alloc connection for notification, dropping.\n"); + return NULL; + } + + if (gh_rm_init_connection_payload(connection, msg, sizeof(*hdr), msg_size - sizeof(*hdr))) { + pr_err("Failed to alloc connection buffer for notification, dropping.\n"); + kfree(connection); + return NULL; + } + + return connection; +} + +static struct gh_rm_connection *gh_rm_process_rply(struct gh_rsc_mgr *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_reply_hdr *reply_hdr = msg; + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) + return ERR_PTR(-ERESTARTSYS); + + connection = idr_find(&rsc_mgr->call_idr, hdr->seq); + mutex_unlock(&rsc_mgr->call_idr_lock); + + if (!connection) { + pr_err("Failed to find connection for sequence %u\n", hdr->seq); + return NULL; + } + if (connection->msg_id != hdr->msg_id) { + pr_err("Reply for sequence %u expected msg_id: %x but got %x\n", hdr->seq, + connection->msg_id, hdr->msg_id); + /* + * Don't complete connection and error the client, maybe resource manager will + * send us the expected reply sequence soon. + */ + return NULL; + } + + if (gh_rm_init_connection_payload(connection, msg, sizeof(*reply_hdr), + msg_size - sizeof(*reply_hdr))) { + pr_err("Failed to alloc connection buffer for sequence %d\n", hdr->seq); + /* Send connection complete and error the client. */ + connection->ret = -ENOMEM; + complete(&connection->seq_done); + return NULL; + } + + connection->rm_error = reply_hdr->err_code; + return connection; +} + +static void gh_rm_process_cont(struct gh_rm_connection *connection, void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t payload_size = msg_size - sizeof(*hdr); + + /* + * hdr->fragments and hdr->msg_id preserves the value from first reply or notif message. + * For sake of sanity, check if it's still intact. + */ + if (connection->msg_id != hdr->msg_id) + pr_warn("Appending mismatched continuation with id %d to connection with id %d\n", + hdr->msg_id, connection->msg_id); + if (connection->num_fragments != hdr->fragments) + pr_warn("Number of fragments mismatch for seq: %d\n", hdr->seq); + + memcpy(connection->payload + connection->size, msg + sizeof(*hdr), payload_size); + connection->size += payload_size; + connection->fragments_received++; +} + +static bool gh_rm_complete_connection(struct gh_rm_connection *connection) +{ + struct gh_rm_notif_complete *notif_work; + + if (!connection) + return false; + + if (connection->fragments_received != connection->num_fragments) + return false; + + switch (connection->type) { + case GH_RM_RPC_TYPE_RPLY: + complete(&connection->seq_done); + break; + case GH_RM_RPC_TYPE_NOTIF: + notif_work = kzalloc(sizeof(*notif_work), GFP_KERNEL); + if (notif_work == NULL) + break; + + notif_work->conn = connection; + INIT_WORK(¬if_work->work, gh_rm_notif_work); + + schedule_work(¬if_work->work); + break; + default: + pr_err("Invalid message type (%d) received\n", connection->type); + break; + } + + return true; +} + +static void gh_rm_abort_connection(struct gh_rm_connection *connection) +{ + switch (connection->type) { + case GH_RM_RPC_TYPE_RPLY: + connection->ret = -EIO; + complete(&connection->seq_done); + break; + case GH_RM_RPC_TYPE_NOTIF: + fallthrough; + default: + kfree(connection->payload); + kfree(connection); + } +} + +static int gh_rm_recv_task_fn(void *data) +{ + struct gh_rsc_mgr *rsc_mgr = data; + struct gh_rm_connection *connection = NULL; + struct gh_rm_rpc_hdr *hdr = NULL; + ssize_t msg_size; + void *msg; + + msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + while (!kthread_should_stop()) { + /* Block until a new message is received */ + msg_size = gh_msgq_recv(rsc_mgr->msgq_rx, msg, GH_MSGQ_MAX_MSG_SIZE, 0); + if (msg_size < 0) { + pr_err("Failed to receive the message: %ld\n", msg_size); + continue; + } else if (msg_size <= sizeof(struct gh_rm_rpc_hdr)) { + pr_err("Invalid message size received: %ld is too small\n", msg_size); + continue; + } + + hdr = msg; + switch (hdr->type) { + case GH_RM_RPC_TYPE_NOTIF: + if (connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + pr_warn("Received start of new notification without finishing existing message series.\n"); + gh_rm_abort_connection(connection); + } + connection = gh_rm_process_notif(rsc_mgr, msg, msg_size); + break; + case GH_RM_RPC_TYPE_RPLY: + if (connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + pr_warn("Received start of new reply without finishing existing message series.\n"); + gh_rm_abort_connection(connection); + } + connection = gh_rm_process_rply(rsc_mgr, msg, msg_size); + break; + case GH_RM_RPC_TYPE_CONT: + if (!connection) { + pr_warn("Received a continuation message without receiving initial message\n"); + break; + } + gh_rm_process_cont(connection, msg, msg_size); + break; + default: + pr_err("Invalid message type (%d) received\n", hdr->type); + continue; + } + + if (gh_rm_complete_connection(connection)) + connection = NULL; + } + + return 0; +} + +static int gh_rm_send_request(struct gh_rsc_mgr *rsc_mgr, u32 message_id, + const void *req_buff, size_t req_buff_size, + struct gh_rm_connection *connection) +{ + size_t buff_size_remaining = req_buff_size; + const void *req_buff_curr = req_buff; + struct gh_rm_rpc_hdr *hdr; + unsigned long tx_flags; + u32 num_fragments = 0; + size_t payload_size; + void *msg; + int i, ret = 0; + + if (req_buff_size > GH_RM_MAX_MSG_SIZE) + num_fragments = req_buff_size / GH_RM_MAX_MSG_SIZE; + + if (WARN(num_fragments > GH_RM_MAX_NUM_FRAGMENTS, + "Limit exceeded for the number of fragments: %u\n", num_fragments)) + return -E2BIG; + + /* + * The above calculation also includes the count for the 'request' packet. + * Exclude it as the header needs to fill the num. of fragments to follow. + */ + if (num_fragments) + num_fragments--; + + if (mutex_lock_interruptible(&rsc_mgr->send_lock)) + return -ERESTARTSYS; + + msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL); + if (!msg) { + mutex_unlock(&rsc_mgr->send_lock); + return -ENOMEM; + } + + /* Consider also the 'request' packet for the loop count */ + for (i = 0; i <= num_fragments; i++) { + if (buff_size_remaining > GH_RM_MAX_MSG_SIZE) { + payload_size = GH_RM_MAX_MSG_SIZE; + buff_size_remaining -= payload_size; + } else { + payload_size = buff_size_remaining; + } + + memset(msg, 0, GH_MSGQ_MAX_MSG_SIZE); + + /* Fill header */ + hdr = msg; + hdr->version = GH_RM_RPC_HDR_VERSION_ONE; + hdr->hdr_words = GH_RM_RPC_HDR_WORDS; + hdr->type = i == 0 ? GH_RM_RPC_TYPE_REQ : GH_RM_RPC_TYPE_CONT; + hdr->fragments = num_fragments; + hdr->seq = connection->seq; + hdr->msg_id = message_id; + + /* Copy payload */ + memcpy(msg + sizeof(*hdr), req_buff_curr, payload_size); + req_buff_curr += payload_size; + + /* Force the last fragment to be sent immediately to the receiver */ + tx_flags = (i == num_fragments) ? GH_MSGQ_TX_PUSH : 0; + + ret = gh_msgq_send(rsc_mgr->msgq_tx, msg, sizeof(*hdr) + payload_size, tx_flags); + + if (ret < 0) + break; + } + + mutex_unlock(&rsc_mgr->send_lock); + return ret < 0 ? ret : 0; +} + +/** + * gh_rm_call: Achieve request-response type communication with RPC + * @message_id: The RM RPC message-id + * @req_buff: Request buffer that contains the payload + * @req_buff_size: Total size of the payload + * @resp_buf: Pointer to a response buffer + * @resp_buff_size: Size of the response buffer + * @reply_err_code: Returns Gunyah standard error code for the response + * + * Make a request to the RM-VM and wait for reply back. For a successful + * response, the function returns the payload. The size of the payload is set in resp_buff_size. + * The resp_buf should be freed by the caller. + * + * Context: Process context. Will sleep waiting for reply. + * Return: >0 is standard reply error from RM. <0 on internal error. + */ +int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, + void **resp_buf, size_t *resp_buff_size) +{ + struct gh_rm_connection *connection; + int ret; + struct gh_rsc_mgr *rsc_mgr = __rsc_mgr; + + /* messaged_id 0 is reserved */ + if (!message_id) + return -EINVAL; + + if (!rsc_mgr) + return -EPROBE_DEFER; + + connection = gh_rm_alloc_connection(message_id, GH_RM_RPC_TYPE_RPLY); + if (!connection) + return -ENOMEM; + + init_completion(&connection->seq_done); + + /* Allocate a new seq number for this connection */ + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) { + kfree(connection); + return -ERESTARTSYS; + } + connection->seq = idr_alloc_cyclic(&rsc_mgr->call_idr, connection, 0, U16_MAX, GFP_KERNEL); + mutex_unlock(&rsc_mgr->call_idr_lock); + + /* Send the request to the Resource Manager */ + ret = gh_rm_send_request(rsc_mgr, message_id, req_buff, req_buff_size, connection); + if (ret < 0) + goto out; + + /* Wait for response */ + wait_for_completion(&connection->seq_done); + + if (connection->ret) { + ret = connection->ret; + kfree(connection->payload); + goto out; + } + + if (connection->rm_error) { + ret = connection->rm_error; + kfree(connection->payload); + goto out; + } + + *resp_buf = connection->payload; + *resp_buff_size = connection->size; + +out: + mutex_lock(&rsc_mgr->call_idr_lock); + idr_remove(&rsc_mgr->call_idr, connection->seq); + mutex_unlock(&rsc_mgr->call_idr_lock); + + kfree(connection); + return ret; +} + +int gh_rm_register_notifier(struct notifier_block *nb) +{ + return srcu_notifier_chain_register(&gh_rm_notifier, nb); +} +EXPORT_SYMBOL_GPL(gh_rm_register_notifier); + +int gh_rm_unregister_notifier(struct notifier_block *nb) +{ + return srcu_notifier_chain_unregister(&gh_rm_notifier, nb); +} +EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier); + +static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev, + u8 gh_type, int idx) +{ + int irq, ret; + u64 capid; + struct device_node *node = pdev->dev.of_node; + struct gunyah_device *ghdev; + + irq = platform_get_irq(pdev, idx); + if (irq < 0) { + dev_err(&pdev->dev, "Failed to get irq%d: %d\n", idx, irq); + return ERR_PTR(irq); + } + + ret = of_property_read_u64_index(node, "reg", idx, &capid); + if (ret) { + dev_err(&pdev->dev, "Failed to get capid%d: %d\n", idx, ret); + return ERR_PTR(ret); + } + + ghdev = gunyah_device_alloc(&pdev->dev, capid, gh_type); + ghdev->irq = irq; + ret = gunyah_device_add(ghdev); + if (ret) { + kfree(ghdev); + return ERR_PTR(ret); + } + + return ghdev; +} + +static int gh_rm_drv_probe(struct platform_device *pdev) +{ + struct gh_rsc_mgr *rsc_mgr; + int ret; + + rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); + if (!rsc_mgr) + return -ENOMEM; + platform_set_drvdata(pdev, rsc_mgr); + + mutex_init(&rsc_mgr->call_idr_lock); + idr_init(&rsc_mgr->call_idr); + mutex_init(&rsc_mgr->send_lock); + + rsc_mgr->msgq_tx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, 0); + if (IS_ERR(rsc_mgr->msgq_tx)) + return PTR_ERR(rsc_mgr->msgq_tx); + rsc_mgr->msgq_rx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_RX, 1); + if (IS_ERR(rsc_mgr->msgq_rx)) { + ret = PTR_ERR(rsc_mgr->msgq_rx); + goto err_msgq_tx; + } + + rsc_mgr->recv_task = kthread_run(gh_rm_recv_task_fn, rsc_mgr, "gh_rm_recv_task"); + if (IS_ERR_OR_NULL(rsc_mgr->recv_task)) { + ret = PTR_ERR(rsc_mgr->recv_task); + goto err_msgq; + } + + __rsc_mgr = rsc_mgr; + + return 0; + +err_msgq: + gunyah_device_remove(rsc_mgr->msgq_rx); +err_msgq_tx: + gunyah_device_remove(rsc_mgr->msgq_tx); + return ret; +} + +static int gh_rm_drv_remove(struct platform_device *pdev) +{ + struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev); + + gunyah_device_remove(rsc_mgr->msgq_tx); + gunyah_device_remove(rsc_mgr->msgq_rx); + + return 0; +} + +static const struct of_device_id gh_rm_of_match[] = { + { .compatible = "gunyah-resource-manager" }, + { } +}; +MODULE_DEVICE_TABLE(of, gh_rm_of_match); + +static struct platform_driver gh_rsc_mgr_driver = { + .probe = gh_rm_drv_probe, + .remove = gh_rm_drv_remove, + .driver = { + .name = "gh_rsc_mgr", + .of_match_table = gh_rm_of_match, + }, +}; + +int __init gh_rsc_mgr_init(void) +{ + return platform_driver_register(&gh_rsc_mgr_driver); +} + +void gh_rsc_mgr_exit(void) +{ + platform_driver_unregister(&gh_rsc_mgr_driver); +} diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h new file mode 100644 index 000000000000..e4f2499267bf --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __GH_RSC_MGR_PRIV_H +#define __GH_RSC_MGR_PRIV_H + +#include + +/* RM Error codes */ +#define GH_RM_ERROR_OK 0x0 +#define GH_RM_ERROR_UNIMPLEMENTED 0xFFFFFFFF +#define GH_RM_ERROR_NOMEM 0x1 +#define GH_RM_ERROR_NORESOURCE 0x2 +#define GH_RM_ERROR_DENIED 0x3 +#define GH_RM_ERROR_INVALID 0x4 +#define GH_RM_ERROR_BUSY 0x5 +#define GH_RM_ERROR_ARGUMENT_INVALID 0x6 +#define GH_RM_ERROR_HANDLE_INVALID 0x7 +#define GH_RM_ERROR_VALIDATE_FAILED 0x8 +#define GH_RM_ERROR_MAP_FAILED 0x9 +#define GH_RM_ERROR_MEM_INVALID 0xA +#define GH_RM_ERROR_MEM_INUSE 0xB +#define GH_RM_ERROR_MEM_RELEASED 0xC +#define GH_RM_ERROR_VMID_INVALID 0xD +#define GH_RM_ERROR_LOOKUP_FAILED 0xE +#define GH_RM_ERROR_IRQ_INVALID 0xF +#define GH_RM_ERROR_IRQ_INUSE 0x10 +#define GH_RM_ERROR_IRQ_RELEASED 0x11 + +int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, + void **resp_buf, size_t *resp_buff_size); + +#endif diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 7589689e5e92..7c0efc80f85e 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -148,7 +148,13 @@ static int __init gunyah_init(void) if (ret) goto err_bus; + ret = gh_rsc_mgr_init(); + if (ret) + goto err_msgq; + return ret; +err_msgq: + gh_msgq_exit(); err_bus: gunyah_bus_exit(); err_sysfs: @@ -159,6 +165,7 @@ module_init(gunyah_init); static void __exit gunyah_exit(void) { + gh_rsc_mgr_exit(); gh_msgq_exit(); gunyah_bus_exit(); gh_sysfs_unregister(); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h new file mode 100644 index 000000000000..015bd851e1a3 --- /dev/null +++ b/include/linux/gunyah_rsc_mgr.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_RSC_MGR_H +#define _GUNYAH_RSC_MGR_H + +#include +#include +#include + +typedef u16 gh_vmid_t; +typedef u32 gh_virq_handle_t; + +#define GH_VMID_INVAL U16_MAX + +/* Gunyah recognizes VMID0 as an alias to the current VM's ID */ +#define GH_VMID_SELF 0 + +struct gh_rm_notification { + const void *buff; + const size_t size; +}; + +int gh_rm_register_notifier(struct notifier_block *nb); +int gh_rm_unregister_notifier(struct notifier_block *nb); + +#endif From patchwork Mon Aug 1 21:12:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31F83C2BB47 for ; Mon, 1 Aug 2022 21:14:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234358AbiHAVOE (ORCPT ); Mon, 1 Aug 2022 17:14:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235059AbiHAVOC (ORCPT ); Mon, 1 Aug 2022 17:14:02 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C03639BA0; Mon, 1 Aug 2022 14:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388441; x=1690924441; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yzAyPrU/qCIlo6Cf2CdARPBaTkg8+aYwmjRonAsM36o=; b=aD5ruyvfjuJ+9FFS5GQRHn+It4/gdAvP7OL8ybcouD3etPzpl1aPF9zB WJsXttkMswxmmjUBZT89KHFfJexBs69kddbg+Nn2cOIZX46meTACXkW7t 9OdcCyxnmXKr/GBXDZZhe9Ykfd3ZtxAe7tPMy/3ttvW2m6MWeWjGp5KBZ E=; Received: from unknown (HELO ironmsg05-sd.qualcomm.com) ([10.53.140.145]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:01 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg05-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:01 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:14:00 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Date: Mon, 1 Aug 2022 14:12:38 -0700 Message-ID: <20220801211240.597859-10-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah resource manager exposes a concrete functionalities which complicate a single resource manager driver. Use auxiliary bus to help split high level functions for the resource manager and keep the primary resource manager driver focused on the RPC with RM itself. Delegate Resource Manager's console functionality to the auxiliary bus. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/rsc_mgr.c | 61 ++++++++++++++++++++++++++++++++++- 1 file changed, 60 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index b8268ee02fab..44b22cef7d44 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -91,6 +91,11 @@ struct gh_rm_notif_complete { struct work_struct work; }; +struct gh_rsc_mgr_adev { + struct auxiliary_device adev; + struct list_head list; +}; + struct gh_rsc_mgr { struct task_struct *recv_task; struct gunyah_device *msgq_tx, *msgq_rx; @@ -99,6 +104,13 @@ struct gh_rsc_mgr { struct mutex call_idr_lock; struct mutex send_lock; + + struct list_head adevs; +}; + +/* List of auxiliary devices which resource manager creates */ +static const char * const adev_names[] = { + "console", }; static struct gh_rsc_mgr *__rsc_mgr; @@ -516,6 +528,14 @@ int gh_rm_unregister_notifier(struct notifier_block *nb) } EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier); +static void gh_rm_adev_release(struct device *dev) +{ + struct gh_rsc_mgr_adev *rm_adev = container_of(dev, struct gh_rsc_mgr_adev, adev.dev); + + list_del(&rm_adev->list); + kfree(rm_adev); +} + static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev, u8 gh_type, int idx) { @@ -550,7 +570,9 @@ static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_de static int gh_rm_drv_probe(struct platform_device *pdev) { struct gh_rsc_mgr *rsc_mgr; - int ret; + struct gh_rsc_mgr_adev *rm_adev; + struct list_head *l, *n; + int ret, i; rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); if (!rsc_mgr) @@ -560,6 +582,7 @@ static int gh_rm_drv_probe(struct platform_device *pdev) mutex_init(&rsc_mgr->call_idr_lock); idr_init(&rsc_mgr->call_idr); mutex_init(&rsc_mgr->send_lock); + INIT_LIST_HEAD(&rsc_mgr->adevs); rsc_mgr->msgq_tx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, 0); if (IS_ERR(rsc_mgr->msgq_tx)) @@ -576,10 +599,38 @@ static int gh_rm_drv_probe(struct platform_device *pdev) goto err_msgq; } + for (i = 0; i < ARRAY_SIZE(adev_names); i++) { + rm_adev = kzalloc(sizeof(*rm_adev), GFP_KERNEL); + + rm_adev->adev.dev.parent = &pdev->dev; + rm_adev->adev.dev.release = gh_rm_adev_release; + rm_adev->adev.name = adev_names[i]; + ret = auxiliary_device_init(&rm_adev->adev); + if (ret) { + kfree(rm_adev); + goto err_adevs; + } + + list_add(&rm_adev->list, &rsc_mgr->adevs); + + ret = auxiliary_device_add(&rm_adev->adev); + if (ret) { + auxiliary_device_uninit(&rm_adev->adev); + goto err_adevs; + } + } + __rsc_mgr = rsc_mgr; return 0; +err_adevs: + list_for_each_safe(l, n, &rsc_mgr->adevs) { + rm_adev = container_of(l, struct gh_rsc_mgr_adev, list); + auxiliary_device_delete(&rm_adev->adev); + auxiliary_device_uninit(&rm_adev->adev); + } + err_msgq: gunyah_device_remove(rsc_mgr->msgq_rx); err_msgq_tx: @@ -590,6 +641,14 @@ static int gh_rm_drv_probe(struct platform_device *pdev) static int gh_rm_drv_remove(struct platform_device *pdev) { struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev); + struct gh_rsc_mgr_adev *rm_adev; + struct list_head *l, *n; + + list_for_each_safe(l, n, &rsc_mgr->adevs) { + rm_adev = container_of(l, struct gh_rsc_mgr_adev, list); + auxiliary_device_delete(&rm_adev->adev); + auxiliary_device_uninit(&rm_adev->adev); + } gunyah_device_remove(rsc_mgr->msgq_tx); gunyah_device_remove(rsc_mgr->msgq_rx); From patchwork Mon Aug 1 21:12:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56D35C32757 for ; Mon, 1 Aug 2022 21:14:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231152AbiHAVOG (ORCPT ); Mon, 1 Aug 2022 17:14:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235085AbiHAVOD (ORCPT ); Mon, 1 Aug 2022 17:14:03 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 143EB3C161; Mon, 1 Aug 2022 14:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388442; x=1690924442; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z7EpwCPH7kpklNw4k3PRnXjN7KwyE+imFag8eqKdl+0=; b=k98/MMzsNs/Bo++eUtZf77a/2elYRD9v2+QQrur+9zfm/9FTuz0Fujub l5I6TFQa5/9QOk2uzwHq9T6OOVPkYtS3mzpp3wa2zasShK0RpElLJjEw8 bTkiWMqkdhhxg6/JdpNuP5A9G0xDWIoopuU0bAKiRGWhLta99/tSr5hLn A=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:01 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:01 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:14:00 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 10/11] gunyah: rsc_mgr: Add RPC for console services Date: Mon, 1 Aug 2022 14:12:39 -0700 Message-ID: <20220801211240.597859-11-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah resource manager defines a simple API for virtual machine log sharing with the console service. A VM's own log can be opened by using GH_VMID_SELF. Another VM's log can be accessed via its VMID. Once opened, characters can be written to the log with a write command. Characters are received with resource manager notifications (using ID GH_RM_NOTIF_VM_CONSOLE_CHARS). These high level rpc calls are kept in drivers/virt/gunyah/rsc_mgr_rpc.c. Future RPC calls, e.g. to launch a VM will also be maintained in this file. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 4 +- drivers/virt/gunyah/rsc_mgr.h | 22 +++++ drivers/virt/gunyah/rsc_mgr_rpc.c | 151 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 16 ++++ 4 files changed, 191 insertions(+), 2 deletions(-) create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 86655bca8944..b3f15c052297 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only gunyah-y += sysfs.o device.o msgq.o -gunyah-y += rsc_mgr.o -obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file +gunyah-y += rsc_mgr.o rsc_mgr_rpc.o +obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index e4f2499267bf..c82a2e97ad49 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -28,6 +28,28 @@ #define GH_RM_ERROR_IRQ_INUSE 0x10 #define GH_RM_ERROR_IRQ_RELEASED 0x11 +/* Message IDs: VM Management */ +#define GH_RM_RPC_VM_GET_VMID 0x56000024 + +/* Message IDs: VM Services */ +#define GH_RM_RPC_VM_CONSOLE_OPEN_ID 0x56000081 +#define GH_RM_RPC_VM_CONSOLE_CLOSE_ID 0x56000082 +#define GH_RM_RPC_VM_CONSOLE_WRITE_ID 0x56000083 +#define GH_RM_RPC_VM_CONSOLE_FLUSH_ID 0x56000084 + +/* Call: CONSOLE_OPEN, CONSOLE_CLOSE, CONSOLE_FLUSH */ +struct gh_vm_console_common_req { + gh_vmid_t vmid; + u16 reserved0; +} __packed; + +/* Call: CONSOLE_WRITE */ +struct gh_vm_console_write_req { + gh_vmid_t vmid; + u16 num_bytes; + u8 data[0]; +} __packed; + int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, void **resp_buf, size_t *resp_buff_size); diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c new file mode 100644 index 000000000000..be9317968537 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr: " fmt + +#include +#include +#include +#include + +#include "rsc_mgr.h" + +/** + * gh_rm_get_vmid: Retrieve VMID of this virtual machine + * @vmid: Filled with the VMID of this VM + */ +int gh_rm_get_vmid(gh_vmid_t *vmid) +{ + void *resp; + size_t resp_size; + int ret; + int payload = 0; + + ret = gh_rm_call(GH_RM_RPC_VM_GET_VMID, &payload, sizeof(payload), &resp, &resp_size); + if (ret) + return ret; + + if (resp_size != sizeof(*vmid)) + return -EIO; + *vmid = *(gh_vmid_t *)resp; + kfree(resp); + + return ret; +} + +/** + * gh_rm_console_open: Open a console with a VM + * @vmid: VMID of the other vmid whose console to open. If VMID is GH_VMID_SELF, the + * console associated with this VM is opened. + */ +int gh_rm_console_open(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_OPEN_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_OPEN: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_open); + +/** + * gh_rm_console_close: Close a console with a VM + * @vmid: The vmid of the vm whose console to close. + */ +int gh_rm_console_close(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_CLOSE_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_CLOSE: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_close); + +/** + * gh_rm_console_write: Write to a VM's console + * @vmid: The vmid of the vm whose console to write to. + * @buf: Buffer to write to the VM's console + * @size: Size of the buffer + */ +int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size) +{ + void *resp; + struct gh_vm_console_write_req *req_payload; + size_t resp_size; + int ret = 0; + size_t req_payload_size = sizeof(*req_payload) + size; + + if (size < 1 || size > (U32_MAX - sizeof(*req_payload))) + return -EINVAL; + + req_payload = kzalloc(req_payload_size, GFP_KERNEL); + + if (!req_payload) + return -ENOMEM; + + req_payload->vmid = vmid; + req_payload->num_bytes = size; + memcpy(req_payload->data, buf, size); + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_WRITE_ID, + req_payload, req_payload_size, + &resp, &resp_size); + kfree(req_payload); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_WRITE: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_write); + +/** + * gh_rm_console_flush: Flush a console with a VM + * @vmid: The vmid of the vm whose console to flush + */ +int gh_rm_console_flush(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_FLUSH_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_FLUSH: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_flush); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 015bd851e1a3..4211ac9a219c 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -26,4 +26,20 @@ struct gh_rm_notification { int gh_rm_register_notifier(struct notifier_block *nb); int gh_rm_unregister_notifier(struct notifier_block *nb); +/* Notification type Message IDs */ +#define GH_RM_NOTIF_VM_CONSOLE_CHARS 0x56100080 + +struct gh_rm_notif_vm_console_chars { + gh_vmid_t vmid; + u16 num_bytes; + u8 bytes[0]; +} __packed; + +/* RPC Calls */ +int gh_rm_get_vmid(gh_vmid_t *vmid); +int gh_rm_console_open(gh_vmid_t vmid); +int gh_rm_console_close(gh_vmid_t vmid); +int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size); +int gh_rm_console_flush(gh_vmid_t vmid); + #endif From patchwork Mon Aug 1 21:12:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 594738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F069EC25B0E for ; Mon, 1 Aug 2022 21:14:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234307AbiHAVOG (ORCPT ); Mon, 1 Aug 2022 17:14:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235103AbiHAVOE (ORCPT ); Mon, 1 Aug 2022 17:14:04 -0400 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 896C83F316; Mon, 1 Aug 2022 14:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388442; x=1690924442; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=apoLqHIFfY6O5tPy1Y3FUD9tFxQUIrkjkeX6NZD7iW0=; b=fix7eFEmC36R9fcCJMh3EHWCHM8TGWtj86ycuB3aFwlsyMjidwhfshH9 7ldcrha8av8Iz8h6R9PwrM+G9LGJcc/yHQdN/IrJw33hxaKtzEdZ3HO1I EUCHinlCc9/uUlsM59hA5+vxbXjEtt7NAnyInwarrPTZqanz01ddmvktv Q=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:02 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:02 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:14:01 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 11/11] gunyah: Add tty console driver for RM Console Serivces Date: Mon, 1 Aug 2022 14:12:40 -0700 Message-ID: <20220801211240.597859-12-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Gunyah provides a console for each VM using the VM console resource manager APIs. This driver allows console data from other VMs to be accessed via a TTY device and exports a console device to dump Linux's own logs to our console. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 7 + drivers/virt/gunyah/Kconfig | 10 + drivers/virt/gunyah/Makefile | 3 + drivers/virt/gunyah/rsc_mgr_console.c | 405 ++++++++++++++++++++++++++ 4 files changed, 425 insertions(+) create mode 100644 drivers/virt/gunyah/rsc_mgr_console.c diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst index e7bb2b14543e..95ba9b71ab30 100644 --- a/Documentation/virt/gunyah/index.rst +++ b/Documentation/virt/gunyah/index.rst @@ -90,3 +90,10 @@ When booting a virtual machine which uses a devicetree, resource manager overlay how to communicate with resource manager, and basic description and capabilities of this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description of this node. + +Resource Manager Consoles +------------------------- +RM provides infrastructure for virtual machines to share an interactive console. This can be used to +interact with a VM which may not have access to a serial port. Linux will register a printk console: +ttyGH0. That console and other VM's consoles can be accessed via ttyGHX. +/sys/class/tty/ttyGHX/vmid will print the VM which is associated with that TTY. diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 0adb6efd4848..ba7bd058e959 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -12,3 +12,13 @@ config GUNYAH Say Y here to enable the drivers needed to interact in a Gunyah virtual environment. + +if GUNYAH +config GUNYAH_RESOURCE_MANAGER_CONSOLE + tristate "Gunyah Resource Manager Consoles" + depends on TTY + help + This enables support for console output using Gunyah's Resource Manager RPC. + This is normally used when a secondary VM which does not have exclusive access + to a real serial device and virtio-console is unavailable. +endif diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b3f15c052297..001cf1630c03 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -3,3 +3,6 @@ gunyah-y += sysfs.o device.o msgq.o gunyah-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o + +gunyah_console-y += rsc_mgr_console.o +obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER_CONSOLE) += gunyah_console.o diff --git a/drivers/virt/gunyah/rsc_mgr_console.c b/drivers/virt/gunyah/rsc_mgr_console.c new file mode 100644 index 000000000000..eda25100314f --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_console.c @@ -0,0 +1,405 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr_console: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * The Linux TTY code does not support dynamic addition of tty derived devices so we need to know + * how many tty devices we might need when space is allocated for the tty device. Since VMs might be + * added/removed dynamically, we need to make sure we have enough allocated. + */ +#define RSC_MGR_TTY_ADAPTERS 16 + +/* # of payload bytes that can fit in a 1-fragment CONSOLE_WRITE message */ +#define RM_CONS_WRITE_MSG_SIZE ((1 * (GH_MSGQ_MAX_MSG_SIZE - 8)) - 4) + +struct rm_cons_port { + struct tty_port port; + gh_vmid_t vmid; + bool open; + unsigned int index; + + DECLARE_KFIFO(put_fifo, char, 1024); + spinlock_t fifo_lock; + struct work_struct put_work; + + struct rm_cons_data *cons_data; +}; + +struct rm_cons_data { + struct tty_driver *tty_driver; + struct device *dev; + + spinlock_t ports_lock; + struct rm_cons_port *ports[RSC_MGR_TTY_ADAPTERS]; + + struct notifier_block rsc_mgr_notif; + struct console console; +}; + +static void put_work_fn(struct work_struct *ws) +{ + char buf[RM_CONS_WRITE_MSG_SIZE]; + int count, ret; + struct rm_cons_port *port = container_of(ws, struct rm_cons_port, put_work); + + while (!kfifo_is_empty(&port->put_fifo)) { + count = kfifo_out_spinlocked(&port->put_fifo, buf, sizeof(buf), &port->fifo_lock); + if (count <= 0) + continue; + + ret = gh_rm_console_write(port->vmid, buf, count); + if (ret) { + pr_warn_once("failed to send characters: %d\n", ret); + break; + } + } +} + +static int rsc_mgr_console_notif(struct notifier_block *nb, unsigned long cmd, void *data) +{ + int count, i; + struct rm_cons_port *rm_port; + struct tty_port *tty_port = NULL; + struct rm_cons_data *cons_data = container_of(nb, struct rm_cons_data, rsc_mgr_notif); + const struct gh_rm_notification *notif = data; + struct gh_rm_notif_vm_console_chars const * const msg = notif->buff; + + if (cmd != GH_RM_NOTIF_VM_CONSOLE_CHARS || + notif->size < sizeof(*msg)) + return NOTIFY_DONE; + + spin_lock(&cons_data->ports_lock); + for (i = 0; i < RSC_MGR_TTY_ADAPTERS; i++) { + if (!cons_data->ports[i]) + continue; + if (cons_data->ports[i]->vmid == msg->vmid) { + rm_port = cons_data->ports[i]; + break; + } + } + if (rm_port) + tty_port = tty_port_get(&rm_port->port); + spin_unlock(&cons_data->ports_lock); + + if (!rm_port) + pr_warn("Received unexpected console characters for VMID %u\n", msg->vmid); + if (!tty_port) + return NOTIFY_DONE; + + count = tty_buffer_request_room(tty_port, msg->num_bytes); + tty_insert_flip_string(tty_port, msg->bytes, count); + tty_flip_buffer_push(tty_port); + + tty_port_put(tty_port); + return NOTIFY_OK; +} + +static ssize_t vmid_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(dev); + + if (rm_port->vmid == GH_VMID_SELF) + return sysfs_emit(buf, "self\n"); + + return sysfs_emit(buf, "%u\n", rm_port->vmid); +} + +static DEVICE_ATTR_RO(vmid); + +static struct attribute *rsc_mgr_tty_dev_attrs[] = { + &dev_attr_vmid.attr, + NULL +}; + +static const struct attribute_group rsc_mgr_tty_dev_attr_group = { + .attrs = rsc_mgr_tty_dev_attrs, +}; + +static const struct attribute_group *rsc_mgr_tty_dev_attr_groups[] = { + &rsc_mgr_tty_dev_attr_group, + NULL +}; + +static int rsc_mgr_tty_open(struct tty_struct *tty, struct file *filp) +{ + int ret; + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + if (!rm_port->open) { + ret = gh_rm_console_open(rm_port->vmid); + if (ret) { + pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = true; + } + + return tty_port_open(&rm_port->port, tty, filp); +} + +static void rsc_mgr_tty_close(struct tty_struct *tty, struct file *filp) +{ + int ret; + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + if (rm_port->open) { + if (rm_port->vmid != GH_VMID_SELF) { + ret = gh_rm_console_close(rm_port->vmid); + if (ret) + pr_warn("Failed to close RM console for vmid %d: %d\n", + rm_port->vmid, ret); + } + rm_port->open = false; + + tty_port_close(&rm_port->port, tty, filp); + } + +} + +static int rsc_mgr_tty_write(struct tty_struct *tty, const unsigned char *buf, int count) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + int ret; + + ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock); + if (ret > 0) + schedule_work(&rm_port->put_work); + + return ret; +} + +static unsigned int rsc_mgr_mgr_tty_write_room(struct tty_struct *tty) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + return kfifo_avail(&rm_port->put_fifo); +} + +static void rsc_mgr_console_write(struct console *co, const char *buf, unsigned count) +{ + struct rm_cons_port *rm_port = co->data; + int ret; + + ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock); + if (ret > 0) + schedule_work(&rm_port->put_work); +} + +static struct tty_driver *rsc_mgr_console_device(struct console *co, int *index) +{ + struct rm_cons_port *rm_port = co->data; + + *index = rm_port->index; + return rm_port->port.tty->driver; +} + +static int rsc_mgr_console_setup(struct console *co, char *unused) +{ + int ret; + struct rm_cons_port *rm_port = co->data; + + if (!rm_port->open) { + ret = gh_rm_console_open(rm_port->vmid); + if (ret) { + pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = true; + } + + return 0; +} + +static int rsc_mgr_console_exit(struct console *co) +{ + int ret; + struct rm_cons_port *rm_port = co->data; + + if (rm_port->open) { + ret = gh_rm_console_close(rm_port->vmid); + if (ret) { + pr_err("Failed to close RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = false; + } + + return 0; +} + +static const struct tty_operations rsc_mgr_tty_ops = { + .open = rsc_mgr_tty_open, + .close = rsc_mgr_tty_close, + .write = rsc_mgr_tty_write, + .write_room = rsc_mgr_mgr_tty_write_room, +}; + +static void rsc_mgr_port_destruct(struct tty_port *port) +{ + struct rm_cons_port *rm_port = container_of(port, struct rm_cons_port, port); + struct rm_cons_data *cons_data = rm_port->cons_data; + + spin_lock(&cons_data->ports_lock); + WARN_ON(cons_data->ports[rm_port->index] != rm_port); + cons_data->ports[rm_port->index] = NULL; + spin_unlock(&cons_data->ports_lock); + kfree(rm_port); +} + +static const struct tty_port_operations rsc_mgr_port_ops = { + .destruct = rsc_mgr_port_destruct, +}; + +static struct rm_cons_port *rsc_mgr_port_create(struct rm_cons_data *cons_data, gh_vmid_t vmid) +{ + struct rm_cons_port *rm_port; + struct device *ttydev; + unsigned int index; + int ret; + + rm_port = kzalloc(sizeof(*rm_port), GFP_KERNEL); + rm_port->vmid = vmid; + INIT_KFIFO(rm_port->put_fifo); + spin_lock_init(&rm_port->fifo_lock); + INIT_WORK(&rm_port->put_work, put_work_fn); + tty_port_init(&rm_port->port); + rm_port->port.ops = &rsc_mgr_port_ops; + + spin_lock(&cons_data->ports_lock); + for (index = 0; index < RSC_MGR_TTY_ADAPTERS; index++) { + if (!cons_data->ports[index]) { + cons_data->ports[index] = rm_port; + rm_port->index = index; + break; + } + } + spin_unlock(&cons_data->ports_lock); + if (index >= RSC_MGR_TTY_ADAPTERS) { + ret = -ENOSPC; + goto err_put_port; + } + + ttydev = tty_port_register_device_attr(&rm_port->port, cons_data->tty_driver, index, + cons_data->dev, rm_port, rsc_mgr_tty_dev_attr_groups); + if (IS_ERR(ttydev)) { + ret = PTR_ERR(ttydev); + goto err_put_port; + } + + return rm_port; +err_put_port: + tty_port_put(&rm_port->port); + return ERR_PTR(ret); +} + +static int rsc_mgr_console_probe(struct auxiliary_device *auxdev, + const struct auxiliary_device_id *aux_dev_id) +{ + struct rm_cons_data *cons_data; + struct rm_cons_port *rm_port; + int ret; + gh_vmid_t vmid; + + cons_data = devm_kzalloc(&auxdev->dev, sizeof(*cons_data), GFP_KERNEL); + if (!cons_data) + return -ENOMEM; + dev_set_drvdata(&auxdev->dev, cons_data); + cons_data->dev = &auxdev->dev; + + cons_data->tty_driver = tty_alloc_driver(RSC_MGR_TTY_ADAPTERS, + TTY_DRIVER_REAL_RAW | TTY_DRIVER_DYNAMIC_DEV); + if (IS_ERR(cons_data->tty_driver)) + return PTR_ERR(cons_data->tty_driver); + + cons_data->tty_driver->driver_name = "gh"; + cons_data->tty_driver->name = "ttyGH"; + cons_data->tty_driver->type = TTY_DRIVER_TYPE_SYSTEM; + cons_data->tty_driver->init_termios = tty_std_termios; + tty_set_operations(cons_data->tty_driver, &rsc_mgr_tty_ops); + + ret = tty_register_driver(cons_data->tty_driver); + if (ret) { + dev_err(&auxdev->dev, "Could not register tty driver: %d\n", ret); + goto err_put_tty; + } + + spin_lock_init(&cons_data->ports_lock); + + cons_data->rsc_mgr_notif.notifier_call = rsc_mgr_console_notif; + ret = gh_rm_register_notifier(&cons_data->rsc_mgr_notif); + if (ret) { + dev_err(&auxdev->dev, "Could not register for resource manager notifications: %d\n", + ret); + goto err_put_tty; + } + + rm_port = rsc_mgr_port_create(cons_data, GH_VMID_SELF); + if (IS_ERR(rm_port)) { + ret = PTR_ERR(rm_port); + dev_err(&auxdev->dev, "Could not create own console: %d\n", ret); + goto err_unreg_notif; + } + + strncpy(cons_data->console.name, "ttyGH", sizeof(cons_data->console.name)); + cons_data->console.write = rsc_mgr_console_write; + cons_data->console.device = rsc_mgr_console_device; + cons_data->console.setup = rsc_mgr_console_setup; + cons_data->console.exit = rsc_mgr_console_exit; + cons_data->console.index = rm_port->index; + cons_data->console.data = rm_port; + register_console(&cons_data->console); + + ret = gh_rm_get_vmid(&vmid); + if (!ret) + rsc_mgr_port_create(cons_data, vmid); + else + pr_warn("Failed to get this VM's VMID: %d. Not creating loop-back console\n", ret); + + return 0; +err_unreg_notif: + gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif); +err_put_tty: + tty_driver_kref_put(cons_data->tty_driver); + return ret; +} + +static void rsc_mgr_console_remove(struct auxiliary_device *auxdev) +{ + struct rm_cons_data *cons_data = dev_get_drvdata(&auxdev->dev); + + unregister_console(&cons_data->console); + gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif); + tty_driver_kref_put(cons_data->tty_driver); +} + +static struct auxiliary_device_id rsc_mgr_console_ids[] = { + { .name = "gunyah.console" }, + {} +}; +MODULE_DEVICE_TABLE(auxiliary, rsc_mgr_console_ids); + +static struct auxiliary_driver rsc_mgr_console_drv = { + .probe = rsc_mgr_console_probe, + .remove = rsc_mgr_console_remove, + .id_table = rsc_mgr_console_ids, +}; +module_auxiliary_driver(rsc_mgr_console_drv); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Console");