From patchwork Mon Dec 19 22:58:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DF76C46467 for ; Mon, 19 Dec 2022 23:00:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232762AbiLSXAX (ORCPT ); Mon, 19 Dec 2022 18:00:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232710AbiLSXAU (ORCPT ); Mon, 19 Dec 2022 18:00:20 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91B5014D2B; Mon, 19 Dec 2022 15:00:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1671490817; x=1703026817; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6IVtig+wbjMvPvMCEJfIk5SwsmOUgU0dSh0c/RdhBEA=; b=UuXFXoyhKGhtHkZ0vBLEBbJUV3xxbmd7yftqVABpDe+r9upaHougE8+/ /yG8BBHtTaZSCgor+PuTwwD8sJXClqr0NOjt8StAehEMxtlWBtXlF8zEI x9AWe3BrlbWXVhYNuU7wt+f+7TrbYWnYM7x8aN4tKqaanUIZmTYHWDCs3 4=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 19 Dec 2022 15:00:17 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2022 15:00:17 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:16 -0800 From: Elliot Berman To: Bjorn Andersson , Rob Herring , Krzysztof Kozlowski , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , "Jonathan Corbet" , Bagas Sanjaya , "Catalin Marinas" , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , "Dmitry Baryshkov" , , , , , , , Rob Herring Subject: [PATCH v8 02/28] dt-bindings: Add binding for gunyah hypervisor Date: Mon, 19 Dec 2022 14:58:23 -0800 Message-ID: <20221219225850.2397345-3-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org When Linux is booted as a guest under the Gunyah hypervisor, the Gunyah Resource Manager applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. This information is not otherwise discoverable by a VM: the Gunyah hypervisor core does not provide a direct interface to discover capability IDs nor a way to communicate with RM without having already known the corresponding message queue capability ID. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Reviewed-by: Rob Herring Signed-off-by: Elliot Berman --- .../bindings/firmware/gunyah-hypervisor.yaml | 82 +++++++++++++++++++ MAINTAINERS | 1 + 2 files changed, 83 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 000000000000..9fd1d254b156 --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Murali Nalajala + - Elliot Berman + +description: |+ + Gunyah virtual machines use this information to determine the capability IDs + of the message queues used to communicate with the Gunyah Resource Manager. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + const: gunyah-resource-manager + + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + + additionalProperties: false + + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "gunyah-hypervisor"; + + gunyah-resource-mgr@0 { + compatible = "gunyah-resource-manager"; + interrupts = , /* TX full IRQ */ + ; /* RX empty IRQ */ + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; + /* TX, RX cap ids */ + }; + }; diff --git a/MAINTAINERS b/MAINTAINERS index 1dd8f58d6e01..f32320a9efa4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8939,6 +8939,7 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Supported +F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ HABANALABS PCI DRIVER From patchwork Mon Dec 19 22:58:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C10DC3DA78 for ; Mon, 19 Dec 2022 23:02:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233038AbiLSXCh (ORCPT ); Mon, 19 Dec 2022 18:02:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232869AbiLSXBr (ORCPT ); Mon, 19 Dec 2022 18:01:47 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92922F53; Mon, 19 Dec 2022 15:00:42 -0800 (PST) Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJMja4n008569; Mon, 19 Dec 2022 23:00:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=c8eofiG3McaTS/A0hhC1mvLBYHAzHVOp0r2z6IdmbO4=; b=kD6/2gmESUJHuksg/gOZiOqZNnN+n/5rpOMZW+HeSoc/tMOuVHSJp7wT98dSRYhyxipe aApNXY5X+J+suFeuigsKyRcaETA3+7zncLj7r8wkKNxosoKqPca6Cqv2n1Txdn0/LecM 2ElOdWjDFf0u+k6A4XMZS4bkjPp0//G4TGb3tjCEwbhLPZV0LuPwZ/Gw5pbAGOmpA0vG dTKMhYuCD5TW9Mm+4urB3PiuEXGLaiWY0rJ/Vt87vzioD5DHa/8wgWvFAYF4UsghfwKm XAbXIJnV/LwLbaMOTu2KjAs1TsejjZQqiosgoLJnvLB8MxUEEYvs9hO3Z4T+1pgTIo7e eQ== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mjkwqsk6s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:19 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0IvM006053 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:18 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:17 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Bagas Sanjaya" , Catalin Marinas , Will Deacon , Jassi Brar , "Sudeep Holla" , Mark Rutland , "Lorenzo Pieralisi" , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 03/28] gunyah: Common types and error codes for Gunyah hypercalls Date: Mon, 19 Dec 2022 14:58:24 -0800 Message-ID: <20221219225850.2397345-4-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: zFReobXLi8YHM11CgAgUJ--sCZ1Y8vom X-Proofpoint-ORIG-GUID: zFReobXLi8YHM11CgAgUJ--sCZ1Y8vom X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxscore=0 priorityscore=1501 spamscore=0 mlxlogscore=863 lowpriorityscore=0 suspectscore=0 impostorscore=0 phishscore=0 bulkscore=0 adultscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Reviewed-by: Dmitry Baryshkov Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + include/linux/gunyah.h | 74 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/MAINTAINERS b/MAINTAINERS index f32320a9efa4..74e76e0ab14d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8941,6 +8941,7 @@ L: linux-arm-msm@vger.kernel.org S: Supported F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ +F: include/linux/gunyah.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..824e20a11d27 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_H +#define _GUNYAH_H + +#include +#include + +/* Common Gunyah macros */ +#define GH_CAPID_INVAL U64_MAX +#define GH_VMID_ROOT_VM 0xff + +#define GH_ERROR_OK 0 + +#define GH_ERROR_UNIMPLEMENTED -1 +#define GH_ERROR_RETRY -2 + +#define GH_ERROR_ARG_INVAL 1 +#define GH_ERROR_ARG_SIZE 2 +#define GH_ERROR_ARG_ALIGN 3 + +#define GH_ERROR_NOMEM 10 + +#define GH_ERROR_ADDR_OVFL 20 +#define GH_ERROR_ADDR_UNFL 21 +#define GH_ERROR_ADDR_INVAL 22 + +#define GH_ERROR_DENIED 30 +#define GH_ERROR_BUSY 31 +#define GH_ERROR_IDLE 32 + +#define GH_ERROR_IRQ_BOUND 40 +#define GH_ERROR_IRQ_UNBOUND 41 + +#define GH_ERROR_CSPACE_CAP_NULL 50 +#define GH_ERROR_CSPACE_CAP_REVOKED 51 +#define GH_ERROR_CSPACE_WRONG_OBJ_TYPE 52 +#define GH_ERROR_CSPACE_INSUF_RIGHTS 53 +#define GH_ERROR_CSPACE_FULL 54 + +#define GH_ERROR_MSGQUEUE_EMPTY 60 +#define GH_ERROR_MSGQUEUE_FULL 61 + +static inline int gh_remap_error(int gh_error) +{ + switch (gh_error) { + case GH_ERROR_OK: + return 0; + case GH_ERROR_NOMEM: + return -ENOMEM; + case GH_ERROR_DENIED: + case GH_ERROR_CSPACE_CAP_NULL: + case GH_ERROR_CSPACE_CAP_REVOKED: + case GH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GH_ERROR_CSPACE_INSUF_RIGHTS: + case GH_ERROR_CSPACE_FULL: + return -EACCES; + case GH_ERROR_BUSY: + case GH_ERROR_IDLE: + return -EBUSY; + case GH_ERROR_IRQ_BOUND: + case GH_ERROR_IRQ_UNBOUND: + case GH_ERROR_MSGQUEUE_FULL: + case GH_ERROR_MSGQUEUE_EMPTY: + return -EPERM; + default: + return -EINVAL; + } +} + +#endif From patchwork Mon Dec 19 22:58:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A18EC4332F for ; Mon, 19 Dec 2022 23:02:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233008AbiLSXCN (ORCPT ); Mon, 19 Dec 2022 18:02:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232586AbiLSXBm (ORCPT ); Mon, 19 Dec 2022 18:01:42 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E222137; Mon, 19 Dec 2022 15:00:39 -0800 (PST) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJMpfGl001530; Mon, 19 Dec 2022 23:00:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=cWnD/VtzmOTFNtRhKXIobLlELZFzKzUnSZ/Rd7g1umo=; b=PmlRtOCGlpLWwm0ZTkPt9XYYEDa4e/SujABuSFO1crU6SgAIA/PIVA+a8LizLzwVjpia Y31J7Gy09dZCGFi8BXlStLU39wm8lJQx8MklUEPqWuxTbOM1GqRPCTYYF7MoWBceTqfZ kiPJMrcqO8DJOHQxcy7AVNN8w3EnMQUVvJZSoyqLEa7cD3wFI2KrgR8nBl680VBhfVXv mjymx7dn/JLNY3Wd0rgIhkIgsvHK7y09/zZ9TuvpJRBGKAG9H7suk69aVJyElaMIYIMn H12TeAwwfOxEN8ZNc6du9Ybj1tGFxNq6hptjFP49FMHx+UXhMiHU5+Bb04phbzcNZ0ni gQ== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh6xtd4b2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:20 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0JPB006062 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:19 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:18 -0800 From: Elliot Berman To: Bjorn Andersson , Mark Rutland , Lorenzo Pieralisi , "Sudeep Holla" CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Bagas Sanjaya , Catalin Marinas , Will Deacon , Jassi Brar , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 04/28] arm64: smccc: Include alternative-macros.h Date: Mon, 19 Dec 2022 14:58:25 -0800 Message-ID: <20221219225850.2397345-5-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: tDToHUGeLaJBNriyK4tXCww7wVIkV_Oz X-Proofpoint-ORIG-GUID: tDToHUGeLaJBNriyK4tXCww7wVIkV_Oz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 adultscore=0 impostorscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 mlxlogscore=666 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Fix build error when CONFIG_ARM64_SVE is selected and asm/alternative-macros.h wasn't implicitly included by another header. In file included from arch/arm64/gunyah/gunyah_hypercall.c:6: arch/arm64/gunyah/gunyah_hypercall.c: In function `gh_hypercall_msgq_send': ./include/linux/arm-smccc.h:387:25: error: expected string literal before `ALTERNATIVE' 387 | #define SMCCC_SVE_CHECK ALTERNATIVE("nop \n", "bl __arm_smccc_sve_check \n", \ Signed-off-by: Elliot Berman --- include/linux/arm-smccc.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 220c8c60e021..6a627cdbbdec 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -383,6 +383,7 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, /* nVHE hypervisor doesn't have a current thread so needs separate checks */ #if defined(CONFIG_ARM64_SVE) && !defined(__KVM_NVHE_HYPERVISOR__) +#include #define SMCCC_SVE_CHECK ALTERNATIVE("nop \n", "bl __arm_smccc_sve_check \n", \ ARM64_SVE) From patchwork Mon Dec 19 22:58:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D49ECC001B2 for ; Mon, 19 Dec 2022 23:03:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233072AbiLSXDE (ORCPT ); Mon, 19 Dec 2022 18:03:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232958AbiLSXBv (ORCPT ); Mon, 19 Dec 2022 18:01:51 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2876B26E7; Mon, 19 Dec 2022 15:00:44 -0800 (PST) Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJMoPmT008515; Mon, 19 Dec 2022 23:00:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=chfYYeBv7lleRWiVsrlZotQk/mJeB3QcEkzrz6VKP5E=; b=NRCiBbX25ycaH+XTo6iWwiEFUVxMSQERb8aZrskRwnMdyuJ5mvLuZPTHV8j2DurTJwbM Sv7Wz6Ri1m2n08iKqYVD3MdHKRuMKf8ejp3St0Emg/m+33gL0wqkM7M5XlVzCDXcdR4m olXZ4nkLnJdGof7wtJYBZKnGKwWLGT9HkyM0biEkd8ICRbXBJFaXW29yCzXRU0XlskDD 2I0OdOx/Z/yUIf23LKU/6YvP5mNvNOxZiUKUlnQmYymavTxvvZNB5f1um+ftLD56/wvf A7q1GOgVeU26+Wub/PIKWgjpxMBqgYHHR+hLh4Kls1FG+VFlOjTW6VhpwXR68x9lUSKM qQ== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mjyk505x1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:21 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0Lol007581 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:21 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:20 -0800 From: Elliot Berman To: Bjorn Andersson , Catalin Marinas , Will Deacon , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Bagas Sanjaya" , Jassi Brar , "Sudeep Holla" , Mark Rutland , "Lorenzo Pieralisi" , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 05/28] virt: gunyah: Add hypercalls to identify Gunyah Date: Mon, 19 Dec 2022 14:58:26 -0800 Message-ID: <20221219225850.2397345-6-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: Q7boJ0Pv-qwYEsRE1bT1QkJQy6HPV3cI X-Proofpoint-ORIG-GUID: Q7boJ0Pv-qwYEsRE1bT1QkJQy6HPV3cI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 adultscore=0 clxscore=1015 suspectscore=0 impostorscore=0 bulkscore=0 priorityscore=1501 spamscore=0 lowpriorityscore=0 phishscore=0 mlxscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Add hypercalls to identify when Linux is running a virtual machine under Gunyah. There are two calls to help identify Gunyah: 1. gh_hypercall_get_uid() returns a UID when running under a Gunyah hypervisor. 2. gh_hypercall_hyp_identify() returns build information and a set of feature flags that are supported by Gunyah. Signed-off-by: Elliot Berman --- MAINTAINERS | 2 + arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 1 + arch/arm64/gunyah/gunyah_hypercall.c | 69 ++++++++++++++++++++++++++++ drivers/virt/Kconfig | 1 + drivers/virt/gunyah/Kconfig | 12 +++++ include/linux/gunyah.h | 25 ++++++++++ 7 files changed, 111 insertions(+) create mode 100644 arch/arm64/gunyah/Makefile create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c create mode 100644 drivers/virt/gunyah/Kconfig diff --git a/MAINTAINERS b/MAINTAINERS index 74e76e0ab14d..36698df6b0e5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8941,6 +8941,8 @@ L: linux-arm-msm@vger.kernel.org S: Supported F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ +F: arch/arm64/gunyah/ +F: drivers/virt/gunyah/ F: include/linux/gunyah.h HABANALABS PCI DRIVER diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild index 5bfbf7d79c99..e4847ba0e3c9 100644 --- a/arch/arm64/Kbuild +++ b/arch/arm64/Kbuild @@ -3,6 +3,7 @@ obj-y += kernel/ mm/ net/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_XEN) += xen/ obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_CRYPTO) += crypto/ # for cleaning diff --git a/arch/arm64/gunyah/Makefile b/arch/arm64/gunyah/Makefile new file mode 100644 index 000000000000..9fbc720b6fb6 --- /dev/null +++ b/arch/arm64/gunyah/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_GUNYAH) += gunyah_hypercall.o diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c new file mode 100644 index 000000000000..0beb3123d650 --- /dev/null +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include + +#define GH_CALL_TYPE_PLATFORM_CALL 0 +#define GH_CALL_TYPE_HYPERCALL 2 +#define GH_CALL_TYPE_SERVICE 3 +#define GH_CALL_TYPE_SHIFT 14 +#define GH_CALL_FUNCTION_NUM_MASK 0x3fff + +#define GH_FN_ID(type, num) ((type) << GH_CALL_TYPE_SHIFT | ((num) & GH_CALL_FUNCTION_NUM_MASK)) + +#define GH_SERVICE(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + GH_FN_ID(GH_CALL_TYPE_SERVICE, fn)) + +#define GH_HYPERCALL_CALL_UID GH_SERVICE(0x3f01) + +#define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + GH_FN_ID(GH_CALL_TYPE_HYPERCALL, fn)) + +#define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000) + +/** + * gh_hypercall_get_uid() - Returns a UID when running under a Gunyah hypervisor + * @uid: An array of 4 u32's (u32 uid[4];) + * + * Caller should compare the resulting UID to a list of known Gunyah UIDs to + * confirm that Linux is running as a guest of Gunyah. + */ +void gh_hypercall_get_uid(u32 uid[4]) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_CALL_UID, &res); + + uid[0] = res.a0; + uid[1] = res.a1; + uid[2] = res.a2; + uid[3] = res.a3; +} +EXPORT_SYMBOL_GPL(gh_hypercall_get_uid); + +/** + * gh_hypercall_hyp_identify() - Returns build information and feature flags + * supported by Gunyah. + * @hyp_identity: filled by the hypercall with the API info and feature flags. + */ +void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GH_HYPERCALL_HYP_IDENTIFY, &res); + + hyp_identity->api_info = res.a0; + hyp_identity->flags[0] = res.a1; + hyp_identity->flags[1] = res.a2; + hyp_identity->flags[2] = res.a3; +} +EXPORT_SYMBOL_GPL(gh_hypercall_hyp_identify); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index 87ef258cec64..259dc2be6cad 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -52,4 +52,5 @@ source "drivers/virt/coco/efi_secret/Kconfig" source "drivers/virt/coco/sev-guest/Kconfig" +source "drivers/virt/gunyah/Kconfig" endif diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..127156a678a6 --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + help + The Gunyah drivers are the helper interfaces that runs in a guest VM + such as basic inter-VM IPC and signaling mechanisms, and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y/M here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 824e20a11d27..2765d2b40198 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,6 +6,7 @@ #ifndef _GUNYAH_H #define _GUNYAH_H +#include #include #include @@ -71,4 +72,28 @@ static inline int gh_remap_error(int gh_error) } } +#define GUNYAH_API_V1 1 + +#define GH_API_INFO_API_VERSION_MASK GENMASK_ULL(13, 0) +#define GH_API_INFO_BIG_ENDIAN BIT_ULL(14) +#define GH_API_INFO_IS_64BIT BIT_ULL(15) +#define GH_API_INFO_VARIANT_MASK GENMASK_ULL(63, 56) + +#define GH_IDENTIFY_PARTITION_CSPACE BIT_ULL(0) +#define GH_IDENTIFY_DOORBELL BIT_ULL(1) +#define GH_IDENTIFY_MSGQUEUE BIT_ULL(2) +#define GH_IDENTIFY_VIC BIT_ULL(3) +#define GH_IDENTIFY_VPM BIT_ULL(4) +#define GH_IDENTIFY_VCPU BIT_ULL(5) +#define GH_IDENTIFY_MEMEXTENT BIT_ULL(6) +#define GH_IDENTIFY_TRACE_CTRL BIT_ULL(7) + +struct gh_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +void gh_hypercall_get_uid(u32 uid[4]); +void gh_hypercall_hyp_identify(struct gh_hypercall_hyp_identify_resp *hyp_identity); + #endif From patchwork Mon Dec 19 22:58:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A363C46467 for ; Mon, 19 Dec 2022 23:01:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232487AbiLSXBE (ORCPT ); Mon, 19 Dec 2022 18:01:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232824AbiLSXAm (ORCPT ); Mon, 19 Dec 2022 18:00:42 -0500 Received: from alexa-out-sd-01.qualcomm.com (alexa-out-sd-01.qualcomm.com [199.106.114.38]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F37C15712; Mon, 19 Dec 2022 15:00:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1671490828; x=1703026828; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=c+vzCFisspmz3J0/NAch3KtfI1rYZdZMcwkYlP9jGj0=; b=DIM+YlFkHP3Uxb5/n3IsZIUpVaxT9ulxTRV2gxuH385h492ShtKeyD24 S4whx0JJxDR9n1CxQQPB/eMEziO8KakjHDm5glM39ltTXmIqjErzFUvHz 82ozt8/PJxrJnT3+E+fNQnn5LHrl1spWC+/nfifsfhziOkAVbLr4pTq6A o=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-01.qualcomm.com with ESMTP; 19 Dec 2022 15:00:28 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2022 15:00:28 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:26 -0800 From: Elliot Berman To: Bjorn Andersson , Jassi Brar , Elliot Berman , "Murali Nalajala" , Jonathan Corbet CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Bagas Sanjaya , Catalin Marinas , Will Deacon , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 09/28] mailbox: Add Gunyah message queue mailbox Date: Mon, 19 Dec 2022 14:58:30 -0800 Message-ID: <20221219225850.2397345-10-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Gunyah message queues are a unidirectional inter-VM pipe for messages up to 1024 bytes. This driver supports pairing a receiver message queue and a transmitter message queue to expose a single mailbox channel. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/message-queue.rst | 8 + MAINTAINERS | 1 + drivers/mailbox/Kconfig | 10 + drivers/mailbox/Makefile | 2 + drivers/mailbox/gunyah-msgq.c | 229 ++++++++++++++++++++ include/linux/gunyah.h | 61 +++++- 6 files changed, 309 insertions(+), 2 deletions(-) create mode 100644 drivers/mailbox/gunyah-msgq.c diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst index be4ab289236a..9245dbe6dd8a 100644 --- a/Documentation/virt/gunyah/message-queue.rst +++ b/Documentation/virt/gunyah/message-queue.rst @@ -54,3 +54,11 @@ vIRQ: two TX message queues will have two vIRQs (and two capability IDs). | | | | | | | | | | | | +---------------+ +-----------------+ +---------------+ + +Gunyah message queues are exposed as mailboxes. To create the mailbox, create +a mbox_client and call `gh_msgq_init`. On receipt of the RX_READY interrupt, +all messages in the RX message queue are read and pushed via the `rx_callback` +of the registered mbox_client. + +.. kernel-doc:: drivers/mailbox/gunyah-msgq.c + :identifiers: gh_msgq_init diff --git a/MAINTAINERS b/MAINTAINERS index 36698df6b0e5..667480bfd387 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8942,6 +8942,7 @@ S: Supported F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/gunyah/ +F: drivers/mailbox/gunyah-msgq.c F: drivers/virt/gunyah/ F: include/linux/gunyah.h diff --git a/drivers/mailbox/Kconfig b/drivers/mailbox/Kconfig index 05d6fae800e3..baf9451c5f04 100644 --- a/drivers/mailbox/Kconfig +++ b/drivers/mailbox/Kconfig @@ -41,6 +41,16 @@ config IMX_MBOX help Mailbox implementation for i.MX Messaging Unit (MU). +config GUNYAH_MESSAGE_QUEUES + tristate "Gunyah Message Queue Mailbox" + depends on GUNYAH + help + Mailbox implementation for Gunyah Message Queues. Gunyah message queues + are an IPC mechanism to pass short messages between virtual machines + running under the Gunyah hypervisor. + + Say Y here if you run Linux as a Gunyah virtual machine. + config PLATFORM_MHU tristate "Platform MHU Mailbox" depends on OF diff --git a/drivers/mailbox/Makefile b/drivers/mailbox/Makefile index fc9376117111..5f929bb55e9a 100644 --- a/drivers/mailbox/Makefile +++ b/drivers/mailbox/Makefile @@ -55,6 +55,8 @@ obj-$(CONFIG_MTK_CMDQ_MBOX) += mtk-cmdq-mailbox.o obj-$(CONFIG_ZYNQMP_IPI_MBOX) += zynqmp-ipi-mailbox.o +obj-$(CONFIG_GUNYAH) += gunyah-msgq.o + obj-$(CONFIG_SUN6I_MSGBOX) += sun6i-msgbox.o obj-$(CONFIG_SPRD_MBOX) += sprd-mailbox.o diff --git a/drivers/mailbox/gunyah-msgq.c b/drivers/mailbox/gunyah-msgq.c new file mode 100644 index 000000000000..d5a324e85428 --- /dev/null +++ b/drivers/mailbox/gunyah-msgq.c @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#define mbox_chan_to_msgq(chan) (container_of(chan->mbox, struct gh_msgq, mbox)) + +static inline bool gh_msgq_has_tx(struct gh_msgq *msgq) +{ + return msgq->tx_ghrsc->type == GUNYAH_RESOURCE_TYPE_MSGQ_TX; +} + +static inline bool gh_msgq_has_rx(struct gh_msgq *msgq) +{ + return msgq->rx_ghrsc->type == GUNYAH_RESOURCE_TYPE_MSGQ_RX; +} + +static irqreturn_t gh_msgq_rx_irq_handler(int irq, void *data) +{ + struct gh_msgq *msgq = data; + struct gh_msgq_rx_data rx_data; + unsigned long gh_err; + ssize_t ret; + bool ready = false; + + do { + gh_err = gh_hypercall_msgq_recv(msgq->rx_ghrsc->capid, + (uintptr_t)&rx_data.data, sizeof(rx_data.data), + &rx_data.length, &ready); + if (gh_err == GH_ERROR_OK) { + mbox_chan_received_data(gh_msgq_chan(msgq), &rx_data); + } else if (GH_ERROR_MSGQUEUE_EMPTY) { + break; + } else { + pr_warn("Failed to receive data from msgq for %s: %ld\n", + msgq->mbox.dev ? dev_name(msgq->mbox.dev) : "", ret); + break; + } + } while (ready); + + return IRQ_HANDLED; +} + +static irqreturn_t gh_msgq_tx_irq_handler(int irq, void *data) +{ + struct gh_msgq *msgq = data; + + mbox_chan_txdone(gh_msgq_chan(msgq), 0); + + return IRQ_HANDLED; +} + +static void gh_msgq_txdone_tasklet(struct tasklet_struct *tasklet) +{ + struct gh_msgq *msgq = container_of(tasklet, struct gh_msgq, txdone_tasklet); + + mbox_chan_txdone(gh_msgq_chan(msgq), msgq->last_status); +} + +static int gh_msgq_send_data(struct mbox_chan *chan, void *data) +{ + struct gh_msgq *msgq = mbox_chan_to_msgq(chan); + struct gh_msgq_tx_data *msgq_data = data; + u64 tx_flags = 0; + unsigned long ret; + bool ready; + + if (msgq_data->push) + tx_flags |= GH_HYPERCALL_MSGQ_TX_FLAGS_PUSH; + + ret = gh_hypercall_msgq_send(msgq->tx_ghrsc->capid, msgq_data->length, + (uintptr_t)msgq_data->data, tx_flags, &ready); + + /** + * unlikely because Linux tracks state of msgq and should not try to + * send message when msgq is full. + */ + if (unlikely(ret == GH_ERROR_MSGQUEUE_FULL)) + return -EAGAIN; + + /** + * Propagate all other errors to client. If we return error to mailbox + * framework, then no other messages can be sent and nobody will know + * to retry this message. + */ + msgq->last_status = gh_remap_error(ret); + + /** + * This message was successfully sent, but message queue isn't ready to + * receive more messages because it's now full. Mailbox framework + * requires that we only report that message was transmitted only when + * we're ready to transmit another message. We'll get that in the form + * of tx IRQ once the other side starts to drain the msgq. + */ + if (ret == GH_ERROR_OK && !ready) + return 0; + + /** + * We can send more messages. Mailbox framework requires that tx done + * happens asynchronously to sending the message. Gunyah message queues + * tell us right away on the hypercall return whether we can send more + * messages. To work around this, defer the txdone to a tasklet. + */ + tasklet_schedule(&msgq->txdone_tasklet); + + return 0; +} + +struct mbox_chan_ops gh_msgq_ops = { + .send_data = gh_msgq_send_data, +}; + +/** + * gh_msgq_init() - Initialize a Gunyah message queue with an mbox_client + * @parent: optional, device parent used for the mailbox controller + * @msgq: Pointer to the gh_msgq to initialize + * @cl: A mailbox client to bind to the mailbox channel that the message queue creates + * @tx_ghrsc: optional, the transmission side of the message queue + * @rx_ghrsc: optional, the receiving side of the message queue + * + * At least one of tx_ghrsc and rx_ghrsc should be not NULL. Most message queue use cases come with + * a pair of message queues to facilitiate bidirectional communication. When tx_ghrsc is set, + * the client can send messages with mbox_send_message(gh_msgq_chan(msgq), msg). When rx_ghrsc + * is set, the mbox_client should register an .rx_callback() and the message queue driver will + * push all available messages upon receiving the RX ready interrupt. The messages should be + * consumed or copied by the client right away as the gh_msgq_rx_data will be replaced/destroyed + * after the callback. + * + * Returns - 0 on success, negative otherwise + */ +int gh_msgq_init(struct device *parent, struct gh_msgq *msgq, struct mbox_client *cl, + struct gunyah_resource *tx_ghrsc, struct gunyah_resource *rx_ghrsc) +{ + int ret; + + /* Must have at least a tx_ghrsc or rx_ghrsc and that they are the right device types */ + if ((!tx_ghrsc && !rx_ghrsc) || + (tx_ghrsc && tx_ghrsc->type != GUNYAH_RESOURCE_TYPE_MSGQ_TX) || + (rx_ghrsc && rx_ghrsc->type != GUNYAH_RESOURCE_TYPE_MSGQ_RX)) + return -EINVAL; + + msgq->tx_ghrsc = tx_ghrsc; + msgq->rx_ghrsc = rx_ghrsc; + + msgq->mbox.dev = parent; + msgq->mbox.ops = &gh_msgq_ops; + msgq->mbox.num_chans = 1; + msgq->mbox.chans = kcalloc(msgq->mbox.num_chans, sizeof(*msgq->mbox.chans), GFP_KERNEL); + if (!msgq->mbox.chans) + return -ENOMEM; + msgq->mbox.txdone_irq = true; + + if (gh_msgq_has_tx(msgq)) { + ret = request_irq(msgq->tx_ghrsc->irq, gh_msgq_tx_irq_handler, 0, "gh_msgq_tx", + msgq); + if (ret) + goto err_chans; + } + + if (gh_msgq_has_rx(msgq)) { + ret = request_threaded_irq(msgq->rx_ghrsc->irq, NULL, gh_msgq_rx_irq_handler, + IRQF_ONESHOT, "gh_msgq_rx", msgq); + if (ret) + goto err_tx_irq; + } + + tasklet_setup(&msgq->txdone_tasklet, gh_msgq_txdone_tasklet); + + ret = mbox_controller_register(&msgq->mbox); + if (ret) + goto err_rx_irq; + + ret = mbox_bind_client(gh_msgq_chan(msgq), cl); + if (ret) + goto err_mbox; + + return 0; +err_mbox: + mbox_controller_unregister(&msgq->mbox); +err_rx_irq: + if (gh_msgq_has_rx(msgq)) + free_irq(msgq->rx_ghrsc->irq, msgq); +err_tx_irq: + if (gh_msgq_has_tx(msgq)) + free_irq(msgq->tx_ghrsc->irq, msgq); +err_chans: + kfree(msgq->mbox.chans); + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_init); + +void gh_msgq_remove(struct gh_msgq *msgq) +{ + mbox_controller_unregister(&msgq->mbox); + + if (gh_msgq_has_rx(msgq)) + free_irq(msgq->rx_ghrsc->irq, msgq); + + if (gh_msgq_has_tx(msgq)) + free_irq(msgq->tx_ghrsc->irq, msgq); + + kfree(msgq->mbox.chans); +} +EXPORT_SYMBOL_GPL(gh_msgq_remove); + + +static int __init gh_msgq_init_module(void) +{ + if (gh_api_version() != GUNYAH_API_V1) { + pr_warn("Unrecognized gunyah version: %u. Currently supported: %d\n", + gh_api_version(), GUNYAH_API_V1); + return -ENODEV; + } + + return 0; +} +module_init(gh_msgq_init_module); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Message Queue Driver"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index c863cac4a3cf..e317d7ac938f 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -7,10 +7,67 @@ #define _GUNYAH_H #include -#include #include +#include +#include +#include +#include + +/* Follows resource manager's resource types for VM_GET_HYP_RESOURCES */ +enum gunyah_resource_type { + GUNYAH_RESOURCE_TYPE_BELL_TX = 0, + GUNYAH_RESOURCE_TYPE_BELL_RX = 1, + GUNYAH_RESOURCE_TYPE_MSGQ_TX = 2, + GUNYAH_RESOURCE_TYPE_MSGQ_RX = 3, + GUNYAH_RESOURCE_TYPE_VCPU = 4, +}; + +struct gunyah_resource { + enum gunyah_resource_type type; + u64 capid; + int irq; +}; + +/** + * Gunyah Message Queues + */ + +#define GH_MSGQ_MAX_MSG_SIZE 240 + +struct gh_msgq_tx_data { + size_t length; + bool push; + char data[]; +}; + +struct gh_msgq_rx_data { + size_t length; + char data[GH_MSGQ_MAX_MSG_SIZE]; +}; + +struct gh_msgq { + struct gunyah_resource *tx_ghrsc; + struct gunyah_resource *rx_ghrsc; + + /* msgq private */ + int last_status; + struct mbox_controller mbox; + struct tasklet_struct txdone_tasklet; +}; + + +int gh_msgq_init(struct device *parent, struct gh_msgq *msgq, struct mbox_client *cl, + struct gunyah_resource *tx_ghrsc, struct gunyah_resource *rx_ghrsc); +void gh_msgq_remove(struct gh_msgq *msgq); + +static inline struct mbox_chan *gh_msgq_chan(struct gh_msgq *msgq) +{ + return &msgq->mbox.chans[0]; +} + +/******************************************************************************/ +/* Common arch-independent macros and definitions for Gunyah hypercalls */ -/* Common Gunyah macros */ #define GH_CAPID_INVAL U64_MAX #define GH_VMID_ROOT_VM 0xff From patchwork Mon Dec 19 22:58:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F21CC3DA7B for ; Mon, 19 Dec 2022 23:03:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233117AbiLSXDy (ORCPT ); Mon, 19 Dec 2022 18:03:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232967AbiLSXCK (ORCPT ); Mon, 19 Dec 2022 18:02:10 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D0C51274F; Mon, 19 Dec 2022 15:00:54 -0800 (PST) Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJMpfGm001530; Mon, 19 Dec 2022 23:00:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=aKAupqStqr6PCuMQ/ko/uzEbZ6rKcoMmTUfe7DRB6HA=; b=ASPcmUHD7sHj/1M0rVoMjJ2SaQ6hdCO+xoVg46n97nbZJjZjULL+SB1CMul++q8KhBdT P6NQl3FyGGt5WtHauaW+b+WIAjyd1lvb/2Wf1WjdRPS+Ua0pxhFWrX43FsctBScLuIWW WRwepOQIk1s5vwyPbntJ1KoVxyqf4HrEcu3RDZIjOk2tziJeEeU0KMhHecmcwJsBidrN In6PYomuCJScuuKJRx6HjzmwVDDNlfL6Rc1aRlumlDb0xG4Ggyf0X9Ny4pAjbyNGmOgj D3ZnM/c8EzhsXqXGPEldC5PONYEgYAjn+Tovxt/Ws6+HZUnhsxEfwntNgxdZeRFBYK7+ ow== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh6xtd4bk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:33 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0Wl9007757 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:32 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:31 -0800 From: Elliot Berman To: Bjorn Andersson , Jonathan Corbet , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Bagas Sanjaya , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 12/28] gunyah: vm_mgr: Introduce basic VM Manager Date: Mon, 19 Dec 2022 14:58:33 -0800 Message-ID: <20221219225850.2397345-13-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: aLW2YXmq8yY539Rz7zU0SfLwhZ3b28DE X-Proofpoint-ORIG-GUID: aLW2YXmq8yY539Rz7zU0SfLwhZ3b28DE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 adultscore=0 impostorscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Gunyah VM manager is a kernel moduel which exposes an interface to Gunyah userspace to load, run, and interact with other Gunyah virtual machines. The interface is a character device at /dev/gunyah. Add a basic VM manager driver. Upcoming patches will add more ioctls into this driver. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- .../userspace-api/ioctl/ioctl-number.rst | 1 + drivers/virt/gunyah/Kconfig | 11 ++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/rsc_mgr.c | 47 ++++++- drivers/virt/gunyah/vm_mgr.c | 115 ++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 25 ++++ include/linux/gunyah_rsc_mgr.h | 3 + include/uapi/linux/gunyah.h | 23 ++++ 8 files changed, 225 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/vm_mgr.c create mode 100644 drivers/virt/gunyah/vm_mgr.h create mode 100644 include/uapi/linux/gunyah.h diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 5f81e2a24a5c..81456c34c85d 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -136,6 +136,7 @@ Code Seq# Include File Comments 'F' DD video/sstfb.h conflict! 'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! 'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict! +'G' 00-0f linux/gunyah.h conflict! 'H' 00-7F linux/hiddev.h conflict! 'H' 00-0F linux/hidraw.h conflict! 'H' 01 linux/mei.h conflict! diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 4a6189dedab2..5e5a329c9866 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -23,4 +23,15 @@ config GUNYAH_RESOURCE_MANAGER Gunyah wanting to have Gunyah-awareness. Say Y/M here if unsure. + +config GUNYAH_VM_MANAGER + bool "Manage Gunyah Virtual Machines" + depends on MAILBOX + depends on GUNYAH_RESOURCE_MANAGER + help + Gunyah VM manager exposes an interface to Gunyah userspace to load, + run, and interact with other Gunyah virtual machines. This option is + required to launch other virtual machines. + + Say Y here if unsure and you want to support Gunyah VMMs. endif diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 5caa05267a58..e817ae6deddc 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,5 @@ obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o +gunyah_rsc_mgr-$(CONFIG_GUNYAH_VM_MANAGER) += vm_mgr.o obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER) += gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index 6b281576f5af..49d7939bbe36 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -16,8 +16,10 @@ #include #include #include +#include #include "rsc_mgr.h" +#include "vm_mgr.h" /* Resource Manager Header */ struct gh_rm_rpc_hdr { @@ -96,6 +98,7 @@ struct gh_rm_rpc { struct mutex send_lock; + struct miscdevice miscdev; struct work_struct recv_work; }; @@ -496,6 +499,33 @@ int gh_rm_call(struct gh_rm_rpc *rsc_mgr, u32 message_id, void *req_buff, size_t return ret; } +void get_gh_rm(struct gh_rm_rpc *rm) +{ + get_device(rm->dev); +} +EXPORT_SYMBOL_GPL(get_gh_rm); + +void put_gh_rm(struct gh_rm_rpc *rm) +{ + put_device(rm->dev); +} +EXPORT_SYMBOL_GPL(put_gh_rm); + +static long gh_dev_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) +{ + struct miscdevice *miscdev = filp->private_data; + struct gh_rm_rpc *rm = container_of(miscdev, struct gh_rm_rpc, miscdev); + + return gh_dev_vm_mgr_ioctl(rm, cmd, arg); +} + +static const struct file_operations gh_dev_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = gh_dev_ioctl, + .compat_ioctl = compat_ptr_ioctl, + .llseek = noop_llseek, +}; + static int gh_msgq_platform_probe_direction(struct platform_device *pdev, u8 gh_type, int idx, struct gunyah_resource *ghrsc) { @@ -550,14 +580,29 @@ static int gh_rm_drv_probe(struct platform_device *pdev) rsc_mgr->msgq_client.rx_callback = gh_rm_msgq_rx_data; rsc_mgr->msgq_client.tx_done = gh_rm_msgq_tx_done; - return gh_msgq_init(&pdev->dev, &rsc_mgr->msgq, &rsc_mgr->msgq_client, + ret = gh_msgq_init(&pdev->dev, &rsc_mgr->msgq, &rsc_mgr->msgq_client, &rsc_mgr->tx_ghrsc, &rsc_mgr->rx_ghrsc); + + rsc_mgr->miscdev.name = "gunyah"; + rsc_mgr->miscdev.minor = MISC_DYNAMIC_MINOR; + rsc_mgr->miscdev.fops = &gh_dev_fops; + + ret = misc_register(&rsc_mgr->miscdev); + if (ret) + goto err_msgq; + + return 0; +err_msgq: + mbox_free_channel(gh_msgq_chan(&rsc_mgr->msgq)); + gh_msgq_remove(&rsc_mgr->msgq); + return ret; } static int gh_rm_drv_remove(struct platform_device *pdev) { struct gh_rm_rpc *rm = platform_get_drvdata(pdev); + misc_deregister(&rm->miscdev); mbox_free_channel(gh_msgq_chan(&rm->msgq)); gh_msgq_remove(&rm->msgq); diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c new file mode 100644 index 000000000000..9c796e5edcf1 --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_vm_mgr: " fmt + +#include +#include +#include +#include +#include + +#include + +#include "vm_mgr.h" + +static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_rm_rpc *rm) +{ + struct gunyah_vm *ghvm; + int vmid; + + vmid = gh_rm_alloc_vmid(rm, 0); + if (vmid < 0) + return ERR_PTR(vmid); + + ghvm = kzalloc(sizeof(*ghvm), GFP_KERNEL); + if (!ghvm) + return ghvm; + + get_gh_rm(rm); + + ghvm->vmid = vmid; + ghvm->rm = rm; + + return ghvm; +} + +static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) +{ + long r; + + switch (cmd) { + default: + r = -ENOTTY; + break; + } + + return r; +} + +static int gh_vm_release(struct inode *inode, struct file *filp) +{ + struct gunyah_vm *ghvm = filp->private_data; + + put_gh_rm(ghvm->rm); + kfree(ghvm); + return 0; +} + +static const struct file_operations gh_vm_fops = { + .unlocked_ioctl = gh_vm_ioctl, + .release = gh_vm_release, + .compat_ioctl = compat_ptr_ioctl, + .llseek = noop_llseek, +}; + +static long gh_dev_ioctl_create_vm(struct gh_rm_rpc *rm, unsigned long arg) +{ + struct gunyah_vm *ghvm; + struct file *file; + int fd, err; + + /* arg reserved for future use. */ + if (arg) + return -EINVAL; + + ghvm = gunyah_vm_alloc(rm); + if (IS_ERR_OR_NULL(ghvm)) + return PTR_ERR(ghvm) ? : -ENOMEM; + + fd = get_unused_fd_flags(O_CLOEXEC); + if (fd < 0) { + err = fd; + goto err_destroy_vm; + } + + file = anon_inode_getfile("gunyah-vm", &gh_vm_fops, ghvm, O_RDWR); + if (IS_ERR(file)) { + err = PTR_ERR(file); + goto err_put_fd; + } + + fd_install(fd, file); + + return fd; + +err_put_fd: + put_unused_fd(fd); +err_destroy_vm: + kfree(ghvm); + return err; +} + +long gh_dev_vm_mgr_ioctl(struct gh_rm_rpc *rm, unsigned int cmd, unsigned long arg) +{ + switch (cmd) { + case GH_CREATE_VM: + return gh_dev_ioctl_create_vm(rm, arg); + default: + return -ENOIOCTLCMD; + } +} +EXPORT_SYMBOL_GPL(gh_dev_vm_mgr_ioctl); + diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h new file mode 100644 index 000000000000..84e3d1a8571d --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GH_PRIV_VM_MGR_H +#define _GH_PRIV_VM_MGR_H + +#include + +#include + +#if IS_ENABLED(CONFIG_GUNYAH_VM_MANAGER) +long gh_dev_vm_mgr_ioctl(struct gh_rm_rpc *rm, unsigned int cmd, unsigned long arg); +#else +static inline long gh_dev_vm_mgr_ioctl(struct gh_rm_rpc *rm, unsigned int cmd, unsigned long arg) + { return -ENOIOCTLCMD; } +#endif + +struct gunyah_vm { + u16 vmid; + struct gh_rm_rpc *rm; +}; + +#endif diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index eb94b48c41de..9c566436687d 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -17,6 +17,9 @@ struct gh_rm_rpc; +void get_gh_rm(struct gh_rm_rpc *rm); +void put_gh_rm(struct gh_rm_rpc *rm); + enum gh_rm_vm_status { GH_RM_VM_STATUS_NO_STATE = 0, GH_RM_VM_STATUS_INIT = 1, diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h new file mode 100644 index 000000000000..88a40d6e0b96 --- /dev/null +++ b/include/uapi/linux/gunyah.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _UAPI_LINUX_GUNYAH +#define _UAPI_LINUX_GUNYAH + +/* + * Userspace interface for /dev/gunyah - gunyah based virtual machine + */ + +#include +#include + +#define GH_IOCTL_TYPE 'G' + +/* + * ioctls for /dev/gunyah fds: + */ +#define GH_CREATE_VM _IO(GH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ + +#endif From patchwork Mon Dec 19 22:58:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DD9BC4332F for ; Mon, 19 Dec 2022 23:02:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233016AbiLSXB7 (ORCPT ); Mon, 19 Dec 2022 18:01:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232787AbiLSXBF (ORCPT ); Mon, 19 Dec 2022 18:01:05 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D68B415A3B; Mon, 19 Dec 2022 15:00:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1671490835; x=1703026835; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jpU+o+x1FMQZ4pcvTcO2IQ2V2sURu0rFXdBI85aXJoQ=; b=sgqhBMHIit9YXzjv39rQPIzx85hYBV9qH19E7d1w6GPX54DOAApavnhC epvRlFuMnWqU4ipq58Bt2ho/TChQrfCNI/DKHMnpxVd4+n3ZofWchq6Kx 8KmKGhcqxsqg0S4GQZH/qEf1Iwky7VTTy1xp6/qacBDo5Mjj0k1Q8kuTz Y=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-02.qualcomm.com with ESMTP; 19 Dec 2022 15:00:35 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2022 15:00:35 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:34 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Bagas Sanjaya" , Catalin Marinas , Will Deacon , Jassi Brar , "Sudeep Holla" , Mark Rutland , "Lorenzo Pieralisi" , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 14/28] gunyah: vm_mgr: Add/remove user memory regions Date: Mon, 19 Dec 2022 14:58:35 -0800 Message-ID: <20221219225850.2397345-15-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org When launching a virtual machine, Gunyah userspace allocates memory for the guest and informs Gunyah about these memory regions through SET_USER_MEMORY_REGION ioctl. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/vm_mgr.c | 45 +++++++ drivers/virt/gunyah/vm_mgr.h | 27 ++++ drivers/virt/gunyah/vm_mgr_mm.c | 221 ++++++++++++++++++++++++++++++++ include/uapi/linux/gunyah.h | 22 ++++ 5 files changed, 316 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/vm_mgr_mm.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index e817ae6deddc..9d6f900ab7b0 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o -gunyah_rsc_mgr-$(CONFIG_GUNYAH_VM_MANAGER) += vm_mgr.o +gunyah_rsc_mgr-$(CONFIG_GUNYAH_VM_MANAGER) += vm_mgr.o vm_mgr_mm.o obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER) += gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 9c796e5edcf1..a7220ad41669 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -33,14 +33,55 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_rm_rpc *rm) ghvm->vmid = vmid; ghvm->rm = rm; + mutex_init(&ghvm->mm_lock); + INIT_LIST_HEAD(&ghvm->memory_mappings); + return ghvm; } static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { + struct gunyah_vm *ghvm = filp->private_data; + void __user *argp = (void __user *)arg; long r; switch (cmd) { + case GH_VM_SET_USER_MEM_REGION: { + struct gunyah_vm_memory_mapping *mapping; + struct gh_userspace_memory_region region; + + r = -EFAULT; + if (copy_from_user(®ion, argp, sizeof(region))) + break; + + r = -EINVAL; + /* All other flag bits are reserved for future use */ + if (region.flags & ~(GH_MEM_ALLOW_READ | GH_MEM_ALLOW_WRITE | GH_MEM_ALLOW_EXEC | + GH_MEM_LENT)) + break; + + + if (region.memory_size) { + r = 0; + mapping = gh_vm_mem_mapping_alloc(ghvm, ®ion); + if (IS_ERR(mapping)) { + r = PTR_ERR(mapping); + break; + } + } else { + mapping = gh_vm_mem_mapping_find(ghvm, region.label); + if (IS_ERR(mapping)) { + r = PTR_ERR(mapping); + break; + } + r = 0; + if (!mapping) + break; + gh_vm_mem_mapping_reclaim(ghvm, mapping); + kfree(mapping); + } + break; + } default: r = -ENOTTY; break; @@ -52,7 +93,11 @@ static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) static int gh_vm_release(struct inode *inode, struct file *filp) { struct gunyah_vm *ghvm = filp->private_data; + struct gunyah_vm_memory_mapping *mapping, *tmp; + list_for_each_entry_safe(mapping, tmp, &ghvm->memory_mappings, list) { + gh_vm_mem_mapping_reclaim(ghvm, mapping); + } put_gh_rm(ghvm->rm); kfree(ghvm); return 0; diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 84e3d1a8571d..955856c1310b 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -7,6 +7,9 @@ #define _GH_PRIV_VM_MGR_H #include +#include +#include +#include #include @@ -17,9 +20,33 @@ static inline long gh_dev_vm_mgr_ioctl(struct gh_rm_rpc *rm, unsigned int cmd, u { return -ENOIOCTLCMD; } #endif +enum gunyah_vm_mem_share_type { + VM_MEM_SHARE, + VM_MEM_LEND, +}; + +struct gunyah_vm_memory_mapping { + struct list_head list; + enum gunyah_vm_mem_share_type share_type; + struct gh_rm_mem_parcel parcel; + + __u64 guest_phys_addr; + __u32 mem_size; + struct page **pages; + unsigned long npages; +}; + struct gunyah_vm { u16 vmid; struct gh_rm_rpc *rm; + + struct mutex mm_lock; + struct list_head memory_mappings; }; +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_alloc(struct gunyah_vm *ghvm, + struct gh_userspace_memory_region *region); +void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_mapping *mapping); +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label); + #endif diff --git a/drivers/virt/gunyah/vm_mgr_mm.c b/drivers/virt/gunyah/vm_mgr_mm.c new file mode 100644 index 000000000000..800d35da14ba --- /dev/null +++ b/drivers/virt/gunyah/vm_mgr_mm.c @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_vm_mgr: " fmt + +#include +#include + +#include + +#include "vm_mgr.h" + +static inline bool page_contiguous(phys_addr_t p, phys_addr_t t) +{ + return t - p == PAGE_SIZE; +} + +static struct gunyah_vm_memory_mapping *__gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label) +{ + struct gunyah_vm_memory_mapping *mapping; + + + list_for_each_entry(mapping, &ghvm->memory_mappings, list) + if (mapping->parcel.label == label) + return mapping; + + return NULL; +} + +void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_mapping *mapping) +{ + int i, ret = 0; + + if (mapping->parcel.mem_handle != GH_MEM_HANDLE_INVAL) { + ret = gh_rm_mem_reclaim(ghvm->rm, &mapping->parcel); + if (ret) + pr_warn("Failed to reclaim memory parcel for label %d: %d\n", + mapping->parcel.label, ret); + } + + if (!ret) + for (i = 0; i < mapping->npages; i++) + unpin_user_page(mapping->pages[i]); + + kfree(mapping->pages); + kfree(mapping->parcel.acl_entries); + kfree(mapping->parcel.mem_entries); + + mutex_lock(&ghvm->mm_lock); + list_del(&mapping->list); + mutex_unlock(&ghvm->mm_lock); +} + +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label) +{ + struct gunyah_vm_memory_mapping *mapping; + int ret; + + ret = mutex_lock_interruptible(&ghvm->mm_lock); + if (ret) + return ERR_PTR(ret); + mapping = __gh_vm_mem_mapping_find(ghvm, label); + mutex_unlock(&ghvm->mm_lock); + return mapping; +} + +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_alloc(struct gunyah_vm *ghvm, + struct gh_userspace_memory_region *region) +{ + phys_addr_t curr_page, prev_page; + struct gunyah_vm_memory_mapping *mapping, *tmp_mapping; + struct gh_rm_mem_entry *mem_entries; + int i, j, pinned, ret = 0; + struct gh_rm_mem_parcel *parcel; + + if (!region->memory_size || !PAGE_ALIGNED(region->memory_size) || + !PAGE_ALIGNED(region->userspace_addr)) + return ERR_PTR(-EINVAL); + + ret = mutex_lock_interruptible(&ghvm->mm_lock); + if (ret) + return ERR_PTR(ret); + mapping = __gh_vm_mem_mapping_find(ghvm, region->label); + if (mapping) { + ret = -EEXIST; + goto unlock; + } + + mapping = kzalloc(sizeof(*mapping), GFP_KERNEL); + if (!mapping) { + ret = -ENOMEM; + goto unlock; + } + + mapping->parcel.label = region->label; + mapping->guest_phys_addr = region->guest_phys_addr; + mapping->npages = region->memory_size >> PAGE_SHIFT; + parcel = &mapping->parcel; + parcel->mem_handle = GH_MEM_HANDLE_INVAL; /* to be filled later by mem_share/mem_lend */ + parcel->mem_type = GH_RM_MEM_TYPE_NORMAL; + + /* Check for overlap */ + list_for_each_entry(tmp_mapping, &ghvm->memory_mappings, list) { + if (!((mapping->guest_phys_addr + (mapping->npages << PAGE_SHIFT) <= + tmp_mapping->guest_phys_addr) || + (mapping->guest_phys_addr >= + tmp_mapping->guest_phys_addr + (tmp_mapping->npages << PAGE_SHIFT)))) { + ret = -EEXIST; + goto unlock; + } + } + + list_add(&mapping->list, &ghvm->memory_mappings); +unlock: + mutex_unlock(&ghvm->mm_lock); + if (ret) + goto free_mapping; + + mapping->pages = kcalloc(mapping->npages, sizeof(*mapping->pages), GFP_KERNEL); + if (!mapping->pages) { + ret = -ENOMEM; + goto reclaim; + } + + pinned = pin_user_pages_fast(region->userspace_addr, mapping->npages, + FOLL_WRITE | FOLL_LONGTERM, mapping->pages); + if (pinned < 0) { + ret = pinned; + goto reclaim; + } else if (pinned != mapping->npages) { + ret = -EFAULT; + mapping->npages = pinned; /* update npages for reclaim */ + goto reclaim; + } + + if (region->flags & GH_MEM_LENT) { + parcel->n_acl_entries = 1; + mapping->share_type = VM_MEM_LEND; + } else { + parcel->n_acl_entries = 2; + mapping->share_type = VM_MEM_SHARE; + } + parcel->acl_entries = kcalloc(parcel->n_acl_entries, + sizeof(*parcel->acl_entries), + GFP_KERNEL); + if (!parcel->acl_entries) { + ret = -ENOMEM; + goto reclaim; + } + + parcel->acl_entries[0].vmid = ghvm->vmid; + if (region->flags & GH_MEM_ALLOW_READ) + parcel->acl_entries[0].perms |= GH_RM_ACL_R; + if (region->flags & GH_MEM_ALLOW_WRITE) + parcel->acl_entries[0].perms |= GH_RM_ACL_W; + if (region->flags & GH_MEM_ALLOW_EXEC) + parcel->acl_entries[0].perms |= GH_RM_ACL_X; + + if (mapping->share_type == VM_MEM_SHARE) { + ret = gh_rm_get_vmid(ghvm->rm, &parcel->acl_entries[1].vmid); + if (ret) + goto reclaim; + /* Host assumed to have all these permissions. Gunyah will not + * grant new permissions if host actually had less than RWX + */ + parcel->acl_entries[1].perms |= GH_RM_ACL_R | GH_RM_ACL_W | GH_RM_ACL_X; + } + + mem_entries = kcalloc(mapping->npages, sizeof(*mem_entries), GFP_KERNEL); + if (!mem_entries) { + ret = -ENOMEM; + goto reclaim; + } + + // reduce number of entries by combining contiguous pages into single memory entry + prev_page = mem_entries[0].ipa_base = page_to_phys(mapping->pages[0]); + mem_entries[0].size = PAGE_SIZE; + for (i = 1, j = 0; i < mapping->npages; i++) { + curr_page = page_to_phys(mapping->pages[i]); + if (page_contiguous(prev_page, curr_page)) { + mem_entries[j].size += PAGE_SIZE; + } else { + j++; + mem_entries[j].ipa_base = curr_page; + mem_entries[j].size = PAGE_SIZE; + } + + prev_page = curr_page; + } + + parcel->n_mem_entries = j + 1; + parcel->mem_entries = kmemdup(mem_entries, sizeof(*mem_entries) * parcel->n_mem_entries, + GFP_KERNEL); + kfree(mem_entries); + if (!parcel->mem_entries) { + ret = -ENOMEM; + goto reclaim; + } + + switch (mapping->share_type) { + case VM_MEM_LEND: + ret = gh_rm_mem_lend(ghvm->rm, parcel); + break; + case VM_MEM_SHARE: + ret = gh_rm_mem_share(ghvm->rm, parcel); + break; + } + if (ret > 0) + ret = -EINVAL; + if (ret) + goto reclaim; + + return mapping; +reclaim: + gh_vm_mem_mapping_reclaim(ghvm, mapping); +free_mapping: + kfree(mapping); + return ERR_PTR(ret); +} diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 88a40d6e0b96..574f33b198d0 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -20,4 +20,26 @@ */ #define GH_CREATE_VM _IO(GH_IOCTL_TYPE, 0x0) /* Returns a Gunyah VM fd */ +/* + * ioctls for VM fds + */ +struct gh_userspace_memory_region { + __u32 label; +#define GH_MEM_ALLOW_READ (1UL << 0) +#define GH_MEM_ALLOW_WRITE (1UL << 1) +#define GH_MEM_ALLOW_EXEC (1UL << 2) +/* + * The guest will be lent the memory instead of shared. + * In other words, the guest has exclusive access to the memory region and the host loses access. + */ +#define GH_MEM_LENT (1UL << 3) + __u32 flags; + __u64 guest_phys_addr; + __u64 memory_size; + __u64 userspace_addr; +}; + +#define GH_VM_SET_USER_MEM_REGION _IOW(GH_IOCTL_TYPE, 0x1, \ + struct gh_userspace_memory_region) + #endif From patchwork Mon Dec 19 22:58:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DF04C4332F for ; Mon, 19 Dec 2022 23:04:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233026AbiLSXEj (ORCPT ); Mon, 19 Dec 2022 18:04:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233036AbiLSXCh (ORCPT ); Mon, 19 Dec 2022 18:02:37 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA41E15FDF; Mon, 19 Dec 2022 15:00:59 -0800 (PST) Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJM0DJp028616; Mon, 19 Dec 2022 23:00:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=S2ANioppZ6/C/QnLy9PdhKAcZQR2bbFolOCyfnsCock=; b=cNKamNrueDE94rH+e8Bxk4FY4sS5CRxDwbrWT7mzLL+QfeW60P+5akTyUDRDa19Otx5F +QZC8n2o1rf0ICaxUUkaPCH9lJ4rmHcS4wp157r1O88I+IaQAL3gEdeeKsviKSxShFwJ 6p94ztRHKaDkaonay9TLjzHwAr4bCIXO1pSgc06YdA0uwv+Ys9CGvUatzoHtJrZnaEsT xIVOEDQL2SCgb0/J0Bmsyx1d7SO6dzI3vLk/nFptIujelj3PD8kE+D0NBLbW8mnCs+i0 B5pCzu3tP1krIPLfdjsquD1iUmU+fNqAytP3K1j6cbbETK4cguLZUAnNEkyAiOoABmfK 9g== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mjyk505xh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:37 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0b2V007824 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:37 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:36 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Bagas Sanjaya" , Catalin Marinas , Will Deacon , Jassi Brar , "Sudeep Holla" , Mark Rutland , "Lorenzo Pieralisi" , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 15/28] gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot Date: Mon, 19 Dec 2022 14:58:36 -0800 Message-ID: <20221219225850.2397345-16-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: oxRFipY145lKFcDx-jwzXz3WOinh7nNE X-Proofpoint-ORIG-GUID: oxRFipY145lKFcDx-jwzXz3WOinh7nNE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 adultscore=0 clxscore=1015 suspectscore=0 impostorscore=0 bulkscore=0 priorityscore=1501 spamscore=0 lowpriorityscore=0 phishscore=0 mlxscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Add remaining ioctls to support non-proxy VM boot: - Gunyah Resource Manager uses the VM's devicetree to configure the virtual machine. The location of the devicetree in the guest's virtual memory can be declared via the SET_DTB_CONFIG ioctl. - Trigger start of the virtual machine with VM_START ioctl. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 92 ++++++++++++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 9 ++++ drivers/virt/gunyah/vm_mgr_mm.c | 24 +++++++++ include/uapi/linux/gunyah.h | 8 +++ 4 files changed, 132 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index a7220ad41669..a96cb7f73754 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -8,7 +8,7 @@ #include #include #include -#include +#include #include #include @@ -35,10 +35,79 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_rm_rpc *rm) mutex_init(&ghvm->mm_lock); INIT_LIST_HEAD(&ghvm->memory_mappings); + init_rwsem(&ghvm->status_lock); return ghvm; } +static int gh_vm_start(struct gunyah_vm *ghvm) +{ + struct gunyah_vm_memory_mapping *mapping; + u64 dtb_offset; + u32 mem_handle; + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE) { + up_write(&ghvm->status_lock); + return 0; + } + + mapping = gh_vm_mem_mapping_find_mapping(ghvm, + ghvm->dtb_config.gpa, ghvm->dtb_config.size); + if (!mapping) { + pr_warn("Failed to find the memory_handle for DTB\n"); + ret = -EINVAL; + goto err; + } + + mem_handle = mapping->parcel.mem_handle; + dtb_offset = ghvm->dtb_config.gpa - mapping->guest_phys_addr; + + ret = gh_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, mem_handle, + 0, 0, dtb_offset, ghvm->dtb_config.size); + if (ret) { + pr_warn("Failed to configure VM: %d\n", ret); + goto err; + } + + ret = gh_rm_vm_init(ghvm->rm, ghvm->vmid); + if (ret) { + pr_warn("Failed to initialize VM: %d\n", ret); + goto err; + } + + ret = gh_rm_vm_start(ghvm->rm, ghvm->vmid); + if (ret) { + pr_warn("Failed to start VM: %d\n", ret); + goto err; + } + + ghvm->vm_status = GH_RM_VM_STATUS_READY; + + up_write(&ghvm->status_lock); + return ret; +err: + ghvm->vm_status = GH_RM_VM_STATUS_INIT_FAILED; + up_write(&ghvm->status_lock); + return ret; +} + +static void gh_vm_stop(struct gunyah_vm *ghvm) +{ + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status == GH_RM_VM_STATUS_READY) { + ret = gh_rm_vm_stop(ghvm->rm, ghvm->vmid); + if (ret) + pr_warn("Failed to stop VM: %d\n", ret); + } + + ghvm->vm_status = GH_RM_VM_STATUS_EXITED; + up_write(&ghvm->status_lock); +} + static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { struct gunyah_vm *ghvm = filp->private_data; @@ -82,6 +151,25 @@ static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) } break; } + case GH_VM_SET_DTB_CONFIG: { + struct gh_vm_dtb_config dtb_config; + + r = -EFAULT; + if (copy_from_user(&dtb_config, argp, sizeof(dtb_config))) + break; + + dtb_config.size = PAGE_ALIGN(dtb_config.size); + ghvm->dtb_config = dtb_config; + + r = 0; + break; + } + case GH_VM_START: { + r = gh_vm_start(ghvm); + if (r) + r = -EINVAL; + break; + } default: r = -ENOTTY; break; @@ -95,6 +183,8 @@ static int gh_vm_release(struct inode *inode, struct file *filp) struct gunyah_vm *ghvm = filp->private_data; struct gunyah_vm_memory_mapping *mapping, *tmp; + gh_vm_stop(ghvm); + list_for_each_entry_safe(mapping, tmp, &ghvm->memory_mappings, list) { gh_vm_mem_mapping_reclaim(ghvm, mapping); } diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 955856c1310b..8dcba9f14b5c 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -10,6 +10,7 @@ #include #include #include +#include #include @@ -40,6 +41,12 @@ struct gunyah_vm { u16 vmid; struct gh_rm_rpc *rm; + enum gh_rm_vm_auth_mechanism auth; + struct gh_vm_dtb_config dtb_config; + + enum gh_rm_vm_status vm_status; + struct rw_semaphore status_lock; + struct mutex mm_lock; struct list_head memory_mappings; }; @@ -48,5 +55,7 @@ struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_alloc(struct gunyah_vm *ghvm, struct gh_userspace_memory_region *region); void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_mapping *mapping); struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label); +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find_mapping(struct gunyah_vm *ghvm, + u64 gpa, u32 size); #endif diff --git a/drivers/virt/gunyah/vm_mgr_mm.c b/drivers/virt/gunyah/vm_mgr_mm.c index 800d35da14ba..82d8e567b1a8 100644 --- a/drivers/virt/gunyah/vm_mgr_mm.c +++ b/drivers/virt/gunyah/vm_mgr_mm.c @@ -53,6 +53,30 @@ void gh_vm_mem_mapping_reclaim(struct gunyah_vm *ghvm, struct gunyah_vm_memory_m mutex_unlock(&ghvm->mm_lock); } +struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find_mapping(struct gunyah_vm *ghvm, + u64 gpa, u32 size) +{ + struct gunyah_vm_memory_mapping *mapping = NULL; + int ret; + + ret = mutex_lock_interruptible(&ghvm->mm_lock); + if (ret) + return ERR_PTR(ret); + + list_for_each_entry(mapping, &ghvm->memory_mappings, list) { + if (gpa >= mapping->guest_phys_addr && + (gpa + size <= mapping->guest_phys_addr + + (mapping->npages << PAGE_SHIFT))) { + goto unlock; + } + } + + mapping = NULL; +unlock: + mutex_unlock(&ghvm->mm_lock); + return mapping; +} + struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, u32 label) { struct gunyah_vm_memory_mapping *mapping; diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 574f33b198d0..36359ad2175e 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -42,4 +42,12 @@ struct gh_userspace_memory_region { #define GH_VM_SET_USER_MEM_REGION _IOW(GH_IOCTL_TYPE, 0x1, \ struct gh_userspace_memory_region) +struct gh_vm_dtb_config { + __u64 gpa; + __u64 size; +}; +#define GH_VM_SET_DTB_CONFIG _IOW(GH_IOCTL_TYPE, 0x2, struct gh_vm_dtb_config) + +#define GH_VM_START _IO(GH_IOCTL_TYPE, 0x3) + #endif From patchwork Mon Dec 19 22:58:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FC7FC4708E for ; Mon, 19 Dec 2022 23:05:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233013AbiLSXFp (ORCPT ); Mon, 19 Dec 2022 18:05:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232054AbiLSXDs (ORCPT ); Mon, 19 Dec 2022 18:03:48 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7722617047; Mon, 19 Dec 2022 15:01:20 -0800 (PST) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJIKvmX015286; Mon, 19 Dec 2022 23:00:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=SWJzjgVg9CxBHb/tHJ5rN/ghb1Be0SCQGn7z+gHv/Rw=; b=fH3cB2T1WJreeiJ/LAGY71vNdTnLJyvOqSGyObN9BGM7zDLU87yKQ/xEMExPM3HvJmzh NAnJRSEX9A98VAA932urzSj6HVTDq7KfRCoDHQUdZm8d3L0ATLdddnBhxcHoker8qmgV MHGdpie/Y8/NwFmxH6WeGONd0CLlkhgnd7l+gWTuU9X97AfpW94HO2hMCJ9CQMWfpHKw uGqDzuO5wyf/XWpig8+NRpCu71VNVhYs/Z15bi33YJIi1lXzg+VuPugEbFq+/6OIbZRP dj/GfsRpJvgRS9M+0ikdc+Ae//Pqe7W5ebkW6WVbAe2Lbj8iPPTf/tM1N8LtuxhWP289 cg== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh72gd4dg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:46 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0gBV006331 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:42 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:41 -0800 From: Elliot Berman To: Bjorn Andersson , Andy Gross , Bjorn Andersson , Konrad Dybcio , Srinivas Kandagatla , Amol Maheshwari , Arnd Bergmann , Greg Kroah-Hartman , Kalle Valo , "David S. Miller" , Eric Dumazet , "Jakub Kicinski" , Paolo Abeni , Mathieu Poirier CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Prakruthi Deepak Heragu , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Bagas Sanjaya , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , "Dmitry Baryshkov" , , , , , , , , , , Subject: [PATCH v8 18/28] firmware: qcom_scm: Use fixed width src vm bitmap Date: Mon, 19 Dec 2022 14:58:39 -0800 Message-ID: <20221219225850.2397345-19-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: rAJFpKPyIcVWf8l1hxpOMIY0DGylboWS X-Proofpoint-ORIG-GUID: rAJFpKPyIcVWf8l1hxpOMIY0DGylboWS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 clxscore=1011 suspectscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org The maximum VMID for assign_mem is 63. Use a u64 to represent this bitmap instead of architecture-dependent "unsigned int" which varies in size on 32-bit and 64-bit platforms. Acked-by: Kalle Valo [ath10k] Signed-off-by: Elliot Berman --- drivers/firmware/qcom_scm.c | 12 +++++++----- drivers/misc/fastrpc.c | 6 ++++-- drivers/net/wireless/ath/ath10k/qmi.c | 4 ++-- drivers/remoteproc/qcom_q6v5_mss.c | 8 ++++---- drivers/soc/qcom/rmtfs_mem.c | 2 +- include/linux/qcom_scm.h | 2 +- 6 files changed, 19 insertions(+), 15 deletions(-) diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c index cdbfe54c8146..92763dce6477 100644 --- a/drivers/firmware/qcom_scm.c +++ b/drivers/firmware/qcom_scm.c @@ -898,7 +898,7 @@ static int __qcom_scm_assign_mem(struct device *dev, phys_addr_t mem_region, * Return negative errno on failure or 0 on success with @srcvm updated. */ int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, - unsigned int *srcvm, + u64 *srcvm, const struct qcom_scm_vmperm *newvm, unsigned int dest_cnt) { @@ -915,9 +915,9 @@ int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, __le32 *src; void *ptr; int ret, i, b; - unsigned long srcvm_bits = *srcvm; + u64 srcvm_bits = *srcvm; - src_sz = hweight_long(srcvm_bits) * sizeof(*src); + src_sz = hweight64(srcvm_bits) * sizeof(*src); mem_to_map_sz = sizeof(*mem_to_map); dest_sz = dest_cnt * sizeof(*destvm); ptr_sz = ALIGN(src_sz, SZ_64) + ALIGN(mem_to_map_sz, SZ_64) + @@ -930,8 +930,10 @@ int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, /* Fill source vmid detail */ src = ptr; i = 0; - for_each_set_bit(b, &srcvm_bits, BITS_PER_LONG) - src[i++] = cpu_to_le32(b); + for (b = 0; b < BITS_PER_TYPE(u64); b++) { + if (srcvm_bits & BIT(b)) + src[i++] = cpu_to_le32(b); + } /* Fill details of mem buff to map */ mem_to_map = ptr + ALIGN(src_sz, SZ_64); diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 7ff0b63c25e3..2ad388f99fe1 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -299,11 +299,13 @@ static void fastrpc_free_map(struct kref *ref) if (map->attr & FASTRPC_ATTR_SECUREMAP) { struct qcom_scm_vmperm perm; int err = 0; + u64 src; + src = BIT(map->fl->cctx->vmperms[0].vmid); perm.vmid = QCOM_SCM_VMID_HLOS; perm.perm = QCOM_SCM_PERM_RWX; err = qcom_scm_assign_mem(map->phys, map->size, - &(map->fl->cctx->vmperms[0].vmid), &perm, 1); + &src, &perm, 1); if (err) { dev_err(map->fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d", map->phys, map->size, err); @@ -744,7 +746,7 @@ static int fastrpc_map_create(struct fastrpc_user *fl, int fd, * If subsystem VMIDs are defined in DTSI, then do * hyp_assign from HLOS to those VM(s) */ - unsigned int perms = BIT(QCOM_SCM_VMID_HLOS); + u64 perms = BIT(QCOM_SCM_VMID_HLOS); map->attr = attr; err = qcom_scm_assign_mem(map->phys, (u64)map->size, &perms, diff --git a/drivers/net/wireless/ath/ath10k/qmi.c b/drivers/net/wireless/ath/ath10k/qmi.c index 66cb7a1e628a..6d1d87e1cdde 100644 --- a/drivers/net/wireless/ath/ath10k/qmi.c +++ b/drivers/net/wireless/ath/ath10k/qmi.c @@ -28,7 +28,7 @@ static int ath10k_qmi_map_msa_permission(struct ath10k_qmi *qmi, { struct qcom_scm_vmperm dst_perms[3]; struct ath10k *ar = qmi->ar; - unsigned int src_perms; + u64 src_perms; u32 perm_count; int ret; @@ -60,7 +60,7 @@ static int ath10k_qmi_unmap_msa_permission(struct ath10k_qmi *qmi, { struct qcom_scm_vmperm dst_perms; struct ath10k *ar = qmi->ar; - unsigned int src_perms; + u64 src_perms; int ret; src_perms = BIT(QCOM_SCM_VMID_MSS_MSA) | BIT(QCOM_SCM_VMID_WLAN); diff --git a/drivers/remoteproc/qcom_q6v5_mss.c b/drivers/remoteproc/qcom_q6v5_mss.c index fddb63cffee0..9e8bde7a7ec4 100644 --- a/drivers/remoteproc/qcom_q6v5_mss.c +++ b/drivers/remoteproc/qcom_q6v5_mss.c @@ -227,8 +227,8 @@ struct q6v5 { bool has_qaccept_regs; bool has_ext_cntl_regs; bool has_vq6; - int mpss_perm; - int mba_perm; + u64 mpss_perm; + u64 mba_perm; const char *hexagon_mdt_image; int version; }; @@ -404,7 +404,7 @@ static void q6v5_pds_disable(struct q6v5 *qproc, struct device **pds, } } -static int q6v5_xfer_mem_ownership(struct q6v5 *qproc, int *current_perm, +static int q6v5_xfer_mem_ownership(struct q6v5 *qproc, u64 *current_perm, bool local, bool remote, phys_addr_t addr, size_t size) { @@ -939,7 +939,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw, struct page *page; dma_addr_t phys; void *metadata; - int mdata_perm; + u64 mdata_perm; int xferop_ret; size_t size; void *vaddr; diff --git a/drivers/soc/qcom/rmtfs_mem.c b/drivers/soc/qcom/rmtfs_mem.c index 0feaae357821..69991e47aa23 100644 --- a/drivers/soc/qcom/rmtfs_mem.c +++ b/drivers/soc/qcom/rmtfs_mem.c @@ -30,7 +30,7 @@ struct qcom_rmtfs_mem { unsigned int client_id; - unsigned int perms; + u64 perms; }; static ssize_t qcom_rmtfs_mem_show(struct device *dev, diff --git a/include/linux/qcom_scm.h b/include/linux/qcom_scm.h index f8335644a01a..77f7b5837216 100644 --- a/include/linux/qcom_scm.h +++ b/include/linux/qcom_scm.h @@ -96,7 +96,7 @@ extern int qcom_scm_mem_protect_video_var(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start, u32 cp_nonpixel_size); extern int qcom_scm_assign_mem(phys_addr_t mem_addr, size_t mem_sz, - unsigned int *src, + u64 *src, const struct qcom_scm_vmperm *newvm, unsigned int dest_cnt); From patchwork Mon Dec 19 22:58:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74B49C4332F for ; Mon, 19 Dec 2022 23:07:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233280AbiLSXHI (ORCPT ); Mon, 19 Dec 2022 18:07:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232959AbiLSXDF (ORCPT ); Mon, 19 Dec 2022 18:03:05 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 227CB1580F; Mon, 19 Dec 2022 15:01:07 -0800 (PST) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJMlBrb021788; Mon, 19 Dec 2022 23:00:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=xxb40SBhINMA+hmRpsx/CP63bdIdqGIdwKa3KzTzd2Q=; b=MFjJBomY/JgQu46hgZ7FcZntWCGdVry1ZFDVMNDxKD3nBjdZKd8TAWrjfU6fHIsBR88h 7fSY/vnQIClBN7dONfPQuE+r5wh/u9Eimd+kC/bB9UvoBn5pj69rqDyHegN+Q3PQQmGK 7WogD5HY3ykcRJuvjnjzOBYs4N+Eyx4XIDwkCr9d5jzinr6T0KnlS49gmPuZgaZkiwrM BLNmR8UhsYZzIjAaapGbJOo4c3YrNlWF/3NTngDtfwEGHRX0l6slZBmG0k5mbyhAO1zO X+gPIJD4sIpqgQIMQIiM0TetBiUPJf/HYq7CwHP6y/0Hlf89aYTaPbqqlLOPdqwSXWgh hA== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh72gd4dm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:45 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0jr1007358 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:45 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:44 -0800 From: Elliot Berman To: Bjorn Andersson , Jonathan Corbet , Bagas Sanjaya , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , "Dmitry Baryshkov" , , , , , , Subject: [PATCH v8 20/28] docs: gunyah: Document Gunyah VM Manager Date: Mon, 19 Dec 2022 14:58:41 -0800 Message-ID: <20221219225850.2397345-21-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: EZZ-VF57SvRhg0-nYdnWAYMpgUnGlh52 X-Proofpoint-ORIG-GUID: EZZ-VF57SvRhg0-nYdnWAYMpgUnGlh52 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 clxscore=1015 suspectscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Document the ioctls and usage of Gunyah VM Manager driver. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 1 + Documentation/virt/gunyah/vm-manager.rst | 100 +++++++++++++++++++++++ 2 files changed, 101 insertions(+) create mode 100644 Documentation/virt/gunyah/vm-manager.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst index fbadbdd24da7..9019a03b6f3e 100644 --- a/Documentation/virt/gunyah/index.rst +++ b/Documentation/virt/gunyah/index.rst @@ -7,6 +7,7 @@ Gunyah Hypervisor .. toctree:: :maxdepth: 1 + vm-manager message-queue Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst new file mode 100644 index 000000000000..62513af09cbf --- /dev/null +++ b/Documentation/virt/gunyah/vm-manager.rst @@ -0,0 +1,100 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================= +Virtual Machine Manager +======================= + +The Gunyah Virtual Machine Manager is a Linux driver to support launching +virtual machines using Gunyah. It presently supports launching non-proxy +scheduled Linux-like virtual machines. + +Except for some basic information about the location of initial binaries, +most of the configuration about a Gunyah virtual machine is described in the +VM's devicetree. The devicetree is generated by userspace. Interacting with the +virtual machine is still done via the kernel and VM configuration requires some +of the corresponding functionality to be set up in the kernel. For instance, +sharing userspace memory with a VM is done via the GH_VM_SET_USER_MEM_REGION +ioctl. The VM itself is configured to use the memory region via the +devicetree. + +Sample Userspace VMM +==================== + +A sample userspace VMM is included in samples/gunyah/ along with a minimal +devicetree that can be used to launch a VM. To build this sample, enable +CONFIG_SAMPLE_GUNYAH. + +IOCTLs and userspace VMM flows +============================== + +The kernel exposes a char device interface at /dev/gunyah. + +To create a VM, use the GH_CREATE_VM ioctl. A successful call will return a +"Gunyah VM" file descriptor. + +/dev/gunyah API Descriptions +---------------------------- + +GH_CREATE_VM +~~~~~~~~~~~~ + +Creates a Gunyah VM. The argument is reserved for future use and must be 0. + +Gunyah VM API Descriptions +-------------------------- + +GH_VM_SET_USER_MEM_REGION +~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + struct gh_userspace_memory_region { + __u32 label; + __u32 flags; + __u64 guest_phys_addr; + __u64 memory_size; + __u64 userspace_addr; + }; + +This ioctl allows the user to create or delete a memory parcel for a guest +virtual machine. Each memory region is uniquely identified by a label; +attempting to create two regions with the same label is not allowed. + +While VMM is guest-agnostic and allows runtime addition of memory regions, +Linux guest virtual machines do not support accepting memory regions at runtime. +Thus, memory regions should be provided before starting the VM and the VM must +be configured to accept these at boot-up. + +The guest physical address is used by Linux kernel to check that the requested +user regions do not overlap and to help find the corresponding memory region +for calls like GH_VM_SET_DTB_CONFIG. + +To delete a memory region, call GH_VM_SET_USER_MEM_REGION with label set to the +desired region and memory_size set to 0. + +The flags field of gh_userspace_memory_region accepts the following bits. All +other bits must be 0 and are reserved for future use. The ioctl will return +-EINVAL if an unsupported bit is detected. + + - GH_MEM_ALLOW_READ/GH_MEM_ALLOW_WRITE/GH_MEM_ALLOW_EXEC sets read/write/exec + permissions for the guest, respectively. + - GH_MEM_LENT means that the memory will be unmapped from the host and be + unaccessible by the host while the guest has the region. + +GH_VM_SET_DTB_CONFIG +~~~~~~~~~~~~~~~~~~~~ + +:: + + struct gh_vm_dtb_config { + __u64 gpa; + __u64 size; + }; + +This ioctl sets the location of the VM's devicetree blob and is used by Gunyah +Resource Manager to allocate resources. + +GH_VM_START +~~~~~~~~~~~ + +This ioctl starts the VM. From patchwork Mon Dec 19 22:58:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDF87C4167B for ; Mon, 19 Dec 2022 23:05:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232777AbiLSXFI (ORCPT ); Mon, 19 Dec 2022 18:05:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232964AbiLSXDG (ORCPT ); Mon, 19 Dec 2022 18:03:06 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22840167E2; Mon, 19 Dec 2022 15:01:07 -0800 (PST) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJM0L6d032477; Mon, 19 Dec 2022 23:00:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=Lrw4Tx4Z1cIpjGCdnAs+WPPB4YmlfGnD0l2AfnEQN6o=; b=dJ1GfEr80ZsvTgC27xJYV2Tsjtd4HpvFQYkKVdaKX4DBq7Q5NBWgDVsC0RyOPqbsLiq0 T1In1KQxrjSvSU5/XyW6Rzbe2Z/nx0YzbxatiJCakLzPxoo03jiPQCaVLnU5UNVf9NI0 IOCK1SsdQxaD7QkTGJxfaEobp8vOuvWNfNa3re3fzKCxrZjm1VqxQ1fIW3+jsKHJJrEq 2p9wWqcPlDq5YNrbi9T+0GzQEQZxlCdlTu7QBpoO+zJVoD8GW7X/0KMcm8qwtC/nEzW7 RDHhELl1mMt3NIQwtsHKtyxwqyTRdRXfzy2/KQGFJxnb1Co+1N4gbO7yA67i3V/6ATNw DQ== Received: from nasanppmta04.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh3fpne0x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:47 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0k6p019209 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:46 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:45 -0800 From: Elliot Berman To: Bjorn Andersson , Catalin Marinas , Will Deacon , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Bagas Sanjaya" , Jassi Brar , "Sudeep Holla" , Mark Rutland , "Lorenzo Pieralisi" , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 21/28] virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource Date: Mon, 19 Dec 2022 14:58:42 -0800 Message-ID: <20221219225850.2397345-22-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: PeJWdzlLywY9nqjNAIgu7oDGTrBda3Mp X-Proofpoint-GUID: PeJWdzlLywY9nqjNAIgu7oDGTrBda3Mp X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxscore=0 adultscore=0 mlxlogscore=999 priorityscore=1501 bulkscore=0 malwarescore=0 phishscore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org When booting a Gunyah virtual machine, the host VM may gain capabilities to interact with resources for the guest virtual machine. Examples of such resources are vCPUs or message queues. To use those resources, we need to translate the RM response into a gunyah_resource structure which are useful to Linux drivers. Presently, Linux drivers need only to know the type of resource, the capability ID, and an interrupt. On ARM64 systems, the interrupt reported by Gunyah is the GIC interrupt ID number and always a SPI. Signed-off-by: Elliot Berman --- arch/arm64/include/asm/gunyah.h | 23 +++++ drivers/virt/gunyah/rsc_mgr.c | 168 +++++++++++++++++++++++++++++++- include/linux/gunyah.h | 3 + include/linux/gunyah_rsc_mgr.h | 4 + 4 files changed, 197 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/gunyah.h diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h new file mode 100644 index 000000000000..64cfb964efee --- /dev/null +++ b/arch/arm64/include/asm/gunyah.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __ASM_GUNYAH_H_ +#define __ASM_GUNYAH_H_ + +#include +#include + +static inline int arch_gh_fill_irq_fwspec_params(u32 virq, struct irq_fwspec *fwspec) +{ + if (virq < 32 || virq > 1019) + return -EINVAL; + + fwspec->param_count = 3; + fwspec->param[0] = GIC_SPI; + fwspec->param[1] = virq - 32; + fwspec->param[2] = IRQ_TYPE_EDGE_RISING; + return 0; +} + +#endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index 49d7939bbe36..1010e725f1bd 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -18,6 +18,8 @@ #include #include +#include + #include "rsc_mgr.h" #include "vm_mgr.h" @@ -99,9 +101,145 @@ struct gh_rm_rpc { struct mutex send_lock; struct miscdevice miscdev; + struct irq_domain *irq_domain; + struct work_struct recv_work; }; +struct gh_irq_chip_data { + u32 gh_virq; +}; + +static struct irq_chip gh_rm_irq_chip = { + .name = "Gunyah", + .irq_enable = irq_chip_enable_parent, + .irq_disable = irq_chip_disable_parent, + .irq_ack = irq_chip_ack_parent, + .irq_mask = irq_chip_mask_parent, + .irq_mask_ack = irq_chip_mask_ack_parent, + .irq_unmask = irq_chip_unmask_parent, + .irq_eoi = irq_chip_eoi_parent, + .irq_set_affinity = irq_chip_set_affinity_parent, + .irq_set_type = irq_chip_set_type_parent, + .irq_set_wake = irq_chip_set_wake_parent, + .irq_set_vcpu_affinity = irq_chip_set_vcpu_affinity_parent, + .irq_retrigger = irq_chip_retrigger_hierarchy, + .irq_get_irqchip_state = irq_chip_get_parent_state, + .irq_set_irqchip_state = irq_chip_set_parent_state, + .flags = IRQCHIP_SET_TYPE_MASKED | + IRQCHIP_SKIP_SET_WAKE | + IRQCHIP_MASK_ON_SUSPEND, +}; + +static int gh_rm_irq_domain_alloc(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs, + void *arg) +{ + struct irq_fwspec parent_fwspec; + struct gh_irq_chip_data *chip_data, *spec = arg; + u32 gh_virq = spec->gh_virq; + struct gh_rm_rpc *rsc_mgr = d->host_data; + int ret; + + if (nr_irqs != 1 || gh_virq == U32_MAX) + return -EINVAL; + + chip_data = kzalloc(sizeof(*chip_data), GFP_KERNEL); + if (!chip_data) + return -ENOMEM; + + chip_data->gh_virq = gh_virq; + + ret = irq_domain_set_hwirq_and_chip(d, virq, chip_data->gh_virq, &gh_rm_irq_chip, + chip_data); + if (ret) + return ret; + + parent_fwspec.fwnode = d->parent->fwnode; + ret = arch_gh_fill_irq_fwspec_params(chip_data->gh_virq, &parent_fwspec); + if (ret) { + dev_err(rsc_mgr->dev, "virq translation failed %u: %d\n", chip_data->gh_virq, ret); + goto err_free_irq_data; + } + + ret = irq_domain_alloc_irqs_parent(d, virq, nr_irqs, &parent_fwspec); + if (ret) + goto err_free_irq_data; + + return ret; +err_free_irq_data: + kfree(chip_data); + return ret; +} + +static void gh_rm_irq_domain_free_single(struct irq_domain *d, unsigned int virq) +{ + struct irq_data *irq_data; + struct gh_irq_chip_data *chip_data; + + irq_data = irq_domain_get_irq_data(d, virq); + if (!irq_data) + return; + + chip_data = irq_data->chip_data; + + kfree(chip_data); + irq_data->chip_data = NULL; +} + +static void gh_rm_irq_domain_free(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs) +{ + unsigned int i; + + for (i = 0; i < nr_irqs; i++) + gh_rm_irq_domain_free_single(d, virq); +} + +static const struct irq_domain_ops gh_rm_irq_domain_ops = { + .alloc = gh_rm_irq_domain_alloc, + .free = gh_rm_irq_domain_free, +}; + +struct gunyah_resource *gh_rm_alloc_resource(struct gh_rm_rpc *rm, + struct gh_rm_hyp_resource *hyp_resource) +{ + struct gunyah_resource *ghrsc; + + ghrsc = kzalloc(sizeof(*ghrsc), GFP_KERNEL); + if (!ghrsc) + return NULL; + + ghrsc->type = hyp_resource->type; + ghrsc->capid = le64_to_cpu(hyp_resource->cap_id); + ghrsc->irq = IRQ_NOTCONNECTED; + ghrsc->rm_label = le32_to_cpu(hyp_resource->resource_label); + /* Gunyah doesn't send us virq_handle anymore and will always give us an interrupt + * that's ready to be used. Defensively check that it's not set. + */ + if (hyp_resource->virq_handle) { + pr_warn_once("Unexpected virq handle.\n"); + } else if (hyp_resource->virq && le32_to_cpu(hyp_resource->virq) != U32_MAX) { + struct gh_irq_chip_data irq_data = { + .gh_virq = le32_to_cpu(hyp_resource->virq), + }; + + ghrsc->irq = irq_domain_alloc_irqs(rm->irq_domain, 1, NUMA_NO_NODE, &irq_data); + if (ghrsc->irq < 0) { + pr_err("Failed to allocate interrupt: %d\n", ghrsc->irq); + ghrsc->irq = IRQ_NOTCONNECTED; + } + } + + return ghrsc; +} +EXPORT_SYMBOL_GPL(gh_rm_alloc_resource); + +void gh_rm_free_resource(struct gunyah_resource *ghrsc) +{ + irq_dispose_mapping(ghrsc->irq); + kfree(ghrsc); +} +EXPORT_SYMBOL_GPL(gh_rm_free_resource); + static struct gh_rm_connection *gh_rm_alloc_connection(u32 msg_id, u8 type) { struct gh_rm_connection *connection; @@ -552,6 +690,8 @@ static int gh_msgq_platform_probe_direction(struct platform_device *pdev, static int gh_rm_drv_probe(struct platform_device *pdev) { struct gh_rm_rpc *rsc_mgr; + struct device_node *parent_irq_node; + struct irq_domain *parent_irq_domain; int ret; rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); @@ -583,15 +723,40 @@ static int gh_rm_drv_probe(struct platform_device *pdev) ret = gh_msgq_init(&pdev->dev, &rsc_mgr->msgq, &rsc_mgr->msgq_client, &rsc_mgr->tx_ghrsc, &rsc_mgr->rx_ghrsc); + parent_irq_node = of_irq_find_parent(pdev->dev.of_node); + if (!parent_irq_node) { + dev_err(&pdev->dev, "Failed to find interrupt parent of resource manager\n"); + ret = -ENODEV; + goto err_msgq; + } + + parent_irq_domain = irq_find_host(parent_irq_node); + if (!parent_irq_domain) { + dev_err(&pdev->dev, "Failed to find interrupt parent domain of resource manager\n"); + ret = -ENODEV; + goto err_msgq; + } + + rsc_mgr->irq_domain = irq_domain_add_hierarchy(parent_irq_domain, 0, 0, pdev->dev.of_node, + &gh_rm_irq_domain_ops, NULL); + if (!rsc_mgr->irq_domain) { + dev_err(&pdev->dev, "Failed to add irq domain\n"); + ret = -ENODEV; + goto err_msgq; + } + rsc_mgr->irq_domain->host_data = rsc_mgr; + rsc_mgr->miscdev.name = "gunyah"; rsc_mgr->miscdev.minor = MISC_DYNAMIC_MINOR; rsc_mgr->miscdev.fops = &gh_dev_fops; ret = misc_register(&rsc_mgr->miscdev); if (ret) - goto err_msgq; + goto err_irq_domain; return 0; +err_irq_domain: + irq_domain_remove(rsc_mgr->irq_domain); err_msgq: mbox_free_channel(gh_msgq_chan(&rsc_mgr->msgq)); gh_msgq_remove(&rsc_mgr->msgq); @@ -603,6 +768,7 @@ static int gh_rm_drv_remove(struct platform_device *pdev) struct gh_rm_rpc *rm = platform_get_drvdata(pdev); misc_deregister(&rm->miscdev); + irq_domain_remove(rm->irq_domain); mbox_free_channel(gh_msgq_chan(&rm->msgq)); gh_msgq_remove(&rm->msgq); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index e317d7ac938f..8cb6af88c75d 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -26,6 +26,9 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; int irq; + + /* To help allocator in resource manager */ + u32 rm_label; }; /** diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index be434bcf9b4e..eee993a33971 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -115,6 +115,10 @@ ssize_t gh_rm_get_hyp_resources(struct gh_rm_rpc *rm, u16 vmid, struct gh_rm_hyp_resource **resources); int gh_rm_get_vmid(struct gh_rm_rpc *rm, u16 *vmid); +struct gunyah_resource *gh_rm_alloc_resource(struct gh_rm_rpc *rm, + struct gh_rm_hyp_resource *hyp_resource); +void gh_rm_free_resource(struct gunyah_resource *ghrsc); + struct gunyah_rm_platform_ops { int (*pre_mem_share)(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel); int (*post_mem_reclaim)(struct gh_rm_rpc *rm, struct gh_rm_mem_parcel *mem_parcel); From patchwork Mon Dec 19 22:58:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4CE6C46467 for ; Mon, 19 Dec 2022 23:03:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233092AbiLSXD3 (ORCPT ); Mon, 19 Dec 2022 18:03:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232993AbiLSXB4 (ORCPT ); Mon, 19 Dec 2022 18:01:56 -0500 Received: from alexa-out-sd-02.qualcomm.com (alexa-out-sd-02.qualcomm.com [199.106.114.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2E5263B2; Mon, 19 Dec 2022 15:00:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1671490848; x=1703026848; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Sf8syiRSzClrN1+SVtumeEs5knGjEyeRSly4hps5WZE=; b=YtsjAxROak9/TbuMqvHoZVjC449AdIquRYl50X4JqC/iOHNKqjSXVuvk zcLmp52ASvl2hck5PT3hLvWN9E+bmnxLZLI5cA585gKZXgWfx/JINSMXn dKdDy1I25Xy35gvQnSG53pRZ/uJ3UNVfUateXKvOItvIKOJO2uTw9rJle s=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-02.qualcomm.com with ESMTP; 19 Dec 2022 15:00:48 -0800 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Dec 2022 15:00:48 -0800 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:47 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala , Jonathan Corbet CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Bagas Sanjaya , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 22/28] gunyah: vm_mgr: Add framework to add VM Functions Date: Mon, 19 Dec 2022 14:58:43 -0800 Message-ID: <20221219225850.2397345-23-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Introduce a framework for Gunyah userspace to install VM functions. VM functions are optional interfaces to the virtual machine. vCPUs, ioeventfs, and irqfds are examples of such VM functions and are implemented in subsequent patches. A generic framework is implemented instead of individual ioctls to create vCPUs, irqfds, etc., in order to simplify the VM manager core implementation and allow dynamic loading of VM function modules. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/vm-manager.rst | 18 +++ drivers/virt/gunyah/vm_mgr.c | 145 ++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 5 + include/linux/gunyah_vm_mgr.h | 68 +++++++++++ include/uapi/linux/gunyah.h | 12 ++ 5 files changed, 247 insertions(+), 1 deletion(-) create mode 100644 include/linux/gunyah_vm_mgr.h diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst index 62513af09cbf..d929f8f20b3b 100644 --- a/Documentation/virt/gunyah/vm-manager.rst +++ b/Documentation/virt/gunyah/vm-manager.rst @@ -17,6 +17,24 @@ sharing userspace memory with a VM is done via the GH_VM_SET_USER_MEM_REGION ioctl. The VM itself is configured to use the memory region via the devicetree. +Gunyah Functions +================ + +Components of a Gunyah VM's configuration that need kernel configuration are +called "functions" and are built on top of a framework. Functions are identified +by a string and have some argument(s) to configure them. They are typically +created by the `GH_VM_ADD_FUNCTION` ioctl. + +Functions typically will always do at least one of these operations: + +1. Create resource ticket(s). Resource tickets allow a function to register + itself as the client for a Gunyah resource (e.g. doorbell or vCPU) and + the function is given the pointer to the `struct gunyah_resource` when the + VM is starting. + +2. Register IO handler(s). IO handlers allow a function to handle stage-2 faults + from the virtual machine. + Sample Userspace VMM ==================== diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index a96cb7f73754..a5bda4d6613b 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -15,6 +15,125 @@ #include "vm_mgr.h" +static DEFINE_MUTEX(functions_lock); +static LIST_HEAD(functions); + +static struct gunyah_vm_function_driver *__find_function(const char name[GUNYAH_FUNCTION_NAME_SIZE]) + __must_hold(functions_lock) +{ + struct gunyah_vm_function_driver *iter, *drv = NULL; + + list_for_each_entry(iter, &functions, list) { + if (!strncmp(iter->name, name, GUNYAH_FUNCTION_NAME_SIZE)) { + drv = iter; + break; + } + } + + return drv; +} + +int gunyah_vm_function_register(struct gunyah_vm_function_driver *drv) +{ + int ret = 0; + + mutex_lock(&functions_lock); + if (__find_function(drv->name)) { + ret = -EEXIST; + goto out; + } + + INIT_LIST_HEAD(&drv->instances); + list_add(&drv->list, &functions); +out: + mutex_unlock(&functions_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_vm_function_register); + +static void gh_vm_remove_function(struct gunyah_vm_function *f) + __must_hold(functions_lock) +{ + f->drv->release(f); + list_del(&f->vm_list); + list_del(&f->drv_list); + module_put(f->drv->mod); +} + +void gunyah_vm_function_unregister(struct gunyah_vm_function_driver *drv) +{ + struct gunyah_vm_function *f, *iter; + + mutex_lock(&functions_lock); + list_for_each_entry_safe(f, iter, &drv->instances, drv_list) + gh_vm_remove_function(f); + list_del(&drv->list); + mutex_unlock(&functions_lock); +} +EXPORT_SYMBOL_GPL(gunyah_vm_function_unregister); + +static long gh_vm_add_function(struct gunyah_vm *ghvm, struct gunyah_vm_function *f) +{ + long r = 0; + + mutex_lock(&functions_lock); + f->drv = __find_function(f->fn.name); + if (!f->drv) { + mutex_unlock(&functions_lock); + r = request_module("ghfunc:%s", f->fn.name); + if (r) + return r; + + mutex_lock(&functions_lock); + f->drv = __find_function(f->fn.name); + } + + if (!f->drv) { + r = -ENOENT; + goto out; + } + + if (!try_module_get(f->drv->mod)) { + r = -ENOENT; + f->drv = NULL; + goto out; + } + + f->ghvm = ghvm; + f->rm = ghvm->rm; + + r = f->drv->bind(f); + if (r < 0) { + module_put(f->drv->mod); + goto out; + } + + list_add(&f->vm_list, &ghvm->functions); + list_add(&f->drv_list, &f->drv->instances); +out: + mutex_unlock(&functions_lock); + return r; +} + +static void ghvm_put(struct kref *kref) +{ + struct gunyah_vm *ghvm = container_of(kref, struct gunyah_vm, kref); + + kfree(ghvm); +} + +int __must_check get_gunyah_vm(struct gunyah_vm *ghvm) +{ + return kref_get_unless_zero(&ghvm->kref); +} +EXPORT_SYMBOL_GPL(get_gunyah_vm); + +void put_gunyah_vm(struct gunyah_vm *ghvm) +{ + kref_put(&ghvm->kref, ghvm_put); +} +EXPORT_SYMBOL_GPL(put_gunyah_vm); + static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_rm_rpc *rm) { struct gunyah_vm *ghvm; @@ -36,6 +155,8 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_rm_rpc *rm) mutex_init(&ghvm->mm_lock); INIT_LIST_HEAD(&ghvm->memory_mappings); init_rwsem(&ghvm->status_lock); + kref_init(&ghvm->kref); + INIT_LIST_HEAD(&ghvm->functions); return ghvm; } @@ -170,6 +291,23 @@ static long gh_vm_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) r = -EINVAL; break; } + case GH_VM_ADD_FUNCTION: { + struct gunyah_vm_function *f; + + r = -ENOMEM; + f = kzalloc(sizeof(*f), GFP_KERNEL); + if (!f) + break; + + r = -EFAULT; + if (copy_from_user(&f->fn, argp, sizeof(f->fn))) + break; + + r = gh_vm_add_function(ghvm, f); + if (r < 0) + kfree(f); + break; + } default: r = -ENOTTY; break; @@ -182,14 +320,19 @@ static int gh_vm_release(struct inode *inode, struct file *filp) { struct gunyah_vm *ghvm = filp->private_data; struct gunyah_vm_memory_mapping *mapping, *tmp; + struct gunyah_vm_function *f, *iter; gh_vm_stop(ghvm); + list_for_each_entry_safe(f, iter, &ghvm->functions, vm_list) { + gh_vm_remove_function(f); + } list_for_each_entry_safe(mapping, tmp, &ghvm->memory_mappings, list) { gh_vm_mem_mapping_reclaim(ghvm, mapping); } + put_gh_rm(ghvm->rm); - kfree(ghvm); + put_gunyah_vm(ghvm); return 0; } diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 8dcba9f14b5c..2c4fd240792a 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -8,6 +8,7 @@ #include #include +#include #include #include #include @@ -47,8 +48,12 @@ struct gunyah_vm { enum gh_rm_vm_status vm_status; struct rw_semaphore status_lock; + struct kref kref; + struct mutex mm_lock; struct list_head memory_mappings; + + struct list_head functions; }; struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_alloc(struct gunyah_vm *ghvm, diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h new file mode 100644 index 000000000000..4bb06f100ae5 --- /dev/null +++ b/include/linux/gunyah_vm_mgr.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_VM_MGR_H +#define _GUNYAH_VM_MGR_H + +#include +#include +#include +#include +#include +#include + +#include + +struct gunyah_vm; + +int __must_check get_gunyah_vm(struct gunyah_vm *ghvm); +void put_gunyah_vm(struct gunyah_vm *ghvm); + +struct gunyah_vm_function; +struct gunyah_vm_function_driver { + const char name[GUNYAH_FUNCTION_NAME_SIZE]; + struct module *mod; + long (*bind)(struct gunyah_vm_function *f); + void (*release)(struct gunyah_vm_function *f); + struct list_head list; + struct list_head instances; +}; + +struct gunyah_vm_function { + struct gh_vm_function fn; + struct gunyah_vm *ghvm; + struct gh_rm_rpc *rm; + struct gunyah_vm_function_driver *drv; + void *data; + struct list_head vm_list; + struct list_head drv_list; +}; + +int gunyah_vm_function_register(struct gunyah_vm_function_driver *f); +void gunyah_vm_function_unregister(struct gunyah_vm_function_driver *f); + +#define DECLARE_GUNYAH_VM_FUNCTION(_name, _bind, _release) \ + static struct gunyah_vm_function_driver _name = { \ + .name = __stringify(_name), \ + .mod = THIS_MODULE, \ + .bind = _bind, \ + .release = _release, \ + }; \ + MODULE_ALIAS("ghfunc:"__stringify(_name)) + +#define DECLARE_GUNYAH_VM_FUNCTION_INIT(_name, _bind, _release) \ + DECLARE_GUNYAH_VM_FUNCTION(_name, _bind, _release); \ + static int __init _name##_mod_init(void) \ + { \ + return gunyah_vm_function_register(&(_name)); \ + } \ + module_init(_name##_mod_init); \ + static void __exit _name##_mod_exit(void) \ + { \ + gunyah_vm_function_unregister(&(_name)); \ + } \ + module_exit(_name##_mod_exit) + +#endif diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index 36359ad2175e..40db32aeb012 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -50,4 +50,16 @@ struct gh_vm_dtb_config { #define GH_VM_START _IO(GH_IOCTL_TYPE, 0x3) +#define GUNYAH_FUNCTION_NAME_SIZE 32 +#define GUNYAH_FUNCTION_MAX_ARG_SIZE 1024 + +struct gh_vm_function { + char name[GUNYAH_FUNCTION_NAME_SIZE]; + union { + char data[GUNYAH_FUNCTION_MAX_ARG_SIZE]; + }; +}; + +#define GH_VM_ADD_FUNCTION _IOW(GH_IOCTL_TYPE, 0x4, struct gh_vm_function) + #endif From patchwork Mon Dec 19 22:58:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61884C4708E for ; Mon, 19 Dec 2022 23:05:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233172AbiLSXFY (ORCPT ); Mon, 19 Dec 2022 18:05:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232989AbiLSXD2 (ORCPT ); Mon, 19 Dec 2022 18:03:28 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B683167E8; Mon, 19 Dec 2022 15:01:09 -0800 (PST) Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJMN9Wk027753; Mon, 19 Dec 2022 23:00:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=CEq8c94IBG6Nvn6CnZAOeBhIxN7kOp8McXlsMOcGt/E=; b=ZDrqc8PJ9eBpd3DhCkpHL8mw95krPSUw8tEyJeWctopdnojP7lQQkWAl7foR4X39MYkT 4j0fO6QyLz1NkBM9v4lMGuCT+Q4R5QKVrZqxvhQYbwcGVDjtekt8WsgOF3rV2W2JuaJp N6j/9Yh0/114HXK03wG7WurEgfh7rzJiAmADpkjJpG5JlF2q4Gh/dvgcTwsiF2tatKan fyA5TnWVvVjMl/xT1jeWnl+ze9meK330aF6X65tVsbh29C7vkhkTWuRXhJDR6WJ2DAuP 9hbumyWnrPghdhBjlHX3eUC0W/v7o0zdnnuqDSryCAhr9xlso6ujWFGZi8RuhVYv17Ng Jg== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh34udrf7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:50 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0n2W007943 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:49 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:48 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Bagas Sanjaya" , Catalin Marinas , Will Deacon , Jassi Brar , "Sudeep Holla" , Mark Rutland , "Lorenzo Pieralisi" , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 23/28] virt: gunyah: Add resource tickets Date: Mon, 19 Dec 2022 14:58:44 -0800 Message-ID: <20221219225850.2397345-24-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 4_OVl2SDQwk6HGf4b6SFB5ryrjLKLYYt X-Proofpoint-GUID: 4_OVl2SDQwk6HGf4b6SFB5ryrjLKLYYt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 bulkscore=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 priorityscore=1501 phishscore=0 impostorscore=0 mlxscore=0 adultscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Some VM functions need to acquire Gunyah resources. For instance, Gunyah vCPUs are exposed to the host as a resource. The Gunyah vCPU function will register a resource ticket and be able to interact with the hypervisor once the resource ticket is filled. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 93 ++++++++++++++++++++++++++++++++++- drivers/virt/gunyah/vm_mgr.h | 5 ++ include/linux/gunyah.h | 1 + include/linux/gunyah_vm_mgr.h | 14 ++++++ 4 files changed, 112 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index a5bda4d6613b..3328b84fa681 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -115,6 +115,56 @@ static long gh_vm_add_function(struct gunyah_vm *ghvm, struct gunyah_vm_function return r; } +int ghvm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_vm_resource_ticket *iter; + struct gunyah_resource *ghrsc; + int ret = 0; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(iter, &ghvm->resource_tickets, list) { + if (iter->resource_type == ticket->resource_type && iter->label == ticket->label) { + ret = -EEXIST; + goto out; + } + } + + if (!try_module_get(ticket->owner)) { + ret = -ENODEV; + goto out; + } + + list_add(&ticket->list, &ghvm->resource_tickets); + INIT_LIST_HEAD(&ticket->resources); + + list_for_each_entry(ghrsc, &ghvm->resources, list) { + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) { + if (!ticket->populate(ticket, ghrsc)) + list_move(&ghrsc->list, &ticket->resources); + } + } +out: + mutex_unlock(&ghvm->resources_lock); + return ret; +} +EXPORT_SYMBOL_GPL(ghvm_add_resource_ticket); + +void ghvm_remove_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_resource *ghrsc; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(ghrsc, &ticket->resources, list) { + ticket->unpopulate(ticket, ghrsc); + list_move(&ghrsc->list, &ghvm->resources); + } + + module_put(ticket->owner); + list_del(&ticket->list); + mutex_unlock(&ghvm->resources_lock); +} +EXPORT_SYMBOL_GPL(ghvm_remove_resource_ticket); + static void ghvm_put(struct kref *kref) { struct gunyah_vm *ghvm = container_of(kref, struct gunyah_vm, kref); @@ -156,17 +206,41 @@ static __must_check struct gunyah_vm *gunyah_vm_alloc(struct gh_rm_rpc *rm) INIT_LIST_HEAD(&ghvm->memory_mappings); init_rwsem(&ghvm->status_lock); kref_init(&ghvm->kref); + mutex_init(&ghvm->resources_lock); + INIT_LIST_HEAD(&ghvm->resources); + INIT_LIST_HEAD(&ghvm->resource_tickets); INIT_LIST_HEAD(&ghvm->functions); return ghvm; } +static void gh_vm_add_resource(struct gunyah_vm *ghvm, struct gunyah_resource *ghrsc) +{ + struct gunyah_vm_resource_ticket *ticket; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(ticket, &ghvm->resource_tickets, list) { + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) { + if (!ticket->populate(ticket, ghrsc)) { + list_add(&ghrsc->list, &ticket->resources); + goto found; + } + } + } + list_add(&ghrsc->list, &ghvm->resources); +found: + mutex_unlock(&ghvm->resources_lock); +} + static int gh_vm_start(struct gunyah_vm *ghvm) { struct gunyah_vm_memory_mapping *mapping; u64 dtb_offset; u32 mem_handle; - int ret; + ssize_t num_hyp_resources; + struct gunyah_resource *ghrsc; + struct gh_rm_hyp_resource *resources; + int ret, i; down_write(&ghvm->status_lock); if (ghvm->vm_status != GH_RM_VM_STATUS_NO_STATE) { @@ -198,6 +272,23 @@ static int gh_vm_start(struct gunyah_vm *ghvm) goto err; } + num_hyp_resources = gh_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); + if (num_hyp_resources < 0) { + pr_warn("Failed to get hypervisor resources for VM: %ld\n", num_hyp_resources); + ret = -EINVAL; + goto err; + } + + for (i = 0; i < num_hyp_resources; i++) { + ghrsc = gh_rm_alloc_resource(ghvm->rm, &resources[i]); + if (!ghrsc) { + ret = -ENOMEM; + goto err; + } + + gh_vm_add_resource(ghvm, ghrsc); + } + ret = gh_rm_vm_start(ghvm->rm, ghvm->vmid); if (ret) { pr_warn("Failed to start VM: %d\n", ret); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 2c4fd240792a..8a71f97960bc 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -7,6 +7,7 @@ #define _GH_PRIV_VM_MGR_H #include +#include #include #include #include @@ -53,6 +54,10 @@ struct gunyah_vm { struct mutex mm_lock; struct list_head memory_mappings; + struct mutex resources_lock; + struct list_head resources; + struct list_head resource_tickets; + struct list_head functions; }; diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 8cb6af88c75d..af69a3479025 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -28,6 +28,7 @@ struct gunyah_resource { int irq; /* To help allocator in resource manager */ + struct list_head list; u32 rm_label; }; diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h index 4bb06f100ae5..3e6e6e0ab7f0 100644 --- a/include/linux/gunyah_vm_mgr.h +++ b/include/linux/gunyah_vm_mgr.h @@ -65,4 +65,18 @@ void gunyah_vm_function_unregister(struct gunyah_vm_function_driver *f); } \ module_exit(_name##_mod_exit) +struct gunyah_vm_resource_ticket { + struct list_head list; + struct list_head resources; + enum gunyah_resource_type resource_type; + u32 label; + + struct module *owner; + int (*populate)(struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc); + void (*unpopulate)(struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc); +}; + +int ghvm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket); +void ghvm_remove_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket); + #endif From patchwork Mon Dec 19 22:58:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30C1BC3DA78 for ; Mon, 19 Dec 2022 23:05:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233073AbiLSXFX (ORCPT ); Mon, 19 Dec 2022 18:05:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233086AbiLSXD3 (ORCPT ); Mon, 19 Dec 2022 18:03:29 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AFBB15807; Mon, 19 Dec 2022 15:01:09 -0800 (PST) Received: from pps.filterd (m0279869.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJMvhas000408; Mon, 19 Dec 2022 23:00:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=8DBfpRw1P55kghMA1skoq1VZPxEn4zlEz457ZcB/ftk=; b=hCAU7T9RFl8J9xpxEVKp89JkjYx9olGDDgbTeo3WtVGJLIeh6lUxYXERCeqwHcRizbZj 70mY7avvqVZCBlOd+pv4EVCEvu2ijGEb0dc9uljfTyrLcuqupe3hEL7c7oCPnHdgoOSm Wcg3OS5HJVArgQXGV0wUnu6b0Qvv1SZ7qkWLeen/irfS1ZlHpG46btS/YplTCt6XTvah e5vNUyWcJN++vi0n4MBf+Pj2Y0RdPTPRmLPAX0gm2DQn7AAI3MVX2p8BAEl/CwLBf5Kj uHo1Lt9EMqdxyK7u43zulEb+Su6QESmjbVMrtI1IyjGv03PP0k/4SbmXHYOrGtjXMNBY Pg== Received: from nasanppmta05.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh4sedk88-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:52 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA05.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0ow7007956 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:50 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:50 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , "Bagas Sanjaya" , Catalin Marinas , Will Deacon , Jassi Brar , "Sudeep Holla" , Mark Rutland , "Lorenzo Pieralisi" , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 24/28] virt: gunyah: Add IO handlers Date: Mon, 19 Dec 2022 14:58:45 -0800 Message-ID: <20221219225850.2397345-25-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 4oHTluD6IInIRKW9dxLPspFOq9X51blc X-Proofpoint-ORIG-GUID: 4oHTluD6IInIRKW9dxLPspFOq9X51blc X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 impostorscore=0 priorityscore=1501 adultscore=0 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Add framework for VM functions to handle stage-2 write faults from Gunyah guest virtual machines. IO handlers have a range of addresses which they apply to. Optionally, they may apply to only when the value written matches the IO handler's value. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- drivers/virt/gunyah/vm_mgr.c | 85 +++++++++++++++++++++++++++++++++++ drivers/virt/gunyah/vm_mgr.h | 4 ++ include/linux/gunyah_vm_mgr.h | 25 +++++++++++ 3 files changed, 114 insertions(+) diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c index 3328b84fa681..dcca255bc11f 100644 --- a/drivers/virt/gunyah/vm_mgr.c +++ b/drivers/virt/gunyah/vm_mgr.c @@ -165,6 +165,91 @@ void ghvm_remove_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resour } EXPORT_SYMBOL_GPL(ghvm_remove_resource_ticket); +static inline bool gh_vm_io_handler_matches(struct gunyah_vm_io_handler *io_hdlr, u64 addr, + u64 len, u64 data) +{ + u64 mask = BIT_ULL(io_hdlr->len * BITS_PER_BYTE) - 1; + + if (io_hdlr->addr != addr) + return false; + + if (!io_hdlr->datamatch) + return true; + + if (io_hdlr->len != len) + return false; + + return (data & mask) == (io_hdlr->data & mask); +} + +static struct gunyah_vm_io_handler *gh_vm_mgr_find_io_hdlr(struct gunyah_vm *ghvm, u64 addr, + u64 len, u64 data) +{ + struct gunyah_vm_io_handler *io_hdlr = NULL; + struct rb_node *root = NULL; + + root = ghvm->mmio_handler_root.rb_node; + while (root) { + io_hdlr = rb_entry(root, struct gunyah_vm_io_handler, node); + if (addr < io_hdlr->addr) + root = root->rb_left; + else if (addr > io_hdlr->addr) + root = root->rb_right; + else if (gh_vm_io_handler_matches(io_hdlr, addr, len, data)) + return io_hdlr; + } + return NULL; +} + +int gh_vm_mgr_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data) +{ + struct gunyah_vm_io_handler *io_hdlr = NULL; + + io_hdlr = gh_vm_mgr_find_io_hdlr(ghvm, addr, len, data); + if (!io_hdlr) + return -ENODEV; + + if (io_hdlr->ops && io_hdlr->ops->write) + return io_hdlr->ops->write(io_hdlr, addr, len, data); + + return -ENODEV; +} +EXPORT_SYMBOL_GPL(gh_vm_mgr_mmio_write); + +int gh_vm_mgr_add_io_handler(struct gunyah_vm *ghvm, struct gunyah_vm_io_handler *io_hdlr) +{ + struct rb_node **root, *parent = NULL; + + if (io_hdlr->datamatch && + (!io_hdlr->len || io_hdlr->len > (sizeof(io_hdlr->data) * BITS_PER_BYTE))) + return -EINVAL; + + root = &ghvm->mmio_handler_root.rb_node; + while (*root) { + struct gunyah_vm_io_handler *curr = rb_entry(*root, struct gunyah_vm_io_handler, + node); + + parent = *root; + if (io_hdlr->addr < curr->addr) + root = &((*root)->rb_left); + else if (io_hdlr->addr > curr->addr) + root = &((*root)->rb_right); + else + return -EEXIST; + } + + rb_link_node(&io_hdlr->node, parent, root); + rb_insert_color(&io_hdlr->node, &ghvm->mmio_handler_root); + return 0; +} +EXPORT_SYMBOL_GPL(gh_vm_mgr_add_io_handler); + +void gh_vm_mgr_remove_io_handler(struct gunyah_vm *ghvm, struct gunyah_vm_io_handler *io_hdlr) +{ + rb_erase(&io_hdlr->node, &ghvm->mmio_handler_root); +} +EXPORT_SYMBOL_GPL(gh_vm_mgr_remove_io_handler); + static void ghvm_put(struct kref *kref) { struct gunyah_vm *ghvm = container_of(kref, struct gunyah_vm, kref); diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h index 8a71f97960bc..46ba2e530be2 100644 --- a/drivers/virt/gunyah/vm_mgr.h +++ b/drivers/virt/gunyah/vm_mgr.h @@ -58,6 +58,8 @@ struct gunyah_vm { struct list_head resources; struct list_head resource_tickets; + struct rb_root mmio_handler_root; + struct list_head functions; }; @@ -68,4 +70,6 @@ struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find(struct gunyah_vm *ghvm, struct gunyah_vm_memory_mapping *gh_vm_mem_mapping_find_mapping(struct gunyah_vm *ghvm, u64 gpa, u32 size); +int gh_vm_mgr_mmio_write(struct gunyah_vm *ghvm, u64 addr, u32 len, u64 data); + #endif diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h index 3e6e6e0ab7f0..5b17cb54ba01 100644 --- a/include/linux/gunyah_vm_mgr.h +++ b/include/linux/gunyah_vm_mgr.h @@ -79,4 +79,29 @@ struct gunyah_vm_resource_ticket { int ghvm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket); void ghvm_remove_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket); +/* + * gunyah_vm_io_handler contains the info about an io device and its associated + * addr and the ops associated with the io device. + */ +struct gunyah_vm_io_handler { + struct rb_node node; + u64 addr; + + bool datamatch; + u8 len; + u64 data; + struct gunyah_vm_io_handler_ops *ops; +}; + +/* + * gunyah_vm_io_handler_ops contains function pointers associated with an iodevice. + */ +struct gunyah_vm_io_handler_ops { + int (*read)(struct gunyah_vm_io_handler *io_dev, u64 addr, u32 len, u64 data); + int (*write)(struct gunyah_vm_io_handler *io_dev, u64 addr, u32 len, u64 data); +}; + +int gh_vm_mgr_add_io_handler(struct gunyah_vm *ghvm, struct gunyah_vm_io_handler *io_dev); +void gh_vm_mgr_remove_io_handler(struct gunyah_vm *ghvm, struct gunyah_vm_io_handler *io_dev); + #endif From patchwork Mon Dec 19 22:58:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 635208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5610C10F1D for ; Mon, 19 Dec 2022 23:05:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233202AbiLSXFn (ORCPT ); Mon, 19 Dec 2022 18:05:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233009AbiLSXDo (ORCPT ); Mon, 19 Dec 2022 18:03:44 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2F98167FE; Mon, 19 Dec 2022 15:01:12 -0800 (PST) Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BJIKvmY015286; Mon, 19 Dec 2022 23:00:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=D3j+GJokMFdBeRJ2Xes3rbW9E8kTJM1GvmXc4zDqJAc=; b=HBu83y0/uAMJArdMUGnpvY1AKDw7TcrYAYpgEkGdprxeopmwSue7o6EatGo1A9yWk4TF p9s5LTyHKEGx6Jie0XwEdXQph1Tw6xe4hJ8xwQ2XVHXQyZjuzWLiX0fZCdrv1UrCem5c WiKFO4nL2v3UIxgmx0H2Wm1P8kXz0Ni83Hz2rP6i4jqseYy+eODM1gAzMtlfxVC0hu+0 6+u+Ozkjp/F/82sJEnxn5+aUjE9yMUa7vFZplLKsPob9xCWH/UNmOjAdVFpDbivtIL9j GKc/SHWiO9QKxn+X8xQhkWypLMkt4frBbIiUzomwDhlw+1QIXWpdXTsebF9N5/BOGmqI mw== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mh72gd4e1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:56 +0000 Received: from nasanex01b.na.qualcomm.com (corens_vlan604_snip.qualcomm.com [10.53.140.1]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BJN0ucQ006910 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 19 Dec 2022 23:00:56 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Mon, 19 Dec 2022 15:00:55 -0800 From: Elliot Berman To: Bjorn Andersson , Elliot Berman , Murali Nalajala , Jonathan Corbet CC: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Bagas Sanjaya , Catalin Marinas , Will Deacon , Jassi Brar , Sudeep Holla , Mark Rutland , Lorenzo Pieralisi , Dmitry Baryshkov , , , , , , Subject: [PATCH v8 27/28] virt: gunyah: Add irqfd interface Date: Mon, 19 Dec 2022 14:58:48 -0800 Message-ID: <20221219225850.2397345-28-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221219225850.2397345-1-quic_eberman@quicinc.com> References: <20221219225850.2397345-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 5or1HWkH4LQxbY2CovGsT7oOrv1RQ2EZ X-Proofpoint-ORIG-GUID: 5or1HWkH4LQxbY2CovGsT7oOrv1RQ2EZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-19_01,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 clxscore=1015 suspectscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212190202 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Enable support for creating irqfds which can raise an interrupt on a Gunyah virtual machine. irqfds are exposed to userspace as a Gunyah VM function with the name "irqfd". If the VM devicetree is not configured to create a doorbell with the corresponding label, userspace will still be able to assert the eventfd but no interrupt will be raised on the guest. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/vm-manager.rst | 22 +++ drivers/virt/gunyah/Kconfig | 10 ++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_irqfd.c | 180 +++++++++++++++++++++++ include/linux/gunyah.h | 5 + include/uapi/linux/gunyah.h | 11 +- 6 files changed, 228 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/gunyah_irqfd.c diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst index d11267d59802..b6cf8db826b8 100644 --- a/Documentation/virt/gunyah/vm-manager.rst +++ b/Documentation/virt/gunyah/vm-manager.rst @@ -142,3 +142,25 @@ The vcpu type will register with the VM Manager to expect to control vCPU number `vcpu_id`. It returns a file descriptor allowing interaction with the vCPU. See the Gunyah vCPU API description sections for interacting with the Gunyah vCPU file descriptors. + +Type: "irqfd" +^^^^^^^^^^^^^ + +:: + + struct gh_fn_irqfd_arg { + __u32 fd; + __u32 label; + #define GH_IRQFD_LEVEL (1UL << 0) + #define GH_IRQFD_DEASSIGN (1UL << 1) + __u32 flags; + }; + +Allows setting an eventfd to directly trigger a guest interrupt. +irqfd.fd specifies the file descriptor to use as the eventfd. +irqfd.label corresponds to the doorbell label used in the guest VM's devicetree. +The irqfd is removed using the GH_IRQFD_DEASSIGN flag and specifying at least +the irqfd.label. + +GH_IRQFD_LEVEL configures the corresponding doorbell to behave like a level +triggered interrupt. diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index f9551cd5dfe4..42842e755a04 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -49,5 +49,15 @@ config GUNYAH_VCPU can schedule the guest VM's vCPUs instead of using Gunyah's scheduler. VMMs can also handle stage 2 faults of the vCPUs. + Say Y/M here if unsure and you want to support Gunyah VMMs. + +config GUNYAH_IRQFD + tristate "Gunyah irqfd interface" + depends on GUNYAH_RESOURCE_MANAGER + depends on GUNYAH_VM_MANAGER + help + Enable kernel support for creating irqfds which can raise an interrupt + on Gunyah virtual machine. + Say Y/M here if unsure and you want to support Gunyah VMMs. endif diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index debdd7e4d9a1..065eeb060597 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -6,3 +6,4 @@ gunyah_rsc_mgr-$(CONFIG_GUNYAH_VM_MANAGER) += vm_mgr.o vm_mgr_mm.o obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER) += gunyah_rsc_mgr.o obj-$(CONFIG_GUNYAH_VCPU) += gunyah_vcpu.o +obj-$(CONFIG_GUNYAH_IRQFD) += gunyah_irqfd.o diff --git a/drivers/virt/gunyah/gunyah_irqfd.c b/drivers/virt/gunyah/gunyah_irqfd.c new file mode 100644 index 000000000000..34ec389323e2 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_irqfd.c @@ -0,0 +1,180 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +struct gunyah_irqfd { + struct gunyah_resource *ghrsc; + struct gunyah_vm_resource_ticket ticket; + struct gunyah_vm_function *f; + + struct kref kref; + bool level; + + struct eventfd_ctx *ctx; + wait_queue_entry_t wait; + poll_table pt; + struct fd fd; + struct work_struct shutdown_work; +}; + +static void gh_irqfd_cleanup(struct kref *kref) +{ + struct gunyah_irqfd *irqfd = container_of(kref, struct gunyah_irqfd, kref); + + kfree(irqfd); +} + + +static void irqfd_shutdown(struct work_struct *work) +{ + struct gunyah_irqfd *irqfd = container_of(work, struct gunyah_irqfd, shutdown_work); + u64 isr; + + if (irqfd->ctx) { + eventfd_ctx_remove_wait_queue(irqfd->ctx, &irqfd->wait, &isr); + eventfd_ctx_put(irqfd->ctx); + fdput(irqfd->fd); + irqfd->ctx = NULL; + irqfd->fd.file = NULL; + } + + kref_put(&irqfd->kref, gh_irqfd_cleanup); +} + +static int irqfd_wakeup(wait_queue_entry_t *wait, unsigned int mode, int sync, void *key) +{ + struct gunyah_irqfd *irqfd = container_of(wait, struct gunyah_irqfd, wait); + __poll_t flags = key_to_poll(key); + u64 enable_mask = GH_DBL_NONBLOCK; + u64 old_flags; + int ret = 0; + + if (flags & EPOLLIN) { + if (irqfd->ghrsc) { + ret = gh_hypercall_dbl_send(irqfd->ghrsc->capid, enable_mask, &old_flags); + if (ret) + pr_err("Failed to assert irq %d\n", irqfd->f->fn.irqfd.label); + } + } + + return 0; +} + +static void irqfd_ptable_queue_proc(struct file *file, wait_queue_head_t *wqh, poll_table *pt) +{ + struct gunyah_irqfd *irq_ctx = container_of(pt, struct gunyah_irqfd, pt); + + add_wait_queue(wqh, &irq_ctx->wait); +} + +static int gunyah_irqfd_populate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_irqfd *irqfd = container_of(ticket, struct gunyah_irqfd, ticket); + u64 enable_mask = GH_DBL_NONBLOCK; + u64 ack_mask = ~0; + int ret = 0; + + irqfd->ghrsc = ghrsc; + if (irqfd->level) { + ret = gh_hypercall_dbl_set_mask(irqfd->ghrsc->capid, enable_mask, ack_mask); + if (ret) + pr_warn("irq %d couldn't be set as level triggered. Might cause IRQ storm if asserted\n", + irqfd->f->fn.irqfd.label); + } + kref_get(&irqfd->kref); + + return 0; +} + +static void gunyah_irqfd_unpopulate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_irqfd *irqfd = container_of(ticket, struct gunyah_irqfd, ticket); + + queue_work(system_wq, &irqfd->shutdown_work); + irqfd->ghrsc = NULL; + kref_put(&irqfd->kref, gh_irqfd_cleanup); +} + +static long gunyah_irqfd_bind(struct gunyah_vm_function *f) +{ + __poll_t events; + struct gunyah_irqfd *irqfd; + long r; + + irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL); + if (!irqfd) + return -ENOMEM; + + irqfd->f = f; + f->data = irqfd; + + irqfd->fd = fdget(f->fn.irqfd.fd); + if (!irqfd->fd.file) { + r = -EBADF; + goto err_free; + } + + irqfd->ctx = eventfd_ctx_fileget(irqfd->fd.file); + if (IS_ERR(irqfd->ctx)) { + r = PTR_ERR(irqfd->ctx); + goto err_fdput; + } + + if (f->fn.irqfd.flags & GH_IRQFD_LEVEL) + irqfd->level = true; + + irqfd->ticket.resource_type = GUNYAH_RESOURCE_TYPE_BELL_TX; + irqfd->ticket.label = f->fn.irqfd.label; + irqfd->ticket.owner = THIS_MODULE; + irqfd->ticket.populate = gunyah_irqfd_populate; + irqfd->ticket.unpopulate = gunyah_irqfd_unpopulate; + + r = ghvm_add_resource_ticket(f->ghvm, &irqfd->ticket); + if (r) + goto err_ctx; + + init_waitqueue_func_entry(&irqfd->wait, irqfd_wakeup); + INIT_WORK(&irqfd->shutdown_work, irqfd_shutdown); + init_poll_funcptr(&irqfd->pt, irqfd_ptable_queue_proc); + kref_init(&irqfd->kref); + + events = vfs_poll(irqfd->fd.file, &irqfd->pt); + if (events & EPOLLIN) + pr_warn("Premature injection of interrupt\n"); + + return 0; +err_ctx: + eventfd_ctx_put(irqfd->ctx); +err_fdput: + fdput(irqfd->fd); +err_free: + kfree(irqfd); + return r; +} + +static void gunyah_irqfd_release(struct gunyah_vm_function *f) +{ + struct gunyah_irqfd *irqfd = f->data; + + /* unpopulate will trigger clean up of the eventfd */ + ghvm_remove_resource_ticket(irqfd->f->ghvm, &irqfd->ticket); +} + +DECLARE_GUNYAH_VM_FUNCTION_INIT(irqfd, gunyah_irqfd_bind, gunyah_irqfd_release); +MODULE_DESCRIPTION("Gunyah irqfds"); +MODULE_LICENSE("GPL"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 2c2b498a792d..db5910adc658 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -32,6 +32,11 @@ struct gunyah_resource { u32 rm_label; }; +/** + * Gunyah Doorbells + */ +#define GH_DBL_NONBLOCK BIT(32) + /** * Gunyah Message Queues */ diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h index ccc0e6d0f985..5eab212c7bf5 100644 --- a/include/uapi/linux/gunyah.h +++ b/include/uapi/linux/gunyah.h @@ -57,10 +57,19 @@ struct gh_fn_vcpu_arg { __u32 vcpu_id; }; +struct gh_fn_irqfd_arg { + __u32 fd; + __u32 label; +#define GH_IRQFD_LEVEL (1UL << 0) +#define GH_IRQFD_DEASSIGN (1UL << 1) + __u32 flags; +}; + struct gh_vm_function { char name[GUNYAH_FUNCTION_NAME_SIZE]; union { - struct gh_device_vcpu_arg vcpu; + struct gh_fn_vcpu_arg vcpu; + struct gh_fn_irqfd_arg irqfd; char data[GUNYAH_FUNCTION_MAX_ARG_SIZE]; }; };