From patchwork Thu Apr 13 09:07:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi-De Wu X-Patchwork-Id: 674252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EA6BC77B77 for ; Thu, 13 Apr 2023 09:08:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230063AbjDMJIV (ORCPT ); Thu, 13 Apr 2023 05:08:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229814AbjDMJIR (ORCPT ); Thu, 13 Apr 2023 05:08:17 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C2BD1FE9; Thu, 13 Apr 2023 02:08:10 -0700 (PDT) X-UUID: b2cd488ed9da11edb6b9f13eb10bd0fe-20230413 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=6/PuYp06JANxyecmjfdkLJaYOHvRKSYhlev5QUYet24=; b=i8oHJz+Tb42zhRG/Ecq97/bF4fDTQ3lZNe7weXKaGMIG6UtkqPBui+3E67lCWW13PQ1nZRPUHqj6OaOXDFNmZsjeQdITK1WD6bhF+NSsGzN0LLMmd4QvDxEL0FM2zwLwI2M+tfBzSmG+RBaAB0wOeL2b2KbvtuynZs5Pw6IAg+A=; X-CID-P-RULE: Spam_GS6885AD X-CID-O-INFO: VERSION:1.1.22, REQID:4028f8c9-7f03-4f3b-8ef4-14f6199d431d, IP:0, U RL:0,TC:0,Content:49,EDM:0,RT:0,SF:100,FILE:0,BULK:0,RULE:Spam_GS6885AD,AC TION:quarantine,TS:149 X-CID-INFO: VERSION:1.1.22, REQID:4028f8c9-7f03-4f3b-8ef4-14f6199d431d, IP:0, URL :0,TC:0,Content:49,EDM:0,RT:0,SF:100,FILE:0,BULK:0,RULE:Spam_US65DF41,ACTI ON:quarantine,TS:149 X-CID-META: VersionHash:120426c, CLOUDID:77cb34a1-8fcb-430b-954a-ba3f00fa94a5, B ulkID:230413170805VK1B60VM,BulkQuantity:0,Recheck:0,SF:48|38|29|28|17|19,T C:nil,Content:3,EDM:-3,IP:nil,URL:0,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-UUID: b2cd488ed9da11edb6b9f13eb10bd0fe-20230413 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1838058108; Thu, 13 Apr 2023 17:08:04 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.185) by mtkmbs13n1.mediatek.inc (172.21.101.193) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Thu, 13 Apr 2023 17:08:03 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Thu, 13 Apr 2023 17:08:03 +0800 From: Yi-De Wu To: Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Matthias Brugger , AngeloGioacchino Del Regno , Yingshiuan Pan CC: , , , , , Jades Shih , Miles Chen , Ivan Tseng , My Chuang , Shawn Hsiao , PeiLun Suei , Ze-Yu Wang , Liju Chen , Yi-De Wu Subject: [PATCH v1 1/6] docs: geniezone: Introduce GenieZone hypervisor Date: Thu, 13 Apr 2023 17:07:30 +0800 Message-ID: <20230413090735.4182-2-yi-de.wu@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230413090735.4182-1-yi-de.wu@mediatek.com> References: <20230413090735.4182-1-yi-de.wu@mediatek.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org From: "Yingshiuan Pan" GenieZone is MediaTek proprietary hypervisor solution, and it is running in EL2 stand alone as a type-I hypervisor. It is a pure EL2 implementation which implies it does not rely any specific host VM, and this behavior improves GenieZone's security as it limits its interface. Signed-off-by: Yingshiuan Pan Signed-off-by: Yi-De Wu --- Documentation/virt/geniezone/introduction.rst | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 Documentation/virt/geniezone/introduction.rst diff --git a/Documentation/virt/geniezone/introduction.rst b/Documentation/virt/geniezone/introduction.rst new file mode 100644 index 000000000000..1fffd6cbb4db --- /dev/null +++ b/Documentation/virt/geniezone/introduction.rst @@ -0,0 +1,34 @@ +.. SPDX-License-Identifier: GPL-2.0 + +====================== +GenieZone Introduction +====================== + + +Overview +======== +GenieZone is MediaTek proprietary hypervisor solution, and it is running in EL2 +stand alone as a type-I hypervisor. It is a pure EL2 implementation which +implies it does not rely any specific host VM, and this behavior improves +GenieZone's security as it limits its interface. + +To enable guest VMs running, a driver (gzvm) is provided for VMM (virtual +machine monitor) to operate. Currently, the gzvm driver supports only crosvm. + + +Supported Architecture +====================== +GenieZone now only supports MediaTek arm64 SoC. + + +Platform Virtualization +======================= +We leverages arm64's timer virtualization and gic virtualization for timer and +interrupts controller. + + +Device Virtualizaton +==================== +We adopts VMM's virtio devices emulations by passing io trap to VMM, and virtio +is a well-known and widely used virtual device implementation. + From patchwork Thu Apr 13 09:07:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi-De Wu X-Patchwork-Id: 672949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8CD6C77B76 for ; Thu, 13 Apr 2023 09:08:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229526AbjDMJIU (ORCPT ); Thu, 13 Apr 2023 05:08:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229894AbjDMJIR (ORCPT ); Thu, 13 Apr 2023 05:08:17 -0400 Received: from mailgw01.mediatek.com (unknown [60.244.123.138]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3ABC82115; Thu, 13 Apr 2023 02:08:11 -0700 (PDT) X-UUID: b2ea07e4d9da11eda9a90f0bb45854f4-20230413 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=vJHA62fePV8KQrTjmeo7P6N3t6CmbXBI+Yee42ikhhA=; b=JTE0oSKbzx/HUynH0CeT7pZ1NvBqEiUx2DIp1Rqe5azePIbcoeCUQt0Ncuzmo03STT/h1wY1SrvBaYjdjaRxGHH196mFgxMvhLxSJqG6JwVn0Uhru8a0aKLeh03AZAxAb6rWKYr8CkmPgPZ8aRIcMXfraJXXD/RhoIMeyUe8JZY=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.22, REQID:d699e2ff-2b66-47c4-ae56-af4a10e23262, IP:0, U RL:25,TC:0,Content:0,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTION :release,TS:25 X-CID-META: VersionHash:120426c, CLOUDID:5fcfef83-cd9c-45f5-8134-710979e3df0e, B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:11|1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-UUID: b2ea07e4d9da11eda9a90f0bb45854f4-20230413 Received: from mtkmbs10n2.mediatek.inc [(172.21.101.183)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1529083461; Thu, 13 Apr 2023 17:08:04 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.185) by mtkmbs13n2.mediatek.inc (172.21.101.108) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Thu, 13 Apr 2023 17:08:03 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Thu, 13 Apr 2023 17:08:03 +0800 From: Yi-De Wu To: Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Matthias Brugger , AngeloGioacchino Del Regno , Yingshiuan Pan CC: , , , , , Jades Shih , Miles Chen , Ivan Tseng , My Chuang , Shawn Hsiao , PeiLun Suei , Ze-Yu Wang , Liju Chen , Yi-De Wu Subject: [PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor Date: Thu, 13 Apr 2023 17:07:31 +0800 Message-ID: <20230413090735.4182-3-yi-de.wu@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230413090735.4182-1-yi-de.wu@mediatek.com> References: <20230413090735.4182-1-yi-de.wu@mediatek.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org From: "Yingshiuan Pan" Add documentation for GenieZone(gzvm) node. This node informs gzvm driver to start probing if geniezone hypervisor is available and able to do virtual machine operations. Signed-off-by: Yingshiuan Pan Signed-off-by: Yi-De Wu --- .../bindings/hypervisor/mediatek,gzvm.yaml | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml diff --git a/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml new file mode 100644 index 000000000000..35e1e5b18e47 --- /dev/null +++ b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml @@ -0,0 +1,30 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/hypervisor/mediatek,gzvm.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: MediaTek GenieZone hypervisor + +maintainers: + - Yingshiuan Pan + +description: + GenieZone is MediaTek proprietary hypervisor. This device node informs its + driver, gzvm, to probe if platform supports running virtual machines. + +properties: + compatible: + const: mediatek,gzvm + +required: + - compatible + +additionalProperties: false + +examples: + - | + hypervisor { + compatible = "mediatek,gzvm"; + status = "okay"; + }; From patchwork Thu Apr 13 09:07:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi-De Wu X-Patchwork-Id: 674250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C222FC77B78 for ; Thu, 13 Apr 2023 09:08:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230356AbjDMJIp (ORCPT ); Thu, 13 Apr 2023 05:08:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230203AbjDMJIc (ORCPT ); Thu, 13 Apr 2023 05:08:32 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2A5372B5; Thu, 13 Apr 2023 02:08:13 -0700 (PDT) X-UUID: b2afe1d6d9da11edb6b9f13eb10bd0fe-20230413 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=olRRmiU1/pTF0B/8ckrG+n+5m4jMa0/aKWyzDPOZNvw=; b=dUm/YN/hBDyTGRg3yAx8RLV80ynL1F+XOdbgRfBsmoHFpjYdGz7C7/s0q6AtorWI6bJLQ/UY+rO1EdPgIj3oA0JxoQuuYOomH+qBiiYoRXVgzytOYLr8pSA7GwIr6eg7cG8OOYXVivAsQpYhWG607QpR5/+U2BstssrSLaSL5wE=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.22, REQID:afd53fbd-ddf6-441d-bfa8-349f7a9e0784, IP:0, U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:70 X-CID-INFO: VERSION:1.1.22, REQID:afd53fbd-ddf6-441d-bfa8-349f7a9e0784, IP:0, URL :0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Spam_GS981B3D,ACTI ON:quarantine,TS:70 X-CID-META: VersionHash:120426c, CLOUDID:48cfef83-cd9c-45f5-8134-710979e3df0e, B ulkID:230413170806V7S30LKE,BulkQuantity:0,Recheck:0,SF:19|48|38|29|28|17,T C:nil,Content:0,EDM:-3,IP:nil,URL:11|1,File:nil,Bulk:nil,QS:nil,BEC:nil,CO L:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-UUID: b2afe1d6d9da11edb6b9f13eb10bd0fe-20230413 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 925928137; Thu, 13 Apr 2023 17:08:04 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.185) by mtkmbs11n1.mediatek.inc (172.21.101.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Thu, 13 Apr 2023 17:08:03 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Thu, 13 Apr 2023 17:08:03 +0800 From: Yi-De Wu To: Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Matthias Brugger , AngeloGioacchino Del Regno , Yingshiuan Pan CC: , , , , , Jades Shih , Miles Chen , Ivan Tseng , My Chuang , Shawn Hsiao , PeiLun Suei , Ze-Yu Wang , Liju Chen , Yi-De Wu Subject: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support Date: Thu, 13 Apr 2023 17:07:32 +0800 Message-ID: <20230413090735.4182-4-yi-de.wu@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230413090735.4182-1-yi-de.wu@mediatek.com> References: <20230413090735.4182-1-yi-de.wu@mediatek.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org From: "Yingshiuan Pan" GenieZone is MediaTek proprietary hypervisor solution, and it is running in EL2 stand alone as a type-I hypervisor. This patch exports a set of ioctl interfaces for userspace VMM (e.g., crosvm) to operate guest VMs lifecycle (creation, running, and destroy) on GenieZone. Signed-off-by: Yingshiuan Pan Signed-off-by: Yi-De Wu --- arch/arm64/include/uapi/asm/gzvm_arch.h | 79 ++++ drivers/soc/mediatek/Kconfig | 2 + drivers/soc/mediatek/Makefile | 1 + drivers/soc/mediatek/virt/geniezone/Kconfig | 17 + drivers/soc/mediatek/virt/geniezone/Makefile | 5 + drivers/soc/mediatek/virt/geniezone/gzvm.h | 103 ++++ .../soc/mediatek/virt/geniezone/gzvm_hyp.h | 72 +++ .../soc/mediatek/virt/geniezone/gzvm_main.c | 233 +++++++++ .../soc/mediatek/virt/geniezone/gzvm_vcpu.c | 266 +++++++++++ drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 444 ++++++++++++++++++ include/uapi/linux/gzvm_common.h | 217 +++++++++ 11 files changed, 1439 insertions(+) create mode 100644 arch/arm64/include/uapi/asm/gzvm_arch.h create mode 100644 drivers/soc/mediatek/virt/geniezone/Kconfig create mode 100644 drivers/soc/mediatek/virt/geniezone/Makefile create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm.h create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_main.c create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c create mode 100644 include/uapi/linux/gzvm_common.h diff --git a/arch/arm64/include/uapi/asm/gzvm_arch.h b/arch/arm64/include/uapi/asm/gzvm_arch.h new file mode 100644 index 000000000000..3714f96c832b --- /dev/null +++ b/arch/arm64/include/uapi/asm/gzvm_arch.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#ifndef __GZVM_ARCH_H__ +#define __GZVM_ARCH_H__ + +#include + +/* + * Architecture specific registers are to be defined in arch headers and + * ORed with the arch identifier. + */ +#define GZVM_REG_ARM 0x4000000000000000ULL +#define GZVM_REG_ARM64 0x6000000000000000ULL + +#define GZVM_REG_SIZE_SHIFT 52 +#define GZVM_REG_SIZE_MASK 0x00f0000000000000ULL +#define GZVM_REG_SIZE_U8 0x0000000000000000ULL +#define GZVM_REG_SIZE_U16 0x0010000000000000ULL +#define GZVM_REG_SIZE_U32 0x0020000000000000ULL +#define GZVM_REG_SIZE_U64 0x0030000000000000ULL +#define GZVM_REG_SIZE_U128 0x0040000000000000ULL +#define GZVM_REG_SIZE_U256 0x0050000000000000ULL +#define GZVM_REG_SIZE_U512 0x0060000000000000ULL +#define GZVM_REG_SIZE_U1024 0x0070000000000000ULL +#define GZVM_REG_SIZE_U2048 0x0080000000000000ULL + +#define GZVM_NR_SPSR 5 +struct gzvm_regs { + struct user_pt_regs regs; /* sp = sp_el0 */ + + __u64 sp_el1; + __u64 elr_el1; + + __u64 spsr[GZVM_NR_SPSR]; + + struct user_fpsimd_state fp_regs; +}; + +/* If you need to interpret the index values, here is the key: */ +#define GZVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 +#define GZVM_REG_ARM_COPROC_SHIFT 16 + +/* Normal registers are mapped as coprocessor 16. */ +#define GZVM_REG_ARM_CORE (0x0010 << GZVM_REG_ARM_COPROC_SHIFT) +#define GZVM_REG_ARM_CORE_REG(name) (offsetof(struct gzvm_regs, name) / sizeof(__u32)) + +/* Some registers need more space to represent values. */ +#define GZVM_REG_ARM_DEMUX (0x0011 << GZVM_REG_ARM_COPROC_SHIFT) +#define GZVM_REG_ARM_DEMUX_ID_MASK 0x000000000000FF00 +#define GZVM_REG_ARM_DEMUX_ID_SHIFT 8 +#define GZVM_REG_ARM_DEMUX_ID_CCSIDR (0x00 << GZVM_REG_ARM_DEMUX_ID_SHIFT) +#define GZVM_REG_ARM_DEMUX_VAL_MASK 0x00000000000000FF +#define GZVM_REG_ARM_DEMUX_VAL_SHIFT 0 + +/* AArch64 system registers */ +#define GZVM_REG_ARM64_SYSREG (0x0013 << GZVM_REG_ARM_COPROC_SHIFT) +#define GZVM_REG_ARM64_SYSREG_OP0_MASK 0x000000000000c000 +#define GZVM_REG_ARM64_SYSREG_OP0_SHIFT 14 +#define GZVM_REG_ARM64_SYSREG_OP1_MASK 0x0000000000003800 +#define GZVM_REG_ARM64_SYSREG_OP1_SHIFT 11 +#define GZVM_REG_ARM64_SYSREG_CRN_MASK 0x0000000000000780 +#define GZVM_REG_ARM64_SYSREG_CRN_SHIFT 7 +#define GZVM_REG_ARM64_SYSREG_CRM_MASK 0x0000000000000078 +#define GZVM_REG_ARM64_SYSREG_CRM_SHIFT 3 +#define GZVM_REG_ARM64_SYSREG_OP2_MASK 0x0000000000000007 +#define GZVM_REG_ARM64_SYSREG_OP2_SHIFT 0 + +/* Physical Timer EL0 Registers */ +#define GZVM_REG_ARM_PTIMER_CTL ARM64_SYS_REG(3, 3, 14, 2, 1) +#define GZVM_REG_ARM_PTIMER_CVAL ARM64_SYS_REG(3, 3, 14, 2, 2) +#define GZVM_REG_ARM_PTIMER_CNT ARM64_SYS_REG(3, 3, 14, 0, 1) + +/* SVE registers */ +#define GZVM_REG_ARM64_SVE (0x0015 << KVM_REG_ARM_COPROC_SHIFT) + +#endif /* __GZVM_ARCH_H__ */ diff --git a/drivers/soc/mediatek/Kconfig b/drivers/soc/mediatek/Kconfig index a88cf04fc803..01fad024a1c1 100644 --- a/drivers/soc/mediatek/Kconfig +++ b/drivers/soc/mediatek/Kconfig @@ -91,4 +91,6 @@ config MTK_SVS chip process corner, temperatures and other factors. Then DVFS driver could apply SVS bank voltage to PMIC/Buck. +source "drivers/soc/mediatek/virt/geniezone/Kconfig" + endmenu diff --git a/drivers/soc/mediatek/Makefile b/drivers/soc/mediatek/Makefile index 8c0ddacbcde8..e5d7225c1d08 100644 --- a/drivers/soc/mediatek/Makefile +++ b/drivers/soc/mediatek/Makefile @@ -9,3 +9,4 @@ obj-$(CONFIG_MTK_SCPSYS_PM_DOMAINS) += mtk-pm-domains.o obj-$(CONFIG_MTK_MMSYS) += mtk-mmsys.o obj-$(CONFIG_MTK_MMSYS) += mtk-mutex.o obj-$(CONFIG_MTK_SVS) += mtk-svs.o +obj-$(CONFIG_MTK_GZVM) += virt/geniezone/ diff --git a/drivers/soc/mediatek/virt/geniezone/Kconfig b/drivers/soc/mediatek/virt/geniezone/Kconfig new file mode 100644 index 000000000000..6fad3c30f8d9 --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/Kconfig @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MTK_GZVM + tristate "GenieZone Hypervisor driver for guest VM operation" + depends on ARM64 + depends on KVM + help + This driver, gzvm, enables to run guest VMs on MTK GenieZone + hypervisor. It exports kvm-like interfaces for VMM (e.g., crosvm) in + order to operate guest VMs on GenieZone hypervisor. + + GenieZone hypervisor now only supports MediaTek SoC and arm64 + architecture. + + Select M if you want it be built as a module (gzvm.ko). + + If unsure, say N. diff --git a/drivers/soc/mediatek/virt/geniezone/Makefile b/drivers/soc/mediatek/virt/geniezone/Makefile new file mode 100644 index 000000000000..e1dfbb9c568d --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0-only + +gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o + +obj-$(CONFIG_MTK_GZVM) += gzvm.o diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h new file mode 100644 index 000000000000..43f215d4b0da --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h @@ -0,0 +1,103 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#ifndef __GZVM_H__ +#define __GZVM_H__ + +#include +#include +#include +#include "gzvm_hyp.h" + +#define MODULE_NAME "gzvm" +#define GZVM_VCPU_MMAP_SIZE PAGE_SIZE +#define INVALID_VM_ID 0xffff + +/* VM's memory slot descriptor */ +struct gzvm_memslot { + u64 base_gfn; /* begin of guest page frame */ + unsigned long npages; /* number of pages this slot covers */ + unsigned long userspace_addr; /* corresponding userspace va */ + struct vm_area_struct *vma; /* vma related to this userspace addr */ + u32 flags; + u32 slot_id; +}; + +/* pre-declaration for circular reference in struct gzvm */ +struct gzvm_vcpu; + +struct gzvm { + struct gzvm_vcpu *vcpus[GZVM_MAX_VCPUS]; + struct mm_struct *mm; /* userspace tied to this vm */ + struct gzvm_memslot memslot[GZVM_MAX_MEM_REGION]; + struct mutex lock; + struct list_head vm_list; + struct list_head devices; + gzvm_id_t vm_id; + + struct { + spinlock_t lock; + struct list_head items; + struct list_head resampler_list; + struct mutex resampler_lock; + } irqfds; + struct hlist_head irq_ack_notifier_list; + struct srcu_struct irq_srcu; + struct mutex irq_lock; +}; + +struct gzvm_vcpu { + struct gzvm *gzvm; + int vcpuid; + struct mutex lock; + struct gzvm_vcpu_run *run; + struct gzvm_vcpu_hwstate *hwstate; +}; + +/** + * allocate 2 pages for data sharing between driver and gz hypervisor + * |- page 0 -|- page 1 -| + * |gzvm_vcpu_run|......|hwstate|.......| + */ +#define GZVM_VCPU_RUN_MAP_SIZE (PAGE_SIZE * 2) + +long gzvm_dev_ioctl_check_extension(struct gzvm *gzvm, unsigned long args); + +void gzvm_destroy_vcpu(struct gzvm_vcpu *vcpu); +int gzvm_vm_ioctl_create_vcpu(struct gzvm *gzvm, u32 cpuid); +int gzvm_dev_ioctl_create_vm(unsigned long vm_type); + +int gzvm_arm_get_reg(struct gzvm_vcpu *vcpu, const struct gzvm_one_reg *reg); +int gzvm_arm_set_reg(struct gzvm_vcpu *vcpu, const struct gzvm_one_reg *reg); + +int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2, + unsigned long a3, unsigned long a4, unsigned long a5, + unsigned long a6, unsigned long a7, + struct arm_smccc_res *res); + +#define SMC_ENTITY_MTK 59 +#define GZVM_FUNCID_START (0x1000) +#define GZVM_HCALL_ID(func) \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + SMC_ENTITY_MTK, (GZVM_FUNCID_START + (func))) + +#define MT_HVC_GZVM_CREATE_VM GZVM_HCALL_ID(GZVM_FUNC_CREATE_VM) +#define MT_HVC_GZVM_DESTROY_VM GZVM_HCALL_ID(GZVM_FUNC_DESTROY_VM) +#define MT_HVC_GZVM_CREATE_VCPU GZVM_HCALL_ID(GZVM_FUNC_CREATE_VCPU) +#define MT_HVC_GZVM_DESTROY_VCPU GZVM_HCALL_ID(GZVM_FUNC_DESTROY_VCPU) +#define MT_HVC_GZVM_SET_MEMREGION GZVM_HCALL_ID(GZVM_FUNC_SET_MEMREGION) +#define MT_HVC_GZVM_RUN GZVM_HCALL_ID(GZVM_FUNC_RUN) +#define MT_HVC_GZVM_GET_REGS GZVM_HCALL_ID(GZVM_FUNC_GET_REGS) +#define MT_HVC_GZVM_SET_REGS GZVM_HCALL_ID(GZVM_FUNC_SET_REGS) +#define MT_HVC_GZVM_GET_ONE_REG GZVM_HCALL_ID(GZVM_FUNC_GET_ONE_REG) +#define MT_HVC_GZVM_SET_ONE_REG GZVM_HCALL_ID(GZVM_FUNC_SET_ONE_REG) +#define MT_HVC_GZVM_IRQ_LINE GZVM_HCALL_ID(GZVM_FUNC_IRQ_LINE) +#define MT_HVC_GZVM_CREATE_DEVICE GZVM_HCALL_ID(GZVM_FUNC_CREATE_DEVICE) +#define MT_HVC_GZVM_PROBE GZVM_HCALL_ID(GZVM_FUNC_PROBE) +#define MT_HVC_GZVM_ENABLE_CAP GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP) + +int gz_err_to_errno(unsigned long err); + +#endif /* __GZVM_H__ */ diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h b/drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h new file mode 100644 index 000000000000..17dd3de5c285 --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#ifndef __GZ_ERR_H__ +#define __GZ_ERR_H__ + +/** + * @brief Definitions of APIs between GenieZone hypervisor and driver + * + * These are not needed to be visible to uapi + */ + +/* We need GenieZone specific error code in order to map to Linux errno */ +#define NO_ERROR (0) +#define ERR_NO_MEMORY (-5) +#define ERR_NOT_SUPPORTED (-24) +#define ERR_NOT_IMPLEMENTED (-27) +#define ERR_FAULT (-40) + +static inline unsigned int +assemble_vm_vcpu_tuple(gzvm_id_t vmid, gzvm_vcpu_id_t vcpuid) +{ + return ((unsigned int)vmid << 16 | vcpuid); +} + +static inline gzvm_id_t get_vmid_from_tuple(unsigned int tuple) +{ + return (gzvm_id_t)(tuple >> 16); +} + +static inline gzvm_vcpu_id_t get_vcpuid_from_tuple(unsigned int tuple) +{ + return (gzvm_vcpu_id_t) (tuple & 0xffff); +} + +static inline void +disassemble_vm_vcpu_tuple(unsigned int tuple, gzvm_id_t *vmid, + gzvm_vcpu_id_t *vcpuid) +{ + *vmid = get_vmid_from_tuple(tuple); + *vcpuid = get_vcpuid_from_tuple(tuple); +} + +/* + * The following data structures are for data transferring between driver and + * hypervisor + */ + +/* align hypervisor definitions */ +#define GZVM_MAX_VCPUS 8 +#define GZVM_MAX_MEM_REGION 10 + +/* identical to ffa memory constituent */ +struct mem_region_addr_range { + /* The base IPA of the constituent memory region, aligned to 4 kiB */ + __u64 address; + /* The number of 4 kiB pages in the constituent memory region. */ + __u32 pg_cnt; + __u32 reserved; +}; + +struct gzvm_memory_region_ranges { + __u32 slot; + __u32 constituent_cnt; + __u64 total_pages; + __u64 gpa; + struct mem_region_addr_range constituents[]; +}; + +#endif /* __GZ_ERR_H__ */ diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_main.c b/drivers/soc/mediatek/virt/geniezone/gzvm_main.c new file mode 100644 index 000000000000..1fabe4a579da --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_main.c @@ -0,0 +1,233 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "gzvm.h" + +static void (*invoke_gzvm_fn)(unsigned long, unsigned long, unsigned long, + unsigned long, unsigned long, unsigned long, + unsigned long, unsigned long, + struct arm_smccc_res *); + +static void gzvm_hvc(unsigned long a0, unsigned long a1, unsigned long a2, + unsigned long a3, unsigned long a4, unsigned long a5, + unsigned long a6, unsigned long a7, + struct arm_smccc_res *res) +{ + arm_smccc_hvc(a0, a1, a2, a3, a4, a5, a6, a7, res); +} + +static void gzvm_smc(unsigned long a0, unsigned long a1, unsigned long a2, + unsigned long a3, unsigned long a4, unsigned long a5, + unsigned long a6, unsigned long a7, + struct arm_smccc_res *res) +{ + arm_smccc_smc(a0, a1, a2, a3, a4, a5, a6, a7, res); +} + +static int gzvm_probe_conduit(void) +{ + struct arm_smccc_res res; + + arm_smccc_hvc(MT_HVC_GZVM_PROBE, 0, 0, 0, 0, 0, 0, 0, &res); + if (res.a0 == 0) { + invoke_gzvm_fn = gzvm_hvc; + return 0; + } + + arm_smccc_smc(MT_HVC_GZVM_PROBE, 0, 0, 0, 0, 0, 0, 0, &res); + if (res.a0 == 0) { + invoke_gzvm_fn = gzvm_smc; + return 0; + } + + return -ENXIO; +} + +/** + * @brief geniezone hypercall wrapper + * @return int geniezone's return value will be converted to Linux errno + */ +int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2, + unsigned long a3, unsigned long a4, unsigned long a5, + unsigned long a6, unsigned long a7, + struct arm_smccc_res *res) +{ + invoke_gzvm_fn(a0, a1, a2, a3, a4, a5, a6, a7, res); + return gz_err_to_errno(res->a0); +} + +/** + * @brief Convert geniezone return value to standard errno + * + * @param err return value from geniezone hypercall (a0) + * @return int errno + */ +int gz_err_to_errno(unsigned long err) +{ + int gz_err = (int) err; + + switch (gz_err) { + case 0: + return 0; + case ERR_NO_MEMORY: + return -ENOMEM; + case ERR_NOT_SUPPORTED: + return -EOPNOTSUPP; + case ERR_NOT_IMPLEMENTED: + return -EOPNOTSUPP; + case ERR_FAULT: + return -EFAULT; + default: + return -EINVAL; + } + + return -EINVAL; +} + +static int gzvm_cap_arm_vm_ipa_size(void __user *argp) +{ + u64 value = CONFIG_ARM64_PA_BITS; + + if (copy_to_user(argp, &value, sizeof(u64))) + return -EFAULT; + + return 0; +} + +/** + * @brief Check if given capability is support or not + * + * @param args in/out u64 pointer from userspace + * @retval 0: support, no error + * @retval -EOPNOTSUPP: not support + * @retval -EFAULT: failed to get data from userspace + */ +long gzvm_dev_ioctl_check_extension(struct gzvm *gzvm, unsigned long args) +{ + int ret = -EOPNOTSUPP; + __u64 cap, success = 1; + void __user *argp = (void __user *) args; + + if (copy_from_user(&cap, argp, sizeof(uint64_t))) + return -EFAULT; + + switch (cap) { + case GZVM_CAP_ARM_PROTECTED_VM: + if (copy_to_user(argp, &success, sizeof(uint64_t))) + return -EFAULT; + ret = 0; + break; + case GZVM_CAP_ARM_VM_IPA_SIZE: + ret = gzvm_cap_arm_vm_ipa_size(argp); + break; + default: + ret = -EOPNOTSUPP; + } + + return ret; +} + +static long gzvm_dev_ioctl(struct file *filp, unsigned int cmd, + unsigned long user_args) +{ + long ret = -ENOTTY; + + switch (cmd) { + case GZVM_CREATE_VM: + ret = gzvm_dev_ioctl_create_vm(user_args); + break; + case GZVM_CHECK_EXTENSION: + if (!user_args) + return -EINVAL; + ret = gzvm_dev_ioctl_check_extension(NULL, user_args); + break; + default: + ret = -ENOTTY; + } + + return ret; +} + +static const struct file_operations gzvm_chardev_ops = { + .unlocked_ioctl = gzvm_dev_ioctl, + .llseek = noop_llseek, +}; + +static struct miscdevice gzvm_dev = { + .minor = MISC_DYNAMIC_MINOR, + .name = MODULE_NAME, + .fops = &gzvm_chardev_ops, +}; + +static int gzvm_drv_probe(struct platform_device *pdev) +{ + if (!of_device_is_available(dev_of_node(&pdev->dev))) { + dev_info(&pdev->dev, "GenieZone hypervisor is not available\n"); + return -ENODEV; + } + + if (gzvm_probe_conduit() != 0) { + dev_err(&pdev->dev, "Not found available conduit\n"); + return -ENODEV; + } + + return 0; +} + +static int gzvm_drv_remove(struct platform_device *pdev) +{ + misc_deregister(&gzvm_dev); + + return 0; +} + +static const struct of_device_id gzvm_of_match[] = { + { .compatible = "mediatek,gzvm", }, + {}, +}; +MODULE_DEVICE_TABLE(of, gzvm_of_match); + +static struct platform_driver gzvm_driver = { + .probe = gzvm_drv_probe, + .remove = gzvm_drv_remove, + .driver = { + .name = MODULE_NAME, + .owner = THIS_MODULE, + .of_match_table = gzvm_of_match, + }, +}; + +static int __init gzvm_init(void) +{ + int ret = 0; + + ret = platform_driver_register(&gzvm_driver); + if (ret) + pr_err("Failed to register gzvm driver.\n"); + + return ret; +} + +static void __exit gzvm_exit(void) +{ + platform_driver_unregister(&gzvm_driver); +} + +module_init(gzvm_init); +module_exit(gzvm_exit); + +MODULE_AUTHOR("MediaTek"); +MODULE_DESCRIPTION("GenieZone interface for VMM"); +MODULE_LICENSE("GPL"); diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c new file mode 100644 index 000000000000..726db866dfcf --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c @@ -0,0 +1,266 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#include +#include +#include +#include +#include +#include "gzvm.h" + +static int gzvm_vcpu_update_one_reg_hyp(struct gzvm_vcpu *vcpu, __u64 reg_id, + bool is_write, __u64 *data) +{ + struct arm_smccc_res res; + unsigned long a1; + int ret; + + a1 = assemble_vm_vcpu_tuple(vcpu->gzvm->vm_id, vcpu->vcpuid); + if (!is_write) { + ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_GET_ONE_REG, + a1, reg_id, 0, 0, 0, 0, 0, &res); + if (ret == 0) + *data = res.a1; + } else { + ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_SET_ONE_REG, + a1, reg_id, *data, 0, 0, 0, 0, &res); + } + + return ret; +} + +static long gzvm_vcpu_update_one_reg(struct gzvm_vcpu *vcpu, void * __user argp, + bool is_write) +{ + long ret; + __u64 reg_size, data = 0; + struct gzvm_one_reg reg; + void __user *reg_addr; + + if (copy_from_user(®, argp, sizeof(reg))) + return -EFAULT; + reg_addr = (void __user *)reg.addr; + + /* reg id follows KVM's encoding */ + switch (reg.id & GZVM_REG_ARM_COPROC_MASK) { + case GZVM_REG_ARM_CORE: + break; + default: + return -EOPNOTSUPP; + } + + reg_size = 1 << ((reg.id & GZVM_REG_SIZE_MASK) >> GZVM_REG_SIZE_SHIFT); + if (is_write) { + if (copy_from_user(&data, reg_addr, reg_size)) + return -EFAULT; + } + + ret = gzvm_vcpu_update_one_reg_hyp(vcpu, reg.id, is_write, &data); + + if (!is_write && ret == 0) { + if (copy_to_user(reg_addr, &data, reg_size)) + return -EFAULT; + } + + return ret; +} + +/** + * @brief Handle vcpu run ioctl, entry point to guest and exit point from guest + * + * @param filp + * @param argp pointer to struct gzvm_vcpu_run in userspace + * @return long + */ +static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * __user argp) +{ + unsigned long id_tuple; + struct arm_smccc_res res; + bool need_userspace = false; + + if (copy_from_user(vcpu->run, argp, sizeof(struct gzvm_vcpu_run))) + return -EFAULT; + + if (vcpu->run->immediate_exit == 1) + return -EINTR; + + id_tuple = assemble_vm_vcpu_tuple(vcpu->gzvm->vm_id, vcpu->vcpuid); + do { + gzvm_hypcall_wrapper(MT_HVC_GZVM_RUN, id_tuple, 0, 0, 0, 0, 0, + 0, &res); + switch (res.a1) { + case GZVM_EXIT_MMIO: + need_userspace = true; + break; + /* + * geniezone's responsibility to fill corresponding data + * structure + */ + case GZVM_EXIT_HVC: + case GZVM_EXIT_EXCEPTION: + case GZVM_EXIT_DEBUG: + case GZVM_EXIT_FAIL_ENTRY: + case GZVM_EXIT_INTERNAL_ERROR: + case GZVM_EXIT_SYSTEM_EVENT: + case GZVM_EXIT_SHUTDOWN: + need_userspace = true; + break; + case GZVM_EXIT_IRQ: + break; + case GZVM_EXIT_UNKNOWN: + default: + pr_err("vcpu unknown exit\n"); + need_userspace = true; + goto out; + } + } while (!need_userspace); + +out: + if (copy_to_user(argp, vcpu->run, sizeof(struct gzvm_vcpu_run))) + return -EFAULT; + return 0; +} + +static long gzvm_vcpu_ioctl(struct file *filp, unsigned int ioctl, + unsigned long arg) +{ + int ret = -ENOTTY; + void __user *argp = (void __user *)arg; + struct gzvm_vcpu *vcpu = filp->private_data; + + switch (ioctl) { + case GZVM_RUN: + ret = gzvm_vcpu_run(vcpu, argp); + break; + case GZVM_GET_ONE_REG: + ret = gzvm_vcpu_update_one_reg(vcpu, argp, false /*is_write*/); + break; + case GZVM_SET_ONE_REG: + ret = gzvm_vcpu_update_one_reg(vcpu, argp, true /*is_write*/); + break; + default: + ret = -ENOTTY; + break; + } + + return ret; +} + +static const struct file_operations gzvm_vcpu_fops = { + .unlocked_ioctl = gzvm_vcpu_ioctl, + .llseek = noop_llseek, +}; + +static int gzvm_destroy_vcpu_hyp(gzvm_id_t vm_id, int vcpuid) +{ + struct arm_smccc_res res; + unsigned long a1; + + a1 = assemble_vm_vcpu_tuple(vm_id, vcpuid); + gzvm_hypcall_wrapper(MT_HVC_GZVM_DESTROY_VCPU, a1, 0, 0, 0, 0, 0, 0, + &res); + + return 0; +} + +/** + * @brief call smc to gz hypervisor to create vcpu + * + * @param run virtual address of vcpu->run + * @return int + */ +static int gzvm_create_vcpu_hyp(gzvm_id_t vm_id, int vcpuid, void *run) +{ + struct arm_smccc_res res; + unsigned long a1, a2; + int ret; + + a1 = assemble_vm_vcpu_tuple(vm_id, vcpuid); + a2 = (__u64)virt_to_phys(run); + ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_VCPU, a1, a2, 0, 0, 0, 0, + 0, &res); + + return ret; +} + +/* Caller must hold the vm lock */ +void gzvm_destroy_vcpu(struct gzvm_vcpu *vcpu) +{ + if (!vcpu) + return; + + gzvm_destroy_vcpu_hyp(vcpu->gzvm->vm_id, vcpu->vcpuid); + /* clean guest's data */ + memset(vcpu->run, 0, GZVM_VCPU_RUN_MAP_SIZE); + free_pages_exact(vcpu->run, GZVM_VCPU_RUN_MAP_SIZE); + kfree(vcpu); +} + +#define ITOA_MAX_LEN 12 /* Maximum size needed for holding an integer. */ +/** + * @brief Allocates an inode for the vcpu. + */ +static int create_vcpu_fd(struct gzvm_vcpu *vcpu) +{ + /* sizeof("gzvm-vcpu:") + max(strlen(itoa(vcpuid))) + null */ + char name[10 + ITOA_MAX_LEN + 1]; + + snprintf(name, sizeof(name), "gzvm-vcpu:%d", vcpu->vcpuid); + return anon_inode_getfd(name, &gzvm_vcpu_fops, vcpu, O_RDWR | O_CLOEXEC); +} + +/** + * @brief GZVM_CREATE_VCPU + * + * @param cpuid = arg + * @return fd of vcpu, negative errno if error occurs + */ +int gzvm_vm_ioctl_create_vcpu(struct gzvm *gzvm, u32 cpuid) +{ + struct gzvm_vcpu *vcpu; + int ret; + + if (cpuid >= GZVM_MAX_VCPUS) + return -EINVAL; + + vcpu = kzalloc(sizeof(*vcpu), GFP_KERNEL); + if (!vcpu) + return -ENOMEM; + + BUILD_BUG_ON((sizeof(*vcpu->run)) > PAGE_SIZE); + BUILD_BUG_ON(sizeof(struct gzvm_vcpu_hwstate) > PAGE_SIZE); + /** + * allocate 2 pages for data sharing between driver and gz hypervisor + * |- page 0 -|- page 1 -| + * |gzvm_vcpu_run|......|hwstate|.......| + */ + vcpu->run = alloc_pages_exact(GZVM_VCPU_RUN_MAP_SIZE, + GFP_KERNEL_ACCOUNT | __GFP_ZERO); + if (!vcpu->run) { + ret = -ENOMEM; + goto free_vcpu; + } + vcpu->hwstate = (void *)vcpu->run + PAGE_SIZE; + vcpu->vcpuid = cpuid; + vcpu->gzvm = gzvm; + mutex_init(&vcpu->lock); + + ret = gzvm_create_vcpu_hyp(gzvm->vm_id, vcpu->vcpuid, vcpu->run); + if (ret < 0) + goto free_vcpu_run; + + ret = create_vcpu_fd(vcpu); + if (ret < 0) + goto free_vcpu_run; + gzvm->vcpus[cpuid] = vcpu; + + return ret; + +free_vcpu_run: + free_pages_exact(vcpu->run, GZVM_VCPU_RUN_MAP_SIZE); +free_vcpu: + kfree(vcpu); + return ret; +} diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c new file mode 100644 index 000000000000..df4ccdc3b7f0 --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c @@ -0,0 +1,444 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "gzvm.h" + +static DEFINE_MUTEX(gzvm_list_lock); +static LIST_HEAD(gzvm_list); + + +/** + * @brief Translate gfn (guest ipa) to pfn (host pa), result is in @pfn + * + * Leverage KVM's `gfn_to_pfn_memslot`. Because `gfn_to_pfn_memslot` needs + * kvm_memory_slot as parameter, this function populates necessary fileds + * for calling `gfn_to_pfn_memslot`. + * + * @retval 0 succeed + * @retval -EFAULT failed to convert + */ +int gzvm_gfn_to_pfn_memslot(struct gzvm_memslot *memslot, u64 gfn, u64 *pfn) +{ + hfn_t __pfn; + struct kvm_memory_slot kvm_slot = {0}; + + kvm_slot.base_gfn = memslot->base_gfn; + kvm_slot.npages = memslot->npages; + kvm_slot.dirty_bitmap = NULL; + kvm_slot.userspace_addr = memslot->userspace_addr; + kvm_slot.flags = memslot->flags; + kvm_slot.id = memslot->slot_id; + kvm_slot.as_id = 0; + + __pfn = gfn_to_pfn_memslot(&kvm_slot, gfn); + if (is_error_noslot_pfn(__pfn)) { + *pfn = 0; + return -EFAULT; + } + + *pfn = __pfn; + return 0; +} + +/** + * @brief Populate pa to buffer until full + * + * @return int how much pages we've fill in, negative if error + */ +static int fill_constituents(struct mem_region_addr_range *consti, + int *consti_cnt, int max_nr_consti, gfn_t gfn, + u32 total_pages, struct gzvm_memslot *slot) +{ + int i, nr_pages; + hfn_t pfn, prev_pfn; + gfn_t gfn_end; + + if (unlikely(total_pages == 0)) + return -EINVAL; + gfn_end = gfn + total_pages; + + /* entry 0 */ + if (gzvm_gfn_to_pfn_memslot(slot, gfn, &pfn) != 0) + return -EFAULT; + consti[0].address = PFN_PHYS(pfn); + consti[0].pg_cnt = 1; + gfn++; + prev_pfn = pfn; + i = 0; + nr_pages = 1; + while (i < max_nr_consti && gfn < gfn_end) { + if (gzvm_gfn_to_pfn_memslot(slot, gfn, &pfn) != 0) + return -EFAULT; + if (pfn == (prev_pfn + 1)) { + consti[i].pg_cnt++; + } else { + i++; + if (i >= max_nr_consti) + break; + consti[i].address = PFN_PHYS(pfn); + consti[i].pg_cnt = 1; + } + prev_pfn = pfn; + gfn++; + nr_pages++; + } + if (i == max_nr_consti) + *consti_cnt = i; + else + *consti_cnt = (i + 1); + + return nr_pages; +} + +/** + * @brief Register memory region to GZ + * + * @param gzvm + * @param memslot + * @return int + */ +static int +register_memslot_addr_range(struct gzvm *gzvm, struct gzvm_memslot *memslot) +{ + struct gzvm_memory_region_ranges *region; + u32 buf_size; + int max_nr_consti, remain_pages; + gfn_t gfn, gfn_end; + + buf_size = PAGE_SIZE * 2; + region = alloc_pages_exact(buf_size, GFP_KERNEL); + if (!region) + return -ENOMEM; + max_nr_consti = (buf_size - sizeof(*region)) / + sizeof(struct mem_region_addr_range); + + region->slot = memslot->slot_id; + remain_pages = memslot->npages; + gfn = memslot->base_gfn; + gfn_end = gfn + remain_pages; + while (gfn < gfn_end) { + struct arm_smccc_res res; + int nr_pages; + + nr_pages = fill_constituents(region->constituents, + ®ion->constituent_cnt, + max_nr_consti, gfn, + remain_pages, memslot); + region->gpa = PFN_PHYS(gfn); + region->total_pages = nr_pages; + + remain_pages -= nr_pages; + gfn += nr_pages; + + gzvm_hypcall_wrapper(MT_HVC_GZVM_SET_MEMREGION, gzvm->vm_id, + buf_size, virt_to_phys(region), 0, 0, 0, 0, &res); + + if (res.a0 != 0) { + pr_err("Failed to register memregion to hypervisor\n"); + free_pages_exact(region, buf_size); + return -EFAULT; + } + } + free_pages_exact(region, buf_size); + return 0; +} + +/** + * @brief Set memory region of guest + * + * @param gzvm struct gzvm + * @param mem struct gzvm_userspace_memory_region: input from user + * @retval -EXIO memslot is out-of-range + * @retval -EFAULT cannot find corresponding vma + * @retval -EINVAL region size and vma size does not match + */ +static int gzvm_vm_ioctl_set_memory_region(struct gzvm *gzvm, + struct gzvm_userspace_memory_region *mem) +{ + struct vm_area_struct *vma; + struct gzvm_memslot *memslot; + unsigned long size; + __u32 slot; + + slot = mem->slot; + if (slot >= GZVM_MAX_MEM_REGION) + return -ENXIO; + memslot = &gzvm->memslot[slot]; + + vma = vma_lookup(gzvm->mm, mem->userspace_addr); + if (!vma) + return -EFAULT; + + size = vma->vm_end - vma->vm_start; + if (size != mem->memory_size) + return -EINVAL; + + memslot->base_gfn = __phys_to_pfn(mem->guest_phys_addr); + memslot->npages = size >> PAGE_SHIFT; + memslot->userspace_addr = mem->userspace_addr; + memslot->vma = vma; + memslot->flags = mem->flags; + memslot->slot_id = mem->slot; + return register_memslot_addr_range(gzvm, memslot); +} + +static int gzvm_vm_enable_cap_hyp(struct gzvm *gzvm, + struct gzvm_enable_cap *cap, + struct arm_smccc_res *res) +{ + int ret; + + ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_ENABLE_CAP, gzvm->vm_id, + cap->cap, cap->args[0], cap->args[1], + cap->args[2], cap->args[3], cap->args[4], + res); + return ret; +} + +/** + * @brief Get pvmfw size from hypervisor, return in x1, and return to userspace + * in args[1]. + * @retval 0 succeed + * @retval -EINVAL hypervisor return invalid results + * @retval -EFAULT fail to copy back to userspace buffer + */ +static int gzvm_vm_ioctl_get_pvmfw_size(struct gzvm *gzvm, + struct gzvm_enable_cap *cap, + void __user *argp) +{ + struct arm_smccc_res res = {0}; + + if (gzvm_vm_enable_cap_hyp(gzvm, cap, &res) != 0) + return -EINVAL; + + cap->args[1] = res.a1; + if (copy_to_user(argp, cap, sizeof(*cap))) + return -EFAULT; + + return 0; +} + +/** + * @brief Proceed GZVM_CAP_ARM_PROTECTED_VM's subcommands + * @retval 0 succeed + * @retval -EINVAL invalid subcommand or arguments + */ +static int gzvm_vm_ioctl_cap_pvm(struct gzvm *gzvm, struct gzvm_enable_cap *cap, + void __user *argp) +{ + int ret = -EINVAL; + struct arm_smccc_res res = {0}; + + switch (cap->args[0]) { + case GZVM_CAP_ARM_PVM_SET_PVMFW_IPA: + ret = gzvm_vm_enable_cap_hyp(gzvm, cap, &res); + break; + case GZVM_CAP_ARM_PVM_GET_PVMFW_SIZE: + ret = gzvm_vm_ioctl_get_pvmfw_size(gzvm, cap, argp); + break; + default: + ret = -EINVAL; + break; + } + + return ret; +} + +static int gzvm_vm_ioctl_enable_cap(struct gzvm *gzvm, + struct gzvm_enable_cap *cap, + void __user *argp) +{ + int ret = -EINVAL; + + switch (cap->cap) { + case GZVM_CAP_ARM_PROTECTED_VM: + ret = gzvm_vm_ioctl_cap_pvm(gzvm, cap, argp); + break; + default: + ret = -EINVAL; + break; + } + + return ret; +} + +/** + * @brief ioctl handler of VM FD + */ +static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl, + unsigned long arg) +{ + long ret = -ENOTTY; + void __user *argp = (void __user *)arg; + struct gzvm *gzvm = filp->private_data; + + switch (ioctl) { + case GZVM_CHECK_EXTENSION: + ret = gzvm_dev_ioctl_check_extension(gzvm, arg); + break; + case GZVM_CREATE_VCPU: + ret = gzvm_vm_ioctl_create_vcpu(gzvm, arg); + break; + case GZVM_SET_USER_MEMORY_REGION: { + struct gzvm_userspace_memory_region userspace_mem; + + ret = -EFAULT; + if (copy_from_user(&userspace_mem, argp, + sizeof(userspace_mem))) + goto out; + ret = gzvm_vm_ioctl_set_memory_region(gzvm, &userspace_mem); + break; + } + case GZVM_ENABLE_CAP: { + struct gzvm_enable_cap cap; + + ret = -EFAULT; + if (copy_from_user(&cap, argp, sizeof(cap))) + goto out; + + ret = gzvm_vm_ioctl_enable_cap(gzvm, &cap, argp); + break; + } + default: + ret = -ENOTTY; + } +out: + return ret; +} + +/** + * @brief Destroy all vcpus + * + * @param gzvm vm struct that owns the vcpus + * Caller has to hold the vm lock + */ +static void gzvm_destroy_vcpus(struct gzvm *gzvm) +{ + int i; + + for (i = 0; i < GZVM_MAX_VCPUS; i++) { + gzvm_destroy_vcpu(gzvm->vcpus[i]); + gzvm->vcpus[i] = NULL; + } +} + +static int gzvm_destroy_vm_hyp(gzvm_id_t vm_id) +{ + struct arm_smccc_res res; + + gzvm_hypcall_wrapper(MT_HVC_GZVM_DESTROY_VM, vm_id, 0, 0, 0, 0, 0, 0, + &res); + + return 0; +} + +static void gzvm_destroy_vm(struct gzvm *gzvm) +{ + pr_info("VM-%u is going to be destroyed\n", gzvm->vm_id); + + mutex_lock(&gzvm->lock); + + gzvm_destroy_vcpus(gzvm); + gzvm_destroy_vm_hyp(gzvm->vm_id); + + mutex_lock(&gzvm_list_lock); + list_del(&gzvm->vm_list); + mutex_unlock(&gzvm_list_lock); + + mutex_unlock(&gzvm->lock); + + kfree(gzvm); +} + +static int gzvm_vm_release(struct inode *inode, struct file *filp) +{ + struct gzvm *gzvm = filp->private_data; + + gzvm_destroy_vm(gzvm); + return 0; +} + +static const struct file_operations gzvm_vm_fops = { + .release = gzvm_vm_release, + .unlocked_ioctl = gzvm_vm_ioctl, + .llseek = noop_llseek, +}; + +static int gzvm_create_vm_hyp(void) +{ + struct arm_smccc_res res; + + gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_VM, 0, 0, 0, 0, 0, 0, 0, &res); + + if (res.a0 != 0) + return -EFAULT; + return res.a1; +} + +static struct gzvm *gzvm_create_vm(unsigned long vm_type) +{ + int ret; + struct gzvm *gzvm; + + gzvm = kzalloc(sizeof(struct gzvm), GFP_KERNEL); + if (!gzvm) + return ERR_PTR(-ENOMEM); + + ret = gzvm_create_vm_hyp(); + if (ret < 0) + goto err; + + gzvm->vm_id = ret; + gzvm->mm = current->mm; + mutex_init(&gzvm->lock); + INIT_LIST_HEAD(&gzvm->devices); + mutex_init(&gzvm->irq_lock); + pr_info("VM-%u is created\n", gzvm->vm_id); + + mutex_lock(&gzvm_list_lock); + list_add(&gzvm->vm_list, &gzvm_list); + mutex_unlock(&gzvm_list_lock); + + return gzvm; + +err: + kfree(gzvm); + return ERR_PTR(ret); +} + +/** + * @brief create vm fd + * + * @param vm_type + * @return int fd of vm, negative if error + */ +int gzvm_dev_ioctl_create_vm(unsigned long vm_type) +{ + struct gzvm *gzvm; + int ret; + + gzvm = gzvm_create_vm(vm_type); + if (IS_ERR(gzvm)) { + ret = PTR_ERR(gzvm); + goto error; + } + + ret = anon_inode_getfd("gzvm-vm", &gzvm_vm_fops, gzvm, + O_RDWR | O_CLOEXEC); + if (ret < 0) + goto error; + +error: + return ret; +} diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h new file mode 100644 index 000000000000..aa97438bab8c --- /dev/null +++ b/include/uapi/linux/gzvm_common.h @@ -0,0 +1,217 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#ifndef __GZVM_COMMON_H__ +#define __GZVM_COMMON_H__ + +#include +#include +#include + +/* gzvm only supports aarch64 platform */ +#if defined(__aarch64__) +#include +#endif + +/** + * @brief This file declares common data structure shared between userspace, + * kernel space, and GZ. + */ +enum { + GZVM_FUNC_CREATE_VM = 0, + GZVM_FUNC_DESTROY_VM, + GZVM_FUNC_CREATE_VCPU, + GZVM_FUNC_DESTROY_VCPU, + GZVM_FUNC_SET_MEMREGION, + GZVM_FUNC_RUN, + GZVM_FUNC_GET_REGS, + GZVM_FUNC_SET_REGS, + GZVM_FUNC_GET_ONE_REG, + GZVM_FUNC_SET_ONE_REG, + GZVM_FUNC_IRQ_LINE, + GZVM_FUNC_CREATE_DEVICE, + GZVM_FUNC_PROBE, + GZVM_FUNC_ENABLE_CAP, + NR_GZVM_FUNC +}; + +typedef __u16 gzvm_id_t; +typedef __u16 gzvm_vcpu_id_t; + +/* VM exit reason */ +enum { + GZVM_EXIT_UNKNOWN = 0x92920000, + GZVM_EXIT_MMIO, + GZVM_EXIT_HVC, + GZVM_EXIT_IRQ, + GZVM_EXIT_EXCEPTION, + GZVM_EXIT_DEBUG, + GZVM_EXIT_FAIL_ENTRY, + GZVM_EXIT_INTERNAL_ERROR, + GZVM_EXIT_SYSTEM_EVENT, + GZVM_EXIT_SHUTDOWN, +}; + +/** + * @brief same purpose as kvm_run, this struct is shared between userspace, + * kernel and GZ + * Note: keep identical layout between the 3 modules + */ +struct gzvm_vcpu_run { + /* to userspace */ + __u32 exit_reason; + __u8 immediate_exit; + __u8 padding1[3]; + /* union structure of collection of guest exit reason */ + union { + /* GZVM_EXIT_MMIO */ + struct { + __u64 phys_addr; /* from FAR_EL2 */ + __u8 data[8]; + __u64 size; /* from ESR_EL2 as */ + __u32 reg_nr; /* from ESR_EL2 */ + __u8 is_write; /* from ESR_EL2 */ + } mmio; + /* GZVM_EXIT_FAIL_ENTRY */ + struct { + __u64 hardware_entry_failure_reason; + __u32 cpu; + } fail_entry; + /* GZVM_EXIT_EXCEPTION */ + struct { + __u32 exception; + __u32 error_code; + } exception; + /* GZVM_EXIT_HVC */ + struct { + __u64 args[8]; /* in-out */ + } hvc; + /* GZVM_EXIT_INTERNAL_ERROR */ + struct { + __u32 suberror; + __u32 ndata; + __u64 data[16]; + } internal; + /* GZVM_EXIT_SYSTEM_EVENT */ + struct { +#define GZVM_SYSTEM_EVENT_SHUTDOWN 1 +#define GZVM_SYSTEM_EVENT_RESET 2 +#define GZVM_SYSTEM_EVENT_CRASH 3 +#define GZVM_SYSTEM_EVENT_WAKEUP 4 +#define GZVM_SYSTEM_EVENT_SUSPEND 5 +#define GZVM_SYSTEM_EVENT_SEV_TERM 6 +#define GZVM_SYSTEM_EVENT_S2IDLE 7 + __u32 type; + __u32 ndata; + __u64 data[16]; + } system_event; + /* Fix the size of the union. */ + char padding[256]; + }; +}; + +#define GIC_V3_NR_LRS 16 + +struct gzvm_vcpu_hwstate { + __u32 nr_lrs; + __u64 lr[GIC_V3_NR_LRS]; +}; + +/* GZVM ioctls */ +#define GZVM_IOC_MAGIC 0x92 /* gz */ + +/* + * ioctls for /dev/gzvm fds: + */ +#define GZVM_GET_API_VERSION _IO(GZVM_IOC_MAGIC, 0x00) +#define GZVM_CREATE_VM _IO(GZVM_IOC_MAGIC, 0x01) + +#define GZVM_CAP_ARM_VM_IPA_SIZE 165 +#define GZVM_CAP_ARM_PROTECTED_VM 0xffbadab1 + +#define GZVM_CHECK_EXTENSION _IO(GZVM_IOC_MAGIC, 0x03) + +/* + * ioctls for VM fds + */ + +/* for GZVM_SET_MEMORY_REGION */ +struct gzvm_memory_region { + __u32 slot; + __u32 flags; + __u64 guest_phys_addr; + __u64 memory_size; /* bytes */ +}; +#define GZVM_SET_MEMORY_REGION _IOW(GZVM_IOC_MAGIC, 0x40, \ + struct gzvm_memory_region) +/* + * GZVM_CREATE_VCPU receives as a parameter the vcpu slot, and returns + * a vcpu fd. + */ +#define GZVM_CREATE_VCPU _IO(GZVM_IOC_MAGIC, 0x41) + +/* for GZVM_SET_USER_MEMORY_REGION */ +struct gzvm_userspace_memory_region { + __u32 slot; + __u32 flags; + __u64 guest_phys_addr; + __u64 memory_size; /* bytes */ + __u64 userspace_addr; /* start of the userspace allocated memory */ +}; +#define GZVM_SET_USER_MEMORY_REGION _IOW(GZVM_IOC_MAGIC, 0x46, \ + struct gzvm_userspace_memory_region) + +/* for GZVM_IRQ_LINE */ +/* GZVM_IRQ_LINE irq field index values */ +#define GZVM_IRQ_VCPU2_SHIFT 28 +#define GZVM_IRQ_VCPU2_MASK 0xf +#define GZVM_IRQ_TYPE_SHIFT 24 +#define GZVM_IRQ_TYPE_MASK 0xf +#define GZVM_IRQ_VCPU_SHIFT 16 +#define GZVM_IRQ_VCPU_MASK 0xff +#define GZVM_IRQ_NUM_SHIFT 0 +#define GZVM_IRQ_NUM_MASK 0xffff + +/* irq_type field */ +#define GZVM_IRQ_TYPE_CPU 0 +#define GZVM_IRQ_TYPE_SPI 1 +#define GZVM_IRQ_TYPE_PPI 2 + +/* out-of-kernel GIC cpu interrupt injection irq_number field */ +#define GZVM_IRQ_CPU_IRQ 0 +#define GZVM_IRQ_CPU_FIQ 1 + +/* + * ioctls for vcpu fds + */ +#define GZVM_RUN _IO(GZVM_IOC_MAGIC, 0x80) + +/* sub-commands put in args[0] for GZVM_CAP_ARM_PROTECTED_VM */ +#define GZVM_CAP_ARM_PVM_SET_PVMFW_IPA 0 +#define GZVM_CAP_ARM_PVM_GET_PVMFW_SIZE 1 + +/* for GZVM_ENABLE_CAP */ +struct gzvm_enable_cap { + /* in */ + __u64 cap; + /* we have total 5 (8 - 3) registers can be used for additional args */ + __u64 args[5]; +}; +#define GZVM_ENABLE_CAP _IOW(GZVM_IOC_MAGIC, 0xa3, \ + struct gzvm_enable_cap) + +struct gzvm_one_reg { + __u64 id; + __u64 addr; +}; +#define GZVM_GET_ONE_REG _IOW(GZVM_IOC_MAGIC, 0xab, \ + struct gzvm_one_reg) +#define GZVM_SET_ONE_REG _IOW(GZVM_IOC_MAGIC, 0xac, \ + struct gzvm_one_reg) + +#define GZVM_REG_ARCH_MASK 0xff00000000000000ULL +#define GZVM_REG_GENERIC 0x0000000000000000ULL + +#endif /* __GZVM_COMMON_H__ */ From patchwork Thu Apr 13 09:07:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi-De Wu X-Patchwork-Id: 672947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C32B2C77B6F for ; Thu, 13 Apr 2023 09:08:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230265AbjDMJIm (ORCPT ); Thu, 13 Apr 2023 05:08:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230081AbjDMJIW (ORCPT ); Thu, 13 Apr 2023 05:08:22 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA0C81734; Thu, 13 Apr 2023 02:08:12 -0700 (PDT) X-UUID: b317ae92d9da11edb6b9f13eb10bd0fe-20230413 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=rYwysHzjyxeU89yIXfyyaxHSc4b5rh92IDLtel9ASgo=; b=LAkbKbrnLSV3Br6/csuQZfRHG/7B2Fe5fC/nKIcbZtMzxQDLbkzHi1gqS3jBScQEx0oqnzz5NTTtErWXBSsePLWwFsfjjdZFfu6xuYcHfok0dq7NCDspY58sdwtn4MdwrRzpadgXJdvEqZiknMrzbuLW9WEDOV2FFszIGgFqW5c=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.22, REQID:03c0bd10-3f8e-4d0b-a4e4-e80269e0b957, IP:0, U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:70 X-CID-INFO: VERSION:1.1.22, REQID:03c0bd10-3f8e-4d0b-a4e4-e80269e0b957, IP:0, URL :0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Spam_GS981B3D,ACTI ON:quarantine,TS:70 X-CID-META: VersionHash:120426c, CLOUDID:3bcfef83-cd9c-45f5-8134-710979e3df0e, B ulkID:2304131708069FYIZSXQ,BulkQuantity:0,Recheck:0,SF:29|28|17|19|48|38,T C:nil,Content:0,EDM:-3,IP:nil,URL:0,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-UUID: b317ae92d9da11edb6b9f13eb10bd0fe-20230413 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1862041784; Thu, 13 Apr 2023 17:08:04 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.185) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Thu, 13 Apr 2023 17:08:03 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Thu, 13 Apr 2023 17:08:03 +0800 From: Yi-De Wu To: Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Matthias Brugger , AngeloGioacchino Del Regno , Yingshiuan Pan CC: , , , , , Jades Shih , Miles Chen , Ivan Tseng , My Chuang , Shawn Hsiao , PeiLun Suei , Ze-Yu Wang , Liju Chen , Yi-De Wu Subject: [PATCH v1 4/6] soc: mediatek: virt: geniezone: Introduce irqchip for virtual interrupt injection Date: Thu, 13 Apr 2023 17:07:33 +0800 Message-ID: <20230413090735.4182-5-yi-de.wu@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230413090735.4182-1-yi-de.wu@mediatek.com> References: <20230413090735.4182-1-yi-de.wu@mediatek.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org From: "Yingshiuan Pan" Enable GenieZone to handle virtual interrupt injection request. Signed-off-by: Yingshiuan Pan Signed-off-by: Yi-De Wu --- drivers/soc/mediatek/virt/geniezone/Makefile | 2 +- drivers/soc/mediatek/virt/geniezone/gzvm.h | 5 + .../mediatek/virt/geniezone/gzvm_irqchip.c | 94 +++++++++++++++++++ .../soc/mediatek/virt/geniezone/gzvm_vcpu.c | 6 ++ drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 77 +++++++++++++++ include/uapi/linux/gzvm_common.h | 32 +++++++ 6 files changed, 215 insertions(+), 1 deletion(-) create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c diff --git a/drivers/soc/mediatek/virt/geniezone/Makefile b/drivers/soc/mediatek/virt/geniezone/Makefile index e1dfbb9c568d..d2302a3a93fc 100644 --- a/drivers/soc/mediatek/virt/geniezone/Makefile +++ b/drivers/soc/mediatek/virt/geniezone/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o +gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o gzvm_irqchip.o obj-$(CONFIG_MTK_GZVM) += gzvm.o diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h index 43f215d4b0da..89cea5441a2d 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm.h +++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h @@ -98,6 +98,11 @@ int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2, #define MT_HVC_GZVM_PROBE GZVM_HCALL_ID(GZVM_FUNC_PROBE) #define MT_HVC_GZVM_ENABLE_CAP GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP) +void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu); +int gzvm_vgic_inject_irq(struct gzvm *gzvm, unsigned int vcpu_idx, u32 irq_type, + u32 irq, bool level); +int gzvm_vgic_inject_spi(struct gzvm *gzvm, unsigned int vcpu_idx, + u32 spi_irq, bool level); int gz_err_to_errno(unsigned long err); #endif /* __GZVM_H__ */ diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c new file mode 100644 index 000000000000..7aa5868a221c --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#include +#include + +#include "gzvm.h" + +/** + * @brief check all LRs synced from gz hypervisor + * Traverse all LRs, see if any EOIed vint, notify_acked_irq if any. + * GZ does not fold/unfold everytime KVM_RUN, so we have to traverse all saved + * LRs. It will not takes much more time comparing to fold/unfold everytime + * GZVM_RUN, because there are only few LRs. + */ +void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu) +{ + int i; + + for (i = 0; i < vcpu->hwstate->nr_lrs; i++) { + uint64_t lr_val = vcpu->hwstate->lr[i]; + /* 0 means unused */ + if (!lr_val) + continue; + } +} + +/** + * @brief Check the irq number and irq_type are matched + */ +static bool is_irq_valid(u32 irq, u32 irq_type) +{ + switch (irq_type) { + case GZVM_IRQ_TYPE_CPU: /* 0 ~ 15: SGI */ + if (likely(irq <= GZVM_IRQ_CPU_FIQ)) + return true; + break; + case GZVM_IRQ_TYPE_PPI: /* 16 ~ 31: PPI */ + if (likely(irq >= VGIC_NR_SGIS && irq < VGIC_NR_PRIVATE_IRQS)) + return true; + break; + case GZVM_IRQ_TYPE_SPI: /* 32 ~ : SPT */ + if (likely(irq >= VGIC_NR_PRIVATE_IRQS)) + return true; + break; + default: + return false; + } + return false; +} + +/** + * @brief Inject virtual interrupt to a VM + * + * @param gzvm + * @param vcpu_idx: vcpu index, only valid if PPI + * @param irq: irq number + * @param irq_type + * @param level, true: 1; false: 0 + */ +int gzvm_vgic_inject_irq(struct gzvm *gzvm, unsigned int vcpu_idx, u32 irq_type, + u32 irq, bool level) +{ + unsigned long a1 = assemble_vm_vcpu_tuple(gzvm->vm_id, vcpu_idx); + struct arm_smccc_res res; + + if (!unlikely(is_irq_valid(irq, irq_type))) + return -EINVAL; + + gzvm_hypcall_wrapper(MT_HVC_GZVM_IRQ_LINE, a1, irq, level, + 0, 0, 0, 0, &res); + if (res.a0) { + pr_err("Failed to set IRQ level (%d) to irq#%u on vcpu %d with ret=%d\n", + level, irq, vcpu_idx, (int)res.a0); + return -EFAULT; + } + + return 0; +} + +/** + * @brief Inject virtual spi interrupt + * + * @param spi_irq This is spi interrupt number (starts from 0 instead of 32) + * @return 0 succeed, other negative values are error + */ +int gzvm_vgic_inject_spi(struct gzvm *gzvm, unsigned int vcpu_idx, + u32 spi_irq, bool level) +{ + return gzvm_vgic_inject_irq(gzvm, 0, GZVM_IRQ_TYPE_SPI, + spi_irq + VGIC_NR_PRIVATE_IRQS, level); +} diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c index 726db866dfcf..5f2e24d13c41 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c @@ -67,6 +67,11 @@ static long gzvm_vcpu_update_one_reg(struct gzvm_vcpu *vcpu, void * __user argp, return ret; } +static void gzvm_sync_hwstate(struct gzvm_vcpu *vcpu) +{ + gzvm_sync_vgic_state(vcpu); +} + /** * @brief Handle vcpu run ioctl, entry point to guest and exit point from guest * @@ -115,6 +120,7 @@ static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * __user argp) need_userspace = true; goto out; } + gzvm_sync_hwstate(vcpu); } while (!need_userspace); out: diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c index df4ccdc3b7f0..7895f40b23eb 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c @@ -193,6 +193,69 @@ static int gzvm_vm_ioctl_set_memory_region(struct gzvm *gzvm, return register_memslot_addr_range(gzvm, memslot); } +static int gzvm_vm_ioctl_irq_line(struct gzvm *gzvm, + struct gzvm_irq_level *irq_level) +{ + u32 irq = irq_level->irq; + unsigned int irq_type, vcpu_idx, irq_num; + bool level = irq_level->level; + + irq_type = (irq >> GZVM_IRQ_TYPE_SHIFT) & GZVM_IRQ_TYPE_MASK; + vcpu_idx = (irq >> GZVM_IRQ_VCPU_SHIFT) & GZVM_IRQ_VCPU_MASK; + vcpu_idx += ((irq >> GZVM_IRQ_VCPU2_SHIFT) & GZVM_IRQ_VCPU2_MASK) * + (GZVM_IRQ_VCPU_MASK + 1); + irq_num = (irq >> GZVM_IRQ_NUM_SHIFT) & GZVM_IRQ_NUM_MASK; + + return gzvm_vgic_inject_irq(gzvm, vcpu_idx, irq_num, irq_type, level); +} + +static int gzvm_vm_ioctl_create_device(struct gzvm *gzvm, void __user *argp) +{ + struct gzvm_create_device *gzvm_dev; + void *dev_data = NULL; + struct arm_smccc_res res = {0}; + int ret; + + gzvm_dev = (struct gzvm_create_device *)alloc_pages_exact(PAGE_SIZE, + GFP_KERNEL); + if (!gzvm_dev) + return -ENOMEM; + if (copy_from_user(gzvm_dev, argp, sizeof(*gzvm_dev))) { + ret = -EFAULT; + goto err_free_dev; + } + + if (gzvm_dev->attr_addr != 0 && gzvm_dev->attr_size != 0) { + size_t attr_size = gzvm_dev->attr_size; + void __user *attr_addr = (void __user *)gzvm_dev->attr_addr; + + /* Size of device specific data should not be over a page. */ + if (attr_size > PAGE_SIZE) + return -EINVAL; + + dev_data = alloc_pages_exact(attr_size, GFP_KERNEL); + if (!dev_data) { + ret = -ENOMEM; + goto err_free_dev; + } + + if (copy_from_user(dev_data, attr_addr, attr_size)) { + ret = -EFAULT; + goto err_free_dev_data; + } + gzvm_dev->attr_addr = virt_to_phys(dev_data); + } + + ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_DEVICE, gzvm->vm_id, + virt_to_phys(gzvm_dev), 0, 0, 0, 0, 0, &res); +err_free_dev_data: + if (dev_data) + free_pages_exact(dev_data, 0); +err_free_dev: + free_pages_exact(gzvm_dev, 0); + return ret; +} + static int gzvm_vm_enable_cap_hyp(struct gzvm *gzvm, struct gzvm_enable_cap *cap, struct arm_smccc_res *res) @@ -300,6 +363,20 @@ static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl, ret = gzvm_vm_ioctl_set_memory_region(gzvm, &userspace_mem); break; } + case GZVM_IRQ_LINE: { + struct gzvm_irq_level irq_event; + + ret = -EFAULT; + if (copy_from_user(&irq_event, argp, sizeof(irq_event))) + goto out; + + ret = gzvm_vm_ioctl_irq_line(gzvm, &irq_event); + break; + } + case GZVM_CREATE_DEVICE: { + ret = gzvm_vm_ioctl_create_device(gzvm, argp); + break; + } case GZVM_ENABLE_CAP: { struct gzvm_enable_cap cap; diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h index aa97438bab8c..6808cfe59450 100644 --- a/include/uapi/linux/gzvm_common.h +++ b/include/uapi/linux/gzvm_common.h @@ -183,6 +183,38 @@ struct gzvm_userspace_memory_region { #define GZVM_IRQ_CPU_IRQ 0 #define GZVM_IRQ_CPU_FIQ 1 +struct gzvm_irq_level { + union { + __u32 irq; + __s32 status; + }; + __u32 level; +}; +#define GZVM_IRQ_LINE _IOW(GZVM_IOC_MAGIC, 0x61, \ + struct gzvm_irq_level) + +enum gzvm_device_type { + GZVM_DEV_TYPE_ARM_VGIC_V3_DIST, + GZVM_DEV_TYPE_ARM_VGIC_V3_REDIST, + GZVM_DEV_TYPE_MAX, +}; + +struct gzvm_create_device { + __u32 dev_type; /* device type */ + __u32 id; /* out: device id */ + __u64 flags; /* device specific flags */ + __u64 dev_addr; /* device ipa address in VM's view */ + __u64 dev_reg_size; /* device register range size */ + /* + * If user -> kernel, this is user virtual address of device specific + * attributes (if needed). If kernel->hypervisor, this is ipa. + */ + __u64 attr_addr; + __u64 attr_size; /* size of device specific attributes */ +}; +#define GZVM_CREATE_DEVICE _IOWR(GZVM_IOC_MAGIC, 0xe0, \ + struct gzvm_create_device) + /* * ioctls for vcpu fds */ From patchwork Thu Apr 13 09:07:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi-De Wu X-Patchwork-Id: 672948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B604C77B61 for ; Thu, 13 Apr 2023 09:08:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230155AbjDMJIZ (ORCPT ); Thu, 13 Apr 2023 05:08:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229936AbjDMJIS (ORCPT ); Thu, 13 Apr 2023 05:08:18 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D87DE180; Thu, 13 Apr 2023 02:08:09 -0700 (PDT) X-UUID: b2b2697ed9da11edb6b9f13eb10bd0fe-20230413 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=9RlUHcwmASbOr8nBNERP6xk9ihOcrJHrMLiKF4qH658=; b=eylvL7/VBIQbkmG/kRjMrWJ9YFz3VZ5F4fULPgoMVuE+41L9X2dkQn74vcX9VTjb+TPSXS9BhpXKsGDfTrney/l/XhNUXFPFmYCRt3fSL6hi0N478WnQcy0w2BmM6lyq9mOIPh2LPs4GfHcJrielzmV8cC4USFK8YjShYi0GgzE=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.22, REQID:552df493-f27a-4950-b576-797a2644279c, IP:0, U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:70 X-CID-INFO: VERSION:1.1.22, REQID:552df493-f27a-4950-b576-797a2644279c, IP:0, URL :0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Spam_GS981B3D,ACTI ON:quarantine,TS:70 X-CID-META: VersionHash:120426c, CLOUDID:80cb34a1-8fcb-430b-954a-ba3f00fa94a5, B ulkID:2304131708060ZN9MV0D,BulkQuantity:0,Recheck:0,SF:48|38|29|28|17|19,T C:nil,Content:0,EDM:-3,IP:nil,URL:11|1,File:nil,Bulk:nil,QS:nil,BEC:nil,CO L:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-UUID: b2b2697ed9da11edb6b9f13eb10bd0fe-20230413 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 2139487939; Thu, 13 Apr 2023 17:08:04 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.185) by mtkmbs11n1.mediatek.inc (172.21.101.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Thu, 13 Apr 2023 17:08:03 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Thu, 13 Apr 2023 17:08:03 +0800 From: Yi-De Wu To: Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Matthias Brugger , AngeloGioacchino Del Regno , Yingshiuan Pan CC: , , , , , Jades Shih , Miles Chen , Ivan Tseng , My Chuang , Shawn Hsiao , PeiLun Suei , Ze-Yu Wang , Liju Chen , Yi-De Wu Subject: [PATCH v1 5/6] soc: mediatek: virt: geniezone: Add ioeventfd support Date: Thu, 13 Apr 2023 17:07:34 +0800 Message-ID: <20230413090735.4182-6-yi-de.wu@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230413090735.4182-1-yi-de.wu@mediatek.com> References: <20230413090735.4182-1-yi-de.wu@mediatek.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org From: "Yingshiuan Pan" ioeventfd leverages eventfd to provide asynchronous notification mechanism for VMM. VMM can register a mmio address and bind with an eventfd. Once a mmio trap occurs on this registered region, its corresponding eventfd will be notified. Signed-off-by: Yingshiuan Pan Signed-off-by: Yi-De Wu --- drivers/soc/mediatek/virt/geniezone/Makefile | 2 +- drivers/soc/mediatek/virt/geniezone/gzvm.h | 10 + .../mediatek/virt/geniezone/gzvm_eventfd.c | 252 ++++++++++++++++++ .../soc/mediatek/virt/geniezone/gzvm_vcpu.c | 26 +- drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 14 + include/uapi/linux/gzvm_common.h | 24 ++ 6 files changed, 326 insertions(+), 2 deletions(-) create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c diff --git a/drivers/soc/mediatek/virt/geniezone/Makefile b/drivers/soc/mediatek/virt/geniezone/Makefile index d2302a3a93fc..dc6c760231e2 100644 --- a/drivers/soc/mediatek/virt/geniezone/Makefile +++ b/drivers/soc/mediatek/virt/geniezone/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o gzvm_irqchip.o +gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o gzvm_irqchip.o gzvm_eventfd.o obj-$(CONFIG_MTK_GZVM) += gzvm.o diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h index 89cea5441a2d..b0edf56c2832 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm.h +++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h @@ -37,6 +37,7 @@ struct gzvm { struct list_head devices; gzvm_id_t vm_id; + struct list_head ioevents; struct { spinlock_t lock; struct list_head items; @@ -98,6 +99,10 @@ int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2, #define MT_HVC_GZVM_PROBE GZVM_HCALL_ID(GZVM_FUNC_PROBE) #define MT_HVC_GZVM_ENABLE_CAP GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP) +int gzvm_init_ioeventfd(struct gzvm *gzvm); +int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args); +bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len, + const void *val); void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu); int gzvm_vgic_inject_irq(struct gzvm *gzvm, unsigned int vcpu_idx, u32 irq_type, u32 irq, bool level); @@ -105,4 +110,9 @@ int gzvm_vgic_inject_spi(struct gzvm *gzvm, unsigned int vcpu_idx, u32 spi_irq, bool level); int gz_err_to_errno(unsigned long err); +#include +void eventfd_ctx_do_read(struct eventfd_ctx *ctx, __u64 *cnt); +struct vm_area_struct *vma_lookup(struct mm_struct *mm, unsigned long addr); +void add_wait_queue_priority(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry); + #endif /* __GZVM_H__ */ diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c new file mode 100644 index 000000000000..63f042fefbe5 --- /dev/null +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c @@ -0,0 +1,252 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2022 MediaTek Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include "gzvm.h" + +struct gzvm_ioevent { + struct list_head list; + __u64 addr; + __u32 len; + struct eventfd_ctx *evt_ctx; + __u64 datamatch; + bool wildcard; +}; + +/* assumes gzvm->slots_lock held */ +static bool +ioeventfd_check_collision(struct gzvm *gzvm, struct gzvm_ioevent *p) +{ + struct gzvm_ioevent *_p; + + list_for_each_entry(_p, &gzvm->ioevents, list) + if (_p->addr == p->addr && + (!_p->len || !p->len || + (_p->len == p->len && + (_p->wildcard || p->wildcard || + _p->datamatch == p->datamatch)))) + return true; + + return false; +} + +static void gzvm_ioevent_release(struct gzvm_ioevent *p) +{ + eventfd_ctx_put(p->evt_ctx); + list_del(&p->list); + kfree(p); +} + +static bool +gzvm_ioevent_in_range(struct gzvm_ioevent *p, __u64 addr, int len, + const void *val) +{ + u64 _val; + + if (addr != p->addr) + /* address must be precise for a hit */ + return false; + + if (!p->len) + /* length = 0 means only look at the address, so always a hit */ + return true; + + if (len != p->len) + /* address-range must be precise for a hit */ + return false; + + if (p->wildcard) + /* all else equal, wildcard is always a hit */ + return true; + + /* otherwise, we have to actually compare the data */ + + WARN_ON_ONCE(!IS_ALIGNED((unsigned long)val, len)); + + switch (len) { + case 1: + _val = *(u8 *)val; + break; + case 2: + _val = *(u16 *)val; + break; + case 4: + _val = *(u32 *)val; + break; + case 8: + _val = *(u64 *)val; + break; + default: + return false; + } + + return _val == p->datamatch; +} + +static int gzvm_deassign_ioeventfd(struct gzvm *gzvm, + struct gzvm_ioeventfd *args) +{ + struct gzvm_ioevent *p, *tmp; + struct eventfd_ctx *evt_ctx; + int ret = -ENOENT; + bool wildcard; + + evt_ctx = eventfd_ctx_fdget(args->fd); + if (IS_ERR(evt_ctx)) + return PTR_ERR(evt_ctx); + + wildcard = !(args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH); + + mutex_lock(&gzvm->lock); + + list_for_each_entry_safe(p, tmp, &gzvm->ioevents, list) { + if (p->evt_ctx != evt_ctx || + p->addr != args->addr || + p->len != args->len || + p->wildcard != wildcard) + continue; + + if (!p->wildcard && p->datamatch != args->datamatch) + continue; + + gzvm_ioevent_release(p); + ret = 0; + break; + } + + mutex_unlock(&gzvm->lock); + + /* got in the front of this function */ + eventfd_ctx_put(evt_ctx); + + return ret; +} + +static int gzvm_assign_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args) +{ + struct eventfd_ctx *evt_ctx; + struct gzvm_ioevent *evt; + int ret; + + evt_ctx = eventfd_ctx_fdget(args->fd); + if (IS_ERR(evt_ctx)) + return PTR_ERR(evt_ctx); + + evt = kmalloc(sizeof(*evt), GFP_KERNEL); + if (!evt) + return -ENOMEM; + *evt = (struct gzvm_ioevent) { + .addr = args->addr, + .len = args->len, + .evt_ctx = evt_ctx, + }; + if (args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH) { + evt->datamatch = args->datamatch; + evt->wildcard = false; + } else { + evt->wildcard = true; + } + + if (ioeventfd_check_collision(gzvm, evt)) { + ret = -EEXIST; + goto err_free; + } + + mutex_lock(&gzvm->lock); + list_add_tail(&evt->list, &gzvm->ioevents); + mutex_unlock(&gzvm->lock); + + return 0; + +err_free: + kfree(evt); + eventfd_ctx_put(evt_ctx); + return ret; +} + +/** + * @brief Check user arguments is valid + * + * @param args + * @retval true valid arguments + * @retval false invalid arguments + */ +static bool gzvm_ioeventfd_check_valid(struct gzvm_ioeventfd *args) +{ + /* must be natural-word sized, or 0 to ignore length */ + switch (args->len) { + case 0: + case 1: + case 2: + case 4: + case 8: + break; + default: + return false; + } + + /* check for range overflow */ + if (args->addr + args->len < args->addr) + return false; + + /* check for extra flags that we don't understand */ + if (args->flags & ~GZVM_IOEVENTFD_VALID_FLAG_MASK) + return false; + + /* ioeventfd with no length can't be combined with DATAMATCH */ + if (!args->len && (args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH)) + return false; + + /* gzvm does not support pio bus ioeventfd */ + if (args->flags & GZVM_IOEVENTFD_FLAG_PIO) + return false; + + return true; +} + +/** + * @brief GZVM_IOEVENTFD, register ioevent to ioevent list + */ +int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args) +{ + if (gzvm_ioeventfd_check_valid(args) == false) + return -EINVAL; + + if (args->flags & GZVM_IOEVENTFD_FLAG_DEASSIGN) + return gzvm_deassign_ioeventfd(gzvm, args); + return gzvm_assign_ioeventfd(gzvm, args); +} + +/** + * @brief Travers this vm's registered ioeventfd to see if need notifying it + * @retval true if this io is already sent to ioeventfd's listner + * @retval false if we cannot find any ioeventfd registering this mmio write + */ +bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len, + const void *val) +{ + struct gzvm_ioevent *e; + + list_for_each_entry(e, &vcpu->gzvm->ioevents, list) { + if (gzvm_ioevent_in_range(e, addr, len, val)) { + eventfd_signal(e->evt_ctx, 1); + return true; + } + } + return false; +} + +int gzvm_init_ioeventfd(struct gzvm *gzvm) +{ + INIT_LIST_HEAD(&gzvm->ioevents); + + return 0; +} diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c index 5f2e24d13c41..14a9fe14e79d 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c @@ -72,6 +72,29 @@ static void gzvm_sync_hwstate(struct gzvm_vcpu *vcpu) gzvm_sync_vgic_state(vcpu); } +/** + * @brief try to handle mmio in kernel space + * + * @param vcpu + * @return true this mmio exit has been processed. + * @return false this mmio exit has not been processed, require userspace. + */ +static bool gzvm_vcpu_handle_mmio(struct gzvm_vcpu *vcpu) +{ + __u64 addr; + __u32 len; + const void *val_ptr; + + /* So far, we don't have in-kernel mmio read handler */ + if (!vcpu->run->mmio.is_write) + return false; + addr = vcpu->run->mmio.phys_addr; + len = vcpu->run->mmio.size; + val_ptr = &vcpu->run->mmio.data; + + return gzvm_ioevent_write(vcpu, addr, len, val_ptr); +} + /** * @brief Handle vcpu run ioctl, entry point to guest and exit point from guest * @@ -97,7 +120,8 @@ static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * __user argp) 0, &res); switch (res.a1) { case GZVM_EXIT_MMIO: - need_userspace = true; + if (!gzvm_vcpu_handle_mmio(vcpu)) + need_userspace = true; break; /* * geniezone's responsibility to fill corresponding data diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c index 7895f40b23eb..cceaa532c2ce 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c @@ -377,6 +377,15 @@ static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl, ret = gzvm_vm_ioctl_create_device(gzvm, argp); break; } + case GZVM_IOEVENTFD: { + struct gzvm_ioeventfd data; + + ret = -EFAULT; + if (copy_from_user(&data, argp, sizeof(data))) + goto out; + ret = gzvm_ioeventfd(gzvm, &data); + break; + } case GZVM_ENABLE_CAP: { struct gzvm_enable_cap cap; @@ -481,6 +490,11 @@ static struct gzvm *gzvm_create_vm(unsigned long vm_type) mutex_init(&gzvm->lock); INIT_LIST_HEAD(&gzvm->devices); mutex_init(&gzvm->irq_lock); + ret = gzvm_init_ioeventfd(gzvm); + if (ret) { + pr_err("Failed to initialize ioeventfd\n"); + goto err; + } pr_info("VM-%u is created\n", gzvm->vm_id); mutex_lock(&gzvm_list_lock); diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h index 6808cfe59450..6dbaddd77ec7 100644 --- a/include/uapi/linux/gzvm_common.h +++ b/include/uapi/linux/gzvm_common.h @@ -193,6 +193,30 @@ struct gzvm_irq_level { #define GZVM_IRQ_LINE _IOW(GZVM_IOC_MAGIC, 0x61, \ struct gzvm_irq_level) +enum { + gzvm_ioeventfd_flag_nr_datamatch, + gzvm_ioeventfd_flag_nr_pio, + gzvm_ioeventfd_flag_nr_deassign, + gzvm_ioeventfd_flag_nr_max, +}; + +#define GZVM_IOEVENTFD_FLAG_DATAMATCH (1 << gzvm_ioeventfd_flag_nr_datamatch) +#define GZVM_IOEVENTFD_FLAG_PIO (1 << gzvm_ioeventfd_flag_nr_pio) +#define GZVM_IOEVENTFD_FLAG_DEASSIGN (1 << gzvm_ioeventfd_flag_nr_deassign) +#define GZVM_IOEVENTFD_VALID_FLAG_MASK ((1 << gzvm_ioeventfd_flag_nr_max) - 1) + +struct gzvm_ioeventfd { + __u64 datamatch; + __u64 addr; /* legal pio/mmio address */ + __u32 len; /* 1, 2, 4, or 8 bytes; or 0 to ignore length */ + __s32 fd; + __u32 flags; + __u8 pad[36]; +}; + +#define GZVM_IOEVENTFD _IOW(GZVM_IOC_MAGIC, 0x79, \ + struct gzvm_ioeventfd) + enum gzvm_device_type { GZVM_DEV_TYPE_ARM_VGIC_V3_DIST, GZVM_DEV_TYPE_ARM_VGIC_V3_REDIST, From patchwork Thu Apr 13 09:07:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi-De Wu X-Patchwork-Id: 674251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53CDFC77B6E for ; Thu, 13 Apr 2023 09:08:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230229AbjDMJIm (ORCPT ); Thu, 13 Apr 2023 05:08:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229995AbjDMJIV (ORCPT ); Thu, 13 Apr 2023 05:08:21 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC9284222; Thu, 13 Apr 2023 02:08:11 -0700 (PDT) X-UUID: b2cfbb5ad9da11edb6b9f13eb10bd0fe-20230413 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=RlFwX5X9kVFzvo1JT9dIiD06WWwxoGOYHnal8lHLeOU=; b=DxkF9jIiSb7/oTS5nZlhU9XmFSyuveh9xM9HyHNtfS4vUD8h/2rnCftPX0o8xZbK+p2R4Wr3rf8GY1UGyS3QpgUDlSYxaNeCLlN3veuTRfDPJCPOBKgPWuKn7FjeqxzRAITGmjHAPmsYXfGqvvgLROuUbR5ZMWZLpHzvAUANDMc=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.22, REQID:943ebe18-d2ba-47be-b6fc-c1172fbd780c, IP:0, U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:70 X-CID-INFO: VERSION:1.1.22, REQID:943ebe18-d2ba-47be-b6fc-c1172fbd780c, IP:0, URL :0,TC:0,Content:-25,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Spam_GS981B3D,ACTI ON:quarantine,TS:70 X-CID-META: VersionHash:120426c, CLOUDID:3fcfef83-cd9c-45f5-8134-710979e3df0e, B ulkID:230413170806J4LRT3NC,BulkQuantity:0,Recheck:0,SF:28|17|19|48|38|29,T C:nil,Content:0,EDM:-3,IP:nil,URL:0,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-UUID: b2cfbb5ad9da11edb6b9f13eb10bd0fe-20230413 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 954268287; Thu, 13 Apr 2023 17:08:04 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.185) by mtkmbs13n1.mediatek.inc (172.21.101.193) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Thu, 13 Apr 2023 17:08:03 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Thu, 13 Apr 2023 17:08:03 +0800 From: Yi-De Wu To: Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Catalin Marinas , Will Deacon , Matthias Brugger , AngeloGioacchino Del Regno , Yingshiuan Pan CC: , , , , , Jades Shih , Miles Chen , Ivan Tseng , My Chuang , Shawn Hsiao , PeiLun Suei , Ze-Yu Wang , Liju Chen , Yi-De Wu Subject: [PATCH v1 6/6] soc: mediatek: virt: geniezone: Add irqfd support Date: Thu, 13 Apr 2023 17:07:35 +0800 Message-ID: <20230413090735.4182-7-yi-de.wu@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230413090735.4182-1-yi-de.wu@mediatek.com> References: <20230413090735.4182-1-yi-de.wu@mediatek.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org From: "Yingshiuan Pan" irqfd enables other threads than vcpu threads to inject virtual interrupt through irqfd asynchronously rather through ioctl interface. This interface is necessary for VMM which creates separated thread for IO handling or uses vhost devices. Signed-off-by: Yingshiuan Pan Signed-off-by: Yi-De Wu --- drivers/soc/mediatek/virt/geniezone/gzvm.h | 8 + .../mediatek/virt/geniezone/gzvm_eventfd.c | 497 ++++++++++++++++++ .../mediatek/virt/geniezone/gzvm_irqchip.c | 13 + drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 18 +- include/uapi/linux/gzvm_common.h | 18 + 5 files changed, 553 insertions(+), 1 deletion(-) diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h index b0edf56c2832..c79f84c17bb1 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm.h +++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h @@ -99,6 +99,14 @@ int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2, #define MT_HVC_GZVM_PROBE GZVM_HCALL_ID(GZVM_FUNC_PROBE) #define MT_HVC_GZVM_ENABLE_CAP GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP) +#define GZVM_USERSPACE_IRQ_SOURCE_ID 0 +#define GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID 1 + +void gzvm_notify_acked_irq(struct gzvm *gzvm, unsigned int gsi); +int gzvm_irqfd(struct gzvm *gzvm, struct gzvm_irqfd *args); +int gzvm_irqfd_init(void); +void gzvm_irqfd_exit(void); +int gzvm_init_irqfd(struct gzvm *gzvm); int gzvm_init_ioeventfd(struct gzvm *gzvm); int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args); bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len, diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c index 63f042fefbe5..d8eea6c1a2d6 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c @@ -12,6 +12,503 @@ #include #include "gzvm.h" +struct gzvm_irq_ack_notifier { + struct hlist_node link; + unsigned int gsi; + void (*irq_acked)(struct gzvm_irq_ack_notifier *ian); +}; + +/* + * Resampling irqfds are a special variety of irqfds used to emulate + * level triggered interrupts. The interrupt is asserted on eventfd + * trigger. On acknowledgment through the irq ack notifier, the + * interrupt is de-asserted and userspace is notified through the + * resamplefd. All resamplers on the same gsi are de-asserted + * together, so we don't need to track the state of each individual + * user. We can also therefore share the same irq source ID. + */ +struct gzvm_kernel_irqfd_resampler { + struct gzvm *gzvm; + /* + * List of resampling struct _irqfd objects sharing this gsi. + * RCU list modified under gzvm->irqfds.resampler_lock + */ + struct list_head list; + struct gzvm_irq_ack_notifier notifier; + /* + * Entry in list of gzvm->irqfd.resampler_list. Use for sharing + * resamplers among irqfds on the same gsi. + * Accessed and modified under gzvm->irqfds.resampler_lock + */ + struct list_head link; +}; + +struct gzvm_kernel_irqfd { + struct gzvm *gzvm; + wait_queue_entry_t wait; + /* Used for level IRQ fast-path */ + int gsi; + /* The resampler used by this irqfd (resampler-only) */ + struct gzvm_kernel_irqfd_resampler *resampler; + /* Eventfd notified on resample (resampler-only) */ + struct eventfd_ctx *resamplefd; + /* Entry in list of irqfds for a resampler (resampler-only) */ + struct list_head resampler_link; + /* Used for setup/shutdown */ + struct eventfd_ctx *eventfd; + struct list_head list; + poll_table pt; + struct work_struct shutdown; +}; + +static struct workqueue_struct *irqfd_cleanup_wq; + +/** + * @brief irqfd to inject virtual interrupt + * @param irq This is spi interrupt number (starts from 0 instead of 32) + */ +static void irqfd_set_spi(struct gzvm *gzvm, int irq_source_id, u32 irq, + int level, bool line_status) +{ + if (level) + gzvm_vgic_inject_spi(gzvm, 0, irq, level); +} + +/* + * Since resampler irqfds share an IRQ source ID, we de-assert once + * then notify all of the resampler irqfds using this GSI. We can't + * do multiple de-asserts or we risk racing with incoming re-asserts. + */ +static void +irqfd_resampler_ack(struct gzvm_irq_ack_notifier *ian) +{ + struct gzvm_kernel_irqfd_resampler *resampler; + struct gzvm *gzvm; + struct gzvm_kernel_irqfd *irqfd; + int idx; + + resampler = container_of(ian, + struct gzvm_kernel_irqfd_resampler, notifier); + gzvm = resampler->gzvm; + + irqfd_set_spi(gzvm, GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID, + resampler->notifier.gsi, 0, false); + + idx = srcu_read_lock(&gzvm->irq_srcu); + + list_for_each_entry_srcu(irqfd, &resampler->list, resampler_link, + srcu_read_lock_held(&gzvm->irq_srcu)) { + eventfd_signal(irqfd->resamplefd, 1); + } + + srcu_read_unlock(&gzvm->irq_srcu, idx); +} + +static void gzvm_register_irq_ack_notifier(struct gzvm *gzvm, + struct gzvm_irq_ack_notifier *ian) +{ + mutex_lock(&gzvm->irq_lock); + hlist_add_head_rcu(&ian->link, &gzvm->irq_ack_notifier_list); + mutex_unlock(&gzvm->irq_lock); +} + +static void gzvm_unregister_irq_ack_notifier(struct gzvm *gzvm, + struct gzvm_irq_ack_notifier *ian) +{ + mutex_lock(&gzvm->irq_lock); + hlist_del_init_rcu(&ian->link); + mutex_unlock(&gzvm->irq_lock); + synchronize_srcu(&gzvm->irq_srcu); +} + +static void +irqfd_resampler_shutdown(struct gzvm_kernel_irqfd *irqfd) +{ + struct gzvm_kernel_irqfd_resampler *resampler = irqfd->resampler; + struct gzvm *gzvm = resampler->gzvm; + + mutex_lock(&gzvm->irqfds.resampler_lock); + + list_del_rcu(&irqfd->resampler_link); + synchronize_srcu(&gzvm->irq_srcu); + + if (list_empty(&resampler->list)) { + list_del(&resampler->link); + gzvm_unregister_irq_ack_notifier(gzvm, &resampler->notifier); + irqfd_set_spi(gzvm, GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID, + resampler->notifier.gsi, 0, false); + kfree(resampler); + } + + mutex_unlock(&gzvm->irqfds.resampler_lock); +} + +/* + * Race-free decouple logic (ordering is critical) + */ +static void +irqfd_shutdown(struct work_struct *work) +{ + struct gzvm_kernel_irqfd *irqfd = + container_of(work, struct gzvm_kernel_irqfd, shutdown); + struct gzvm *gzvm = irqfd->gzvm; + u64 cnt; + + /* Make sure irqfd has been initialized in assign path. */ + synchronize_srcu(&gzvm->irq_srcu); + + /* + * Synchronize with the wait-queue and unhook ourselves to prevent + * further events. + */ + eventfd_ctx_remove_wait_queue(irqfd->eventfd, &irqfd->wait, &cnt); + + if (irqfd->resampler) { + irqfd_resampler_shutdown(irqfd); + eventfd_ctx_put(irqfd->resamplefd); + } + + /* + * It is now safe to release the object's resources + */ + eventfd_ctx_put(irqfd->eventfd); + kfree(irqfd); +} + + +/* assumes gzvm->irqfds.lock is held */ +static bool +irqfd_is_active(struct gzvm_kernel_irqfd *irqfd) +{ + return list_empty(&irqfd->list) ? false : true; +} + +/* + * Mark the irqfd as inactive and schedule it for removal + * + * assumes gzvm->irqfds.lock is held + */ +static void +irqfd_deactivate(struct gzvm_kernel_irqfd *irqfd) +{ + if (!irqfd_is_active(irqfd)) + return; + + list_del_init(&irqfd->list); + + queue_work(irqfd_cleanup_wq, &irqfd->shutdown); +} + +/** + * @brief Wake up irqfd to do virtual interrupt injection + */ +static int +irqfd_wakeup(wait_queue_entry_t *wait, unsigned int mode, int sync, void *key) +{ + struct gzvm_kernel_irqfd *irqfd = + container_of(wait, struct gzvm_kernel_irqfd, wait); + __poll_t flags = key_to_poll(key); + struct gzvm *gzvm = irqfd->gzvm; + + if (flags & EPOLLIN) { + u64 cnt; + + eventfd_ctx_do_read(irqfd->eventfd, &cnt); + /* gzvm's irq injection is not blocked, don't need workq */ + irqfd_set_spi(gzvm, GZVM_USERSPACE_IRQ_SOURCE_ID, irqfd->gsi, 1, + false); + } + + if (flags & EPOLLHUP) { + /* The eventfd is closing, detach from GZVM */ + unsigned long iflags; + + spin_lock_irqsave(&gzvm->irqfds.lock, iflags); + + /* + * Do more check if someone deactivated the irqfd before + * we could acquire the irqfds.lock. + */ + if (irqfd_is_active(irqfd)) + irqfd_deactivate(irqfd); + + spin_unlock_irqrestore(&gzvm->irqfds.lock, iflags); + } + + return 0; +} + +static void +irqfd_ptable_queue_proc(struct file *file, wait_queue_head_t *wqh, + poll_table *pt) +{ + struct gzvm_kernel_irqfd *irqfd = + container_of(pt, struct gzvm_kernel_irqfd, pt); + add_wait_queue_priority(wqh, &irqfd->wait); +} + +static int +gzvm_irqfd_assign(struct gzvm *gzvm, struct gzvm_irqfd *args) +{ + struct gzvm_kernel_irqfd *irqfd, *tmp; + struct fd f; + struct eventfd_ctx *eventfd = NULL, *resamplefd = NULL; + int ret; + __poll_t events; + int idx; + + irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL_ACCOUNT); + if (!irqfd) + return -ENOMEM; + + irqfd->gzvm = gzvm; + irqfd->gsi = args->gsi; + irqfd->resampler = NULL; + INIT_LIST_HEAD(&irqfd->list); + INIT_WORK(&irqfd->shutdown, irqfd_shutdown); + + f = fdget(args->fd); + if (!f.file) { + ret = -EBADF; + goto out; + } + + eventfd = eventfd_ctx_fileget(f.file); + if (IS_ERR(eventfd)) { + ret = PTR_ERR(eventfd); + goto fail; + } + + irqfd->eventfd = eventfd; + + if (args->flags & GZVM_IRQFD_FLAG_RESAMPLE) { + struct gzvm_kernel_irqfd_resampler *resampler; + + resamplefd = eventfd_ctx_fdget(args->resamplefd); + if (IS_ERR(resamplefd)) { + ret = PTR_ERR(resamplefd); + goto fail; + } + + irqfd->resamplefd = resamplefd; + INIT_LIST_HEAD(&irqfd->resampler_link); + + mutex_lock(&gzvm->irqfds.resampler_lock); + + list_for_each_entry(resampler, + &gzvm->irqfds.resampler_list, link) { + if (resampler->notifier.gsi == irqfd->gsi) { + irqfd->resampler = resampler; + break; + } + } + + if (!irqfd->resampler) { + resampler = kzalloc(sizeof(*resampler), + GFP_KERNEL_ACCOUNT); + if (!resampler) { + ret = -ENOMEM; + mutex_unlock(&gzvm->irqfds.resampler_lock); + goto fail; + } + + resampler->gzvm = gzvm; + INIT_LIST_HEAD(&resampler->list); + resampler->notifier.gsi = irqfd->gsi; + resampler->notifier.irq_acked = irqfd_resampler_ack; + INIT_LIST_HEAD(&resampler->link); + + list_add(&resampler->link, &gzvm->irqfds.resampler_list); + gzvm_register_irq_ack_notifier(gzvm, + &resampler->notifier); + irqfd->resampler = resampler; + } + + list_add_rcu(&irqfd->resampler_link, &irqfd->resampler->list); + synchronize_srcu(&gzvm->irq_srcu); + + mutex_unlock(&gzvm->irqfds.resampler_lock); + } + + /* + * Install our own custom wake-up handling so we are notified via + * a callback whenever someone signals the underlying eventfd + */ + init_waitqueue_func_entry(&irqfd->wait, irqfd_wakeup); + init_poll_funcptr(&irqfd->pt, irqfd_ptable_queue_proc); + + spin_lock_irq(&gzvm->irqfds.lock); + + ret = 0; + list_for_each_entry(tmp, &gzvm->irqfds.items, list) { + if (irqfd->eventfd != tmp->eventfd) + continue; + /* This fd is used for another irq already. */ + pr_err("already used: gsi=%d fd=%d\n", args->gsi, args->fd); + ret = -EBUSY; + spin_unlock_irq(&gzvm->irqfds.lock); + goto fail; + } + + idx = srcu_read_lock(&gzvm->irq_srcu); + + list_add_tail(&irqfd->list, &gzvm->irqfds.items); + + spin_unlock_irq(&gzvm->irqfds.lock); + + /* + * Check if there was an event already pending on the eventfd + * before we registered, and trigger it as if we didn't miss it. + */ + events = vfs_poll(f.file, &irqfd->pt); + + /* In case there is already a pending event */ + if (events & EPOLLIN) + irqfd_set_spi(gzvm, GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID, + irqfd->gsi, 1, false); + + srcu_read_unlock(&gzvm->irq_srcu, idx); + + /* + * do not drop the file until the irqfd is fully initialized, otherwise + * we might race against the EPOLLHUP + */ + fdput(f); + return 0; + +fail: + if (irqfd->resampler) + irqfd_resampler_shutdown(irqfd); + + if (resamplefd && !IS_ERR(resamplefd)) + eventfd_ctx_put(resamplefd); + + if (eventfd && !IS_ERR(eventfd)) + eventfd_ctx_put(eventfd); + + fdput(f); + +out: + kfree(irqfd); + return ret; +} + +static void gzvm_notify_acked_gsi(struct gzvm *gzvm, int gsi) +{ + struct gzvm_irq_ack_notifier *gian; + + hlist_for_each_entry_srcu(gian, &gzvm->irq_ack_notifier_list, + link, srcu_read_lock_held(&gzvm->irq_srcu)) + if (gian->gsi == gsi) + gian->irq_acked(gian); +} + +void gzvm_notify_acked_irq(struct gzvm *gzvm, unsigned int gsi) +{ + int idx; + + idx = srcu_read_lock(&gzvm->irq_srcu); + gzvm_notify_acked_gsi(gzvm, gsi); + srcu_read_unlock(&gzvm->irq_srcu, idx); +} + +/* + * shutdown any irqfd's that match fd+gsi + */ +static int gzvm_irqfd_deassign(struct gzvm *gzvm, struct gzvm_irqfd *args) +{ + struct gzvm_kernel_irqfd *irqfd, *tmp; + struct eventfd_ctx *eventfd; + + eventfd = eventfd_ctx_fdget(args->fd); + if (IS_ERR(eventfd)) + return PTR_ERR(eventfd); + + spin_lock_irq(&gzvm->irqfds.lock); + + list_for_each_entry_safe(irqfd, tmp, &gzvm->irqfds.items, list) { + if (irqfd->eventfd == eventfd && irqfd->gsi == args->gsi) + irqfd_deactivate(irqfd); + } + + spin_unlock_irq(&gzvm->irqfds.lock); + eventfd_ctx_put(eventfd); + + /* + * Block until we know all outstanding shutdown jobs have completed + * so that we guarantee there will not be any more interrupts on this + * gsi once this deassign function returns. + */ + flush_workqueue(irqfd_cleanup_wq); + + return 0; +} + +int gzvm_irqfd(struct gzvm *gzvm, struct gzvm_irqfd *args) +{ + if (args->flags & + ~(GZVM_IRQFD_FLAG_DEASSIGN | GZVM_IRQFD_FLAG_RESAMPLE)) + return -EINVAL; + + if (args->flags & GZVM_IRQFD_FLAG_DEASSIGN) + return gzvm_irqfd_deassign(gzvm, args); + + return gzvm_irqfd_assign(gzvm, args); +} + +/* + * This function is called as the gzvm VM fd is being released. Shutdown all + * irqfds that still remain open + */ +void gzvm_irqfd_release(struct gzvm *gzvm) +{ + struct gzvm_kernel_irqfd *irqfd, *tmp; + + spin_lock_irq(&gzvm->irqfds.lock); + + list_for_each_entry_safe(irqfd, tmp, &gzvm->irqfds.items, list) + irqfd_deactivate(irqfd); + + spin_unlock_irq(&gzvm->irqfds.lock); + + /* + * Block until we know all outstanding shutdown jobs have completed. + */ + flush_workqueue(irqfd_cleanup_wq); +} + +/* + * create a host-wide workqueue for issuing deferred shutdown requests + * aggregated from all vm* instances. We need our own isolated + * queue to ease flushing work items when a VM exits. + */ +int gzvm_irqfd_init(void) +{ + irqfd_cleanup_wq = alloc_workqueue("gzvm-irqfd-cleanup", 0, 0); + if (!irqfd_cleanup_wq) + return -ENOMEM; + + return 0; +} + +void gzvm_irqfd_exit(void) +{ + destroy_workqueue(irqfd_cleanup_wq); +} + +int gzvm_init_irqfd(struct gzvm *gzvm) +{ + spin_lock_init(&gzvm->irqfds.lock); + INIT_LIST_HEAD(&gzvm->irqfds.items); + INIT_LIST_HEAD(&gzvm->irqfds.resampler_list); + if (init_srcu_struct(&gzvm->irq_srcu)) + return -EINVAL; + INIT_HLIST_HEAD(&gzvm->irq_ack_notifier_list); + mutex_init(&gzvm->irqfds.resampler_lock); + + return 0; +} + struct gzvm_ioevent { struct list_head list; __u64 addr; diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c index 7aa5868a221c..134fa5e5715e 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c @@ -8,6 +8,12 @@ #include "gzvm.h" +static bool lr_signals_eoi(uint64_t lr_val) +{ + return !(lr_val & ICH_LR_STATE) && (lr_val & ICH_LR_EOI) && + !(lr_val & ICH_LR_HW); +} + /** * @brief check all LRs synced from gz hypervisor * Traverse all LRs, see if any EOIed vint, notify_acked_irq if any. @@ -20,10 +26,17 @@ void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu) int i; for (i = 0; i < vcpu->hwstate->nr_lrs; i++) { + uint32_t vintid; uint64_t lr_val = vcpu->hwstate->lr[i]; /* 0 means unused */ if (!lr_val) continue; + + vintid = lr_val & ICH_LR_VIRTUAL_ID_MASK; + if (lr_signals_eoi(lr_val)) { + gzvm_notify_acked_irq(vcpu->gzvm, + vintid - VGIC_NR_PRIVATE_IRQS); + } } } diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c index cceaa532c2ce..e4bea025e45e 100644 --- a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c @@ -386,6 +386,15 @@ static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl, ret = gzvm_ioeventfd(gzvm, &data); break; } + case GZVM_IRQFD: { + struct gzvm_irqfd data; + + ret = -EFAULT; + if (copy_from_user(&data, argp, sizeof(data))) + goto out; + ret = gzvm_irqfd(gzvm, &data); + break; + } case GZVM_ENABLE_CAP: { struct gzvm_enable_cap cap; @@ -490,13 +499,20 @@ static struct gzvm *gzvm_create_vm(unsigned long vm_type) mutex_init(&gzvm->lock); INIT_LIST_HEAD(&gzvm->devices); mutex_init(&gzvm->irq_lock); + ret = gzvm_init_ioeventfd(gzvm); if (ret) { pr_err("Failed to initialize ioeventfd\n"); goto err; } - pr_info("VM-%u is created\n", gzvm->vm_id); + ret = gzvm_init_irqfd(gzvm); + if (ret) { + pr_err("Failed to initialize irqfd\n"); + goto err; + } + + pr_info("VM-%u is created\n", gzvm->vm_id); mutex_lock(&gzvm_list_lock); list_add(&gzvm->vm_list, &gzvm_list); mutex_unlock(&gzvm_list_lock); diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h index 6dbaddd77ec7..3af9d148042f 100644 --- a/include/uapi/linux/gzvm_common.h +++ b/include/uapi/linux/gzvm_common.h @@ -193,6 +193,24 @@ struct gzvm_irq_level { #define GZVM_IRQ_LINE _IOW(GZVM_IOC_MAGIC, 0x61, \ struct gzvm_irq_level) +#define GZVM_IRQFD_FLAG_DEASSIGN (1 << 0) +/* + * GZVM_IRQFD_FLAG_RESAMPLE indicates resamplefd is valid and specifies + * the irqfd to operate in resampling mode for level triggered interrupt + * emulation. + */ +#define GZVM_IRQFD_FLAG_RESAMPLE (1 << 1) + +struct gzvm_irqfd { + __u32 fd; + __u32 gsi; + __u32 flags; + __u32 resamplefd; + __u8 pad[16]; +}; +#define GZVM_IRQFD _IOW(GZVM_IOC_MAGIC, 0x76, \ + struct gzvm_irqfd) + enum { gzvm_ioeventfd_flag_nr_datamatch, gzvm_ioeventfd_flag_nr_pio,