From patchwork Mon Sep 8 23:01:59 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Schopp X-Patchwork-Id: 37018 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ig0-f197.google.com (mail-ig0-f197.google.com [209.85.213.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 4423E20491 for ; Mon, 8 Sep 2014 23:04:18 +0000 (UTC) Received: by mail-ig0-f197.google.com with SMTP id r2sf73954862igi.8 for ; Mon, 08 Sep 2014 16:04:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:subject:from:to:date:message-id :user-agent:mime-version:cc:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :content-type:content-transfer-encoding; bh=jWPw7ejx6IeozRh2t+07UEKsQX7yIKb1QAHUompq/yI=; b=mXpbUaifqTZehi3PrHHosVurp6uSCOQQazeEkCVcpMLaTuA9imp5ijU+4y5gxfpk/S TvytDTZSXTAYuNip9s37kR53XqxMQljRQ2O/345VIvyKacA6lPz9rthh2VgA+g3LTeFE mYkxmDfJkIOjIdc/zEJeM3STQdC2tIqOLD4eqz+gNxjjMn+NOmipScJkkxQhgNWxkgeZ lqOviQ+n115PBuD4YsTUb1OwBvg+xOcTxguZx+v91VdSRgmYBH3dhKzE5Zlgg2IyN4Gg rDMFdAgT87AKCLIJNJWD4NfFNLvSD3o8yidBr2IADkqu2n8ApMdUuSfJ1ZWDm3/jdLvP bYXQ== X-Gm-Message-State: ALoCoQkXDpEpOdSLYL7N9pG/TVSNQ2NOyef9cEml+wf0umLHTDQREAL+rKQ2T6qdERLHphFmRDbo X-Received: by 10.42.121.72 with SMTP id i8mr20180790icr.10.1410217457853; Mon, 08 Sep 2014 16:04:17 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.104.18 with SMTP id z18ls1881293qge.0.gmail; Mon, 08 Sep 2014 16:04:17 -0700 (PDT) X-Received: by 10.221.68.66 with SMTP id xx2mr27855889vcb.1.1410217457710; Mon, 08 Sep 2014 16:04:17 -0700 (PDT) Received: from mail-vc0-f176.google.com (mail-vc0-f176.google.com [209.85.220.176]) by mx.google.com with ESMTPS id xn5si4533330vcb.95.2014.09.08.16.04.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 08 Sep 2014 16:04:17 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.176 as permitted sender) client-ip=209.85.220.176; Received: by mail-vc0-f176.google.com with SMTP id la4so149926vcb.21 for ; Mon, 08 Sep 2014 16:04:17 -0700 (PDT) X-Received: by 10.52.61.99 with SMTP id o3mr3986657vdr.46.1410217457534; Mon, 08 Sep 2014 16:04:17 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.45.67 with SMTP id uj3csp201463vcb; Mon, 8 Sep 2014 16:04:16 -0700 (PDT) X-Received: by 10.66.120.99 with SMTP id lb3mr8905471pab.152.1410217456587; Mon, 08 Sep 2014 16:04:16 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id ow5si20013895pdb.143.2014.09.08.16.04.16 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Sep 2014 16:04:16 -0700 (PDT) Received-SPF: none (google.com: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org does not designate permitted sender hosts) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XR7x6-00068V-Eq; Mon, 08 Sep 2014 23:02:28 +0000 Received: from mail-by2on0131.outbound.protection.outlook.com ([207.46.100.131] helo=na01-by2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XR7x2-00065j-TQ for linux-arm-kernel@lists.infradead.org; Mon, 08 Sep 2014 23:02:26 +0000 Received: from BY1PR0201CA0034.namprd02.prod.outlook.com (25.160.191.172) by CO1PR02MB047.namprd02.prod.outlook.com (10.242.163.24) with Microsoft SMTP Server (TLS) id 15.0.1024.12; Mon, 8 Sep 2014 23:02:02 +0000 Received: from BY2FFO11FD012.protection.gbl (2a01:111:f400:7c0c::102) by BY1PR0201CA0034.outlook.office365.com (2a01:111:e400:4814::44) with Microsoft SMTP Server (TLS) id 15.0.1024.12 via Frontend Transport; Mon, 8 Sep 2014 23:02:02 +0000 Received: from atltwp02.amd.com (165.204.84.222) by BY2FFO11FD012.mail.protection.outlook.com (10.1.14.130) with Microsoft SMTP Server id 15.0.1019.14 via Frontend Transport; Mon, 8 Sep 2014 23:02:01 +0000 X-WSS-ID: 0NBLUN7-08-8E4-02 X-M-MSG: Received: from satlvexedge01.amd.com (satlvexedge01.amd.com [10.177.96.28]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by atltwp02.amd.com (Axway MailGate 5.3.1) with ESMTPS id 25955D160A6; Mon, 8 Sep 2014 18:01:54 -0500 (CDT) Received: from SATLEXDAG01.amd.com (10.181.40.3) by satlvexedge01.amd.com (10.177.96.28) with Microsoft SMTP Server (TLS) id 14.3.195.1; Mon, 8 Sep 2014 18:02:00 -0500 Received: from joelaarch64.amd.com (10.180.168.240) by SATLEXDAG01.amd.com (10.181.40.3) with Microsoft SMTP Server (TLS) id 14.3.195.1; Mon, 8 Sep 2014 19:01:58 -0400 Subject: [PATCH 6] From nobody Wed Sep 3 13:22:18 2014 From: Joel Schopp To: , , Date: Mon, 8 Sep 2014 23:01:59 +0000 Message-ID: <20140908230158.1302.13372.stgit@joelaarch64.amd.com> User-Agent: StGit/0.17.1-4-g4a0c1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:165.204.84.222; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019018)(6009001)(428002)(189002)(199003)(19580395003)(83322001)(76482001)(19580405001)(15975445006)(79102001)(85852003)(83072002)(77982001)(68736004)(86362001)(2171001)(47776003)(87936001)(20776003)(84676001)(97746001)(64706001)(575784001)(83506001)(81342001)(23676002)(102836001)(46102001)(97736003)(44976005)(4396001)(103116003)(33646002)(74662001)(81542001)(74502001)(31966008)(77096002)(50986999)(92566001)(2201001)(80022001)(50466002)(54356999)(106466001)(21056001)(92726001)(512874002)(101416001)(90102001)(107046002)(95666004)(99396002)(229853001)(105586002)(53416004)(85306004)(71626003); DIR:OUT; SFP:1102; SCL:1; SRVR:CO1PR02MB047; H:atltwp02.amd.com; FPR:; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:;UriScan:; X-Forefront-PRVS: 03283976A6 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-OriginatorOrg: amd4.onmicrosoft.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140908_160225_002378_C9DC1BDA X-CRM114-Status: GOOD ( 21.30 ) X-Spam-Score: -1.8 (-) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-1.8 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [207.46.100.131 listed in list.dnswl.org] -1.8 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [207.46.100.131 listed in wl.mailspike.net] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record Cc: Marc Zyngier , Sungjinn Chung , Jungseok Lee , Christoffer Dall X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: joel.schopp@amd.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.176 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Subject: [PATCH] From nobody Tue Sep 2 17:49:46 2014 From: Joel Schopp To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Sungjinn Chung , Christoffer Dall , Jungseok Lee Bcc: ssg.sos.patches@amd.com Date: Wed, 03 Sep 2014 13:22:18 -0500 Message-ID: <20140903182218.2879.80647.stgit@joelaarch64.amd.com> User-Agent: StGit/0.17.1-4-g4a0c1 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Subject: [PATCH] arm64: fix VTTBR_BADDR_MASK From: Joel Schopp To: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Sungjinn Chung , Christoffer Dall , Jungseok Lee Bcc: ssg.sos.patches@amd.com Date: Tue, 02 Sep 2014 17:49:46 -0500 Message-ID: <20140902224946.1695.38669.stgit@joelaarch64.amd.com> User-Agent: StGit/0.17.1-4-g4a0c1 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit The current VTTBR_BADDR_MASK only masks 39 bits, which is broken on current systems. Rather than just add a bit it seems like a good time to also set things at run-time instead of compile time to accomodate more hardware. This patch sets TCR_EL2.PS, VTCR_EL2.T0SZ and vttbr_baddr_mask in runtime, not compile time. In ARMv8, EL2 physical address size (TCR_EL2.PS) and stage2 input address size (VTCR_EL2.T0SZE) cannot be determined in compile time since they depend on hardware capability. According to Table D4-23 and Table D4-25 in ARM DDI 0487A.b document, vttbr_x is calculated using different fixed values with consideration of T0SZ, granule size and the level of translation tables. Therefore, vttbr_baddr_mask should be determined dynamically. Changes since v5: Fixed declaration of vttbr_baddr_mask to not create multiple instances Refactored return codes based on feedback For 32 bit included kvm_arm.h in kvm_mmu.h to explictly pick up VTTBR_BADDR_MASK Changes since v4: More minor cleanups from review Moved some functions into headers Added runtime check in kvm_alloc_stage2_pgd Changes since v3: Another rebase Addressed minor comments from v2 Changes since v2: Rebased on https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm.git next branch Changes since v1: Rebased fix on Jungseok Lee's patch https://lkml.org/lkml/2014/5/12/189 to provide better long term fix. Updated that patch to log error instead of silently fail on unaligned vttbr. Cc: Marc Zyngier Cc: Christoffer Dall Cc: Sungjinn Chung Signed-off-by: Jungseok Lee Signed-off-by: Joel Schopp --- arch/arm/include/asm/kvm_mmu.h | 13 +++++++ arch/arm/kvm/arm.c | 23 +++++++++++- arch/arm64/include/asm/kvm_arm.h | 17 ++------- arch/arm64/include/asm/kvm_mmu.h | 75 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp-init.S | 20 +++++++--- 5 files changed, 126 insertions(+), 22 deletions(-) diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index 5cc0b0f..03a08bb 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -21,6 +21,7 @@ #include #include +#include /* * We directly use the kernel VA for the HYP, as we can directly share @@ -178,6 +179,18 @@ static inline void coherent_cache_guest_page(struct kvm_vcpu *vcpu, hva_t hva, void stage2_flush_vm(struct kvm *kvm); +static inline int kvm_get_phys_addr_shift(void) +{ + return KVM_PHYS_SHIFT; +} + + +static inline u32 get_vttbr_baddr_mask(void) +{ + return VTTBR_BADDR_MASK; +} + + #endif /* !__ASSEMBLY__ */ #endif /* __ARM_KVM_MMU_H__ */ diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index a99e0cd..d0fca8f 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -61,6 +62,12 @@ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); static u8 kvm_next_vmid; static DEFINE_SPINLOCK(kvm_vmid_lock); +#ifdef CONFIG_ARM64 +static u64 vttbr_baddr_mask; +#else +static u32 vttbr_baddr_mask; +#endif + static bool vgic_present; static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu) @@ -429,8 +436,14 @@ static void update_vttbr(struct kvm *kvm) /* update vttbr to be used with the new vmid */ pgd_phys = virt_to_phys(kvm->arch.pgd); vmid = ((u64)(kvm->arch.vmid) << VTTBR_VMID_SHIFT) & VTTBR_VMID_MASK; - kvm->arch.vttbr = pgd_phys & VTTBR_BADDR_MASK; - kvm->arch.vttbr |= vmid; + + /* + * If the VTTBR isn't aligned there is something wrong with the system + * or kernel. + */ + BUG_ON(pgd_phys & ~vttbr_baddr_mask); + + kvm->arch.vttbr = pgd_phys | vmid; spin_unlock(&kvm_vmid_lock); } @@ -1015,6 +1028,12 @@ int kvm_arch_init(void *opaque) } } + vttbr_baddr_mask = get_vttbr_baddr_mask(); + if (vttbr_baddr_mask == ~0) { + kvm_err("Cannot set vttbr_baddr_mask\n"); + return -EINVAL; + } + cpu_notifier_register_begin(); err = init_hyp_mode(); diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index cc83520..ff4a4fa 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -95,7 +95,6 @@ /* TCR_EL2 Registers bits */ #define TCR_EL2_TBI (1 << 20) #define TCR_EL2_PS (7 << 16) -#define TCR_EL2_PS_40B (2 << 16) #define TCR_EL2_TG0 (1 << 14) #define TCR_EL2_SH0 (3 << 12) #define TCR_EL2_ORGN0 (3 << 10) @@ -104,8 +103,6 @@ #define TCR_EL2_MASK (TCR_EL2_TG0 | TCR_EL2_SH0 | \ TCR_EL2_ORGN0 | TCR_EL2_IRGN0 | TCR_EL2_T0SZ) -#define TCR_EL2_FLAGS (TCR_EL2_PS_40B) - /* VTCR_EL2 Registers bits */ #define VTCR_EL2_PS_MASK (7 << 16) #define VTCR_EL2_TG0_MASK (1 << 14) @@ -120,36 +117,28 @@ #define VTCR_EL2_SL0_MASK (3 << 6) #define VTCR_EL2_SL0_LVL1 (1 << 6) #define VTCR_EL2_T0SZ_MASK 0x3f -#define VTCR_EL2_T0SZ_40B 24 +#define VTCR_EL2_T0SZ(bits) (64 - (bits)) #ifdef CONFIG_ARM64_64K_PAGES /* * Stage2 translation configuration: - * 40bits output (PS = 2) - * 40bits input (T0SZ = 24) * 64kB pages (TG0 = 1) * 2 level page tables (SL = 1) */ #define VTCR_EL2_FLAGS (VTCR_EL2_TG0_64K | VTCR_EL2_SH0_INNER | \ VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \ - VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B) -#define VTTBR_X (38 - VTCR_EL2_T0SZ_40B) + VTCR_EL2_SL0_LVL1) #else /* * Stage2 translation configuration: - * 40bits output (PS = 2) - * 40bits input (T0SZ = 24) * 4kB pages (TG0 = 0) * 3 level page tables (SL = 1) */ #define VTCR_EL2_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SH0_INNER | \ VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \ - VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B) -#define VTTBR_X (37 - VTCR_EL2_T0SZ_40B) + VTCR_EL2_SL0_LVL1) #endif -#define VTTBR_BADDR_SHIFT (VTTBR_X - 1) -#define VTTBR_BADDR_MASK (((1LLU << (40 - VTTBR_X)) - 1) << VTTBR_BADDR_SHIFT) #define VTTBR_VMID_SHIFT (48LLU) #define VTTBR_VMID_MASK (0xffLLU << VTTBR_VMID_SHIFT) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 8e138c7..1c70b2f 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -167,5 +167,80 @@ static inline void coherent_cache_guest_page(struct kvm_vcpu *vcpu, hva_t hva, void stage2_flush_vm(struct kvm *kvm); +/* + * ARMv8 64K architecture limitations: + * 16 <= T0SZ <= 21 is valid under 3 level of translation tables + * 18 <= T0SZ <= 34 is valid under 2 level of translation tables + * 31 <= T0SZ <= 39 is valid under 1 level of transltaion tables + * + * ARMv8 4K architecture limitations: + * 16 <= T0SZ <= 24 is valid under 4 level of translation tables + * 21 <= T0SZ <= 33 is valid under 3 level of translation tables + * 30 <= T0SZ <= 39 is valid under 2 level of translation tables + * + * For 4K pages we only support 3 or 4 level, giving T0SZ a range of 16 to 33. + * For 64K pages we only support 2 or 3 level, giving T0SZ a range of 16 to 34. + * + * See Table D4-23 and Table D4-25 in ARM DDI 0487A.b to figure out + * the origin of the hardcoded values, 38 and 37. + */ + +#ifdef CONFIG_ARM64_64K_PAGES +static inline int t0sz_to_vttbr_x(int t0sz) +{ + if (t0sz < 16 || t0sz > 34) { + kvm_err("Cannot support %d-bit address space\n", 64 - t0sz); + return -EINVAL; + } + + return 38 - t0sz; +} +#else /* 4K pages */ +static inline int t0sz_to_vttbr_x(int t0sz) +{ + if (t0sz < 16 || t0sz > 33) { + kvm_err("Cannot support %d-bit address space\n", 64 - t0sz); + return -EINVAL; + } + return 37 - t0sz; +} +#endif +static inline int kvm_get_phys_addr_shift(void) +{ + int pa_range = read_cpuid(ID_AA64MMFR0_EL1) & 0xf; + + switch (pa_range) { + case 0: return 32; + case 1: return 36; + case 2: return 40; + case 3: return 42; + case 4: return 44; + case 5: return 48; + default: + BUG(); + return 0; + } +} + +/** + * get_vttbr_baddr_mask - get mask value for vttbr base address + * + * In ARMv8, vttbr_baddr_mask cannot be determined in compile time since the + * stage2 input address size depends on hardware capability. Thus, we first + * need to read ID_AA64MMFR0_EL1.PARange and then set vttbr_baddr_mask with + * consideration of both the granule size and the level of translation tables. + */ +static inline u64 get_vttbr_baddr_mask(void) +{ + int t0sz, vttbr_x; + + t0sz = VTCR_EL2_T0SZ(kvm_get_phys_addr_shift()); + vttbr_x = t0sz_to_vttbr_x(t0sz); + if (vttbr_x < 0) + return ~0; + return GENMASK_ULL(48, (vttbr_x - 1)); + +} + #endif /* __ASSEMBLY__ */ #endif /* __ARM64_KVM_MMU_H__ */ diff --git a/arch/arm64/kvm/hyp-init.S b/arch/arm64/kvm/hyp-init.S index c319116..fa7e67e 100644 --- a/arch/arm64/kvm/hyp-init.S +++ b/arch/arm64/kvm/hyp-init.S @@ -63,17 +63,21 @@ __do_hyp_init: mrs x4, tcr_el1 ldr x5, =TCR_EL2_MASK and x4, x4, x5 - ldr x5, =TCR_EL2_FLAGS - orr x4, x4, x5 - msr tcr_el2, x4 - - ldr x4, =VTCR_EL2_FLAGS /* * Read the PARange bits from ID_AA64MMFR0_EL1 and set the PS bits in - * VTCR_EL2. + * TCR_EL2 and both PS bits and T0SZ bits in VTCR_EL2. */ mrs x5, ID_AA64MMFR0_EL1 bfi x4, x5, #16, #3 + msr tcr_el2, x4 + + ldr x4, =VTCR_EL2_FLAGS + bfi x4, x5, #16, #3 + and x5, x5, #0xf + adr x6, t0sz + add x6, x6, x5, lsl #2 + ldr w5, [x6] + orr x4, x4, x5 msr vtcr_el2, x4 mrs x4, mair_el1 @@ -113,6 +117,10 @@ target: /* We're now in the trampoline code, switch page tables */ /* Hello, World! */ eret + +t0sz: + .word VTCR_EL2_T0SZ(32), VTCR_EL2_T0SZ(36), VTCR_EL2_T0SZ(40) + .word VTCR_EL2_T0SZ(42), VTCR_EL2_T0SZ(44), VTCR_EL2_T0SZ(48) ENDPROC(__kvm_hyp_init) .ltorg