From patchwork Mon May 22 07:13:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 750C3C7EE23 for ; Mon, 22 May 2023 07:15:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232079AbjEVHPJ (ORCPT ); Mon, 22 May 2023 03:15:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232476AbjEVHOq (ORCPT ); Mon, 22 May 2023 03:14:46 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 604E91A4; Mon, 22 May 2023 00:14:43 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E6DD661DDD; Mon, 22 May 2023 07:14:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BDECC4339E; Mon, 22 May 2023 07:14:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739682; bh=6vIoJTudACrMHaUZpEmA9kKzuuAA0lBBUtfzLDgP/Vg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZIZ8oOkyrgL8CeF7w9z7a27wkdrj1AdQ7rNdEm+nMnPCxkpbMJp3dv9SKJSsVZHCe 9Dp9Y85TJ9fhl4i+6uP3Kyfiu1IroKotwfmWNqltXEcrZsHAaUa/rv/BP3gMW9BimW 2v+8PSMXuLm8bykkzdYluJunTzGZk9a1Z6MBkVqYnbQwrL6j4/epy/srH1AcwYjUnr SIQR8RbJ19LebnRxWhJz77XqdVCxmKI//AWIHhG1yZVGQqNe3prgs5jJC36PgB8oK8 M6MOoEZ1m826M4xN/jNc93INx1dqXb1EnGWY75XNHEj1Bq12ZMp3r5Dq2BCXzhWHto 0L/J/fHdWqUJg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 02/21] x86/efistub: Simplify and clean up handover entry code Date: Mon, 22 May 2023 09:13:56 +0200 Message-Id: <20230522071415.501717-3-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7508; i=ardb@kernel.org; h=from:subject; bh=6vIoJTudACrMHaUZpEmA9kKzuuAA0lBBUtfzLDgP/Vg=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbTP9P8dlXrbdVNQ6tbFDZuGLnA+uQxpZzGrtOsQrMb T9Tu2xlRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZhIfQwjw8/E2zvPcc9R/crF 8UXA+rp34MHlgVdc755iU7r77OyNKUWMDHc97zc+c3Ga84015suXHx9XMa5zUNrm/Mzvyu9v/+T ZjzADAA== X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Now that the EFI entry code in assembler is only used by the optional and deprecated EFI handover protocol, and given that the EFI stub C code no longer returns to it, most of it can simply be dropped. While at it, clarify the symbol naming, by merging efi_main() and efi_stub_entry(), making the latter the shared entry point for all different boot modes that enter via the EFI stub. The efi32_stub_entry() and efi64_stub_entry() names are referenced explicitly by the tooling that populates the setup header, so these must be retained, but can be emitted as aliases of efi_stub_entry() where appropriate. Signed-off-by: Ard Biesheuvel --- Documentation/arch/x86/boot.rst | 2 +- arch/x86/boot/compressed/efi_mixed.S | 22 +++++++++++--------- arch/x86/boot/compressed/head_32.S | 11 ---------- arch/x86/boot/compressed/head_64.S | 12 ++--------- drivers/firmware/efi/libstub/x86-stub.c | 20 ++++++++++++++---- 5 files changed, 31 insertions(+), 36 deletions(-) diff --git a/Documentation/arch/x86/boot.rst b/Documentation/arch/x86/boot.rst index 33520ecdb37abfda..cdbca15a4fc23833 100644 --- a/Documentation/arch/x86/boot.rst +++ b/Documentation/arch/x86/boot.rst @@ -1417,7 +1417,7 @@ execution context provided by the EFI firmware. The function prototype for the handover entry point looks like this:: - efi_main(void *handle, efi_system_table_t *table, struct boot_params *bp) + efi_stub_entry(void *handle, efi_system_table_t *table, struct boot_params *bp) 'handle' is the EFI image handle passed to the boot loader by the EFI firmware, 'table' is the EFI system table - these are the first two diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S index 4ca70bf93dc0bdcd..dcc562c8f7f35162 100644 --- a/arch/x86/boot/compressed/efi_mixed.S +++ b/arch/x86/boot/compressed/efi_mixed.S @@ -26,8 +26,8 @@ * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode() * is the first thing that runs after switching to long mode. Depending on * whether the EFI handover protocol or the compat entry point was used to - * enter the kernel, it will either branch to the 64-bit EFI handover - * entrypoint at offset 0x390 in the image, or to the 64-bit EFI PE/COFF + * enter the kernel, it will either branch to the common 64-bit EFI stub + * entrypoint efi_stub_entry() directly, or via the 64-bit EFI PE/COFF * entrypoint efi_pe_entry(). In the former case, the bootloader must provide a * struct bootparams pointer as the third argument, so the presence of such a * pointer is used to disambiguate. @@ -37,21 +37,23 @@ * | efi32_pe_entry |---->| | | +-----------+--+ * +------------------+ | | +------+----------------+ | * | startup_32 |---->| startup_64_mixed_mode | | - * +------------------+ | | +------+----------------+ V - * | efi32_stub_entry |---->| | | +------------------+ - * +------------------+ +------------+ +---->| efi64_stub_entry | - * +-------------+----+ - * +------------+ +----------+ | - * | startup_64 |<----| efi_main |<--------------+ - * +------------+ +----------+ + * +------------------+ | | +------+----------------+ | + * | efi32_stub_entry |---->| | | | + * +------------------+ +------------+ | | + * V | + * +------------+ +----------------+ | + * | startup_64 |<----| efi_stub_entry |<--------+ + * +------------+ +----------------+ */ SYM_FUNC_START(startup_64_mixed_mode) lea efi32_boot_args(%rip), %rdx mov 0(%rdx), %edi mov 4(%rdx), %esi +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL mov 8(%rdx), %edx // saved bootparams pointer test %edx, %edx - jnz efi64_stub_entry + jnz efi_stub_entry +#endif /* * efi_pe_entry uses MS calling convention, which requires 32 bytes of * shadow space on the stack even if all arguments are passed in diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 987ae727cf9f0d04..8876ffe30e9a4819 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -150,17 +150,6 @@ SYM_FUNC_START(startup_32) jmp *%eax SYM_FUNC_END(startup_32) -#ifdef CONFIG_EFI_STUB -SYM_FUNC_START(efi32_stub_entry) - add $0x4, %esp - movl 8(%esp), %esi /* save boot_params pointer */ - call efi_main - /* efi_main returns the possibly relocated address of startup_32 */ - jmp *%eax -SYM_FUNC_END(efi32_stub_entry) -SYM_FUNC_ALIAS(efi_stub_entry, efi32_stub_entry) -#endif - .text SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 03c4328a88cbd5d0..71c1f40a7ac067b9 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -523,19 +523,11 @@ trampoline_return: jmp *%rax SYM_CODE_END(startup_64) -#ifdef CONFIG_EFI_STUB -#ifdef CONFIG_EFI_HANDOVER_PROTOCOL +#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL) .org 0x390 -#endif SYM_FUNC_START(efi64_stub_entry) - and $~0xf, %rsp /* realign the stack */ - movq %rdx, %rbx /* save boot_params pointer */ - call efi_main - movq %rbx,%rsi - leaq rva(startup_64)(%rax), %rax - jmp *%rax + jmp efi_stub_entry SYM_FUNC_END(efi64_stub_entry) -SYM_FUNC_ALIAS(efi_stub_entry, efi64_stub_entry) #endif .text diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 095aaa8b0ee30fb9..d6a376e52cbe1399 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -782,9 +782,9 @@ static void __noreturn enter_kernel(unsigned long kernel_addr, * On success, we jump to the relocated kernel directly and never return. * On failure, we exit to the firmware via efi_exit instead of returning. */ -asmlinkage unsigned long efi_main(efi_handle_t handle, - efi_system_table_t *sys_table_arg, - struct boot_params *boot_params) +void __noreturn efi_stub_entry(efi_handle_t handle, + efi_system_table_t *sys_table_arg, + struct boot_params *boot_params) { unsigned long bzimage_addr = (unsigned long)startup_32; unsigned long buffer_start, buffer_end; @@ -928,7 +928,19 @@ asmlinkage unsigned long efi_main(efi_handle_t handle, enter_kernel(bzimage_addr, boot_params); fail: - efi_err("efi_main() failed!\n"); + efi_err("efi_stub_entry() failed!\n"); efi_exit(handle, status); } + +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL +#ifndef CONFIG_EFI_MIXED +extern __alias(efi_stub_entry) +void efi32_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, + struct boot_params *boot_params); + +extern __alias(efi_stub_entry) +void efi64_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, + struct boot_params *boot_params); +#endif +#endif From patchwork Mon May 22 07:13:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6FBCC77B73 for ; Mon, 22 May 2023 07:15:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229647AbjEVHPM (ORCPT ); Mon, 22 May 2023 03:15:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232523AbjEVHOx (ORCPT ); Mon, 22 May 2023 03:14:53 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A882594; Mon, 22 May 2023 00:14:51 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3BF3961DC4; Mon, 22 May 2023 07:14:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EA4DBC4339E; Mon, 22 May 2023 07:14:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739690; bh=GI53KZcq3ZI3RxFd9aG0XytLXRdQ1EhHXwVomNHohgI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=otLwxEON9iJ+ZI6BCN9YAbKtZnBDHq6IdNPb/4XDqFwgee1+vXR5i5ac88qs0acoQ G67JEC/bGL1GStjgIDfG9HAmyx/PR1byEARlw3pScOH7bG8LkUxtPAtzLpTHfM+qWP Y26SWLmoKv2OwL7OrRwGQZELJUsfsCyHSTS6tX1nndOOdQCDfJ8DQWQo4kND9djWVG lWWWGEHVsvp8cRhPBspQpkgCys9yZyJRXJcA3Xqs3KzmNwViA/BXoZAH5kvrvQtWEz Y2Yf1PkKKKMebsw9TzPo+SEJrokn+xLLvF9LhcKe6zeq1EOXk8HhnsXVHxlM5MIWC+ 0jUyU+BU7vyjA== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 04/21] x86/efistub: Clear BSS in EFI handover protocol entrypoint Date: Mon, 22 May 2023 09:13:58 +0200 Message-Id: <20230522071415.501717-5-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2971; i=ardb@kernel.org; h=from:subject; bh=GI53KZcq3ZI3RxFd9aG0XytLXRdQ1EhHXwVomNHohgI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzCh+k2N2zVP21WnK71riE97Oi5L3ShPeM9esfPuSx i2fDRM7SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwET2GTIyXFu947S07Bq/pJS9 RRvOWewM0mj2ynzyvyB8ooh2R2fKX4b/LlJ2N3KZDJw8xXJT7Go7di1Kfdu/efIK3oNZKcf57H7 wAAA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org The so-called EFI handover protocol is value-add from the distros that permits a loader to simply copy a PE kernel image into memory and call an alternative entrypoint that is described by an embedded boot_params structure. Most implementations of this protocol do not bother to check the PE header for minimum alignment, section placement, etc, and therefore also don't clear the image's BSS, or even allocate enough memory for it. Allocating more memory on the fly is rather difficult, but at least clear the BSS region explicitly when entering in this manner, so that the EFI stub code does not get confused by global variables that were not zero-initialized correctly. When booting in mixed mode, this BSS clearing must occur before any global state is created, so clear it in the 32-bit asm entry point. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/efi_mixed.S | 14 +++++++++++++- drivers/firmware/efi/libstub/x86-stub.c | 13 +++++++++++-- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S index 9308b595f6f0a5de..8a02a151806df14c 100644 --- a/arch/x86/boot/compressed/efi_mixed.S +++ b/arch/x86/boot/compressed/efi_mixed.S @@ -142,6 +142,18 @@ SYM_FUNC_END(__efi64_thunk) .code32 #ifdef CONFIG_EFI_HANDOVER_PROTOCOL SYM_FUNC_START(efi32_stub_entry) + call 1f +1: popl %ecx + + /* Clear BSS */ + xorl %eax, %eax + leal (_bss - 1b)(%ecx), %edi + leal (_ebss - 1b)(%ecx), %ecx + subl %edi, %ecx + shrl $2, %ecx + cld + rep stosl + add $0x4, %esp /* Discard return address */ popl %ecx popl %edx @@ -334,7 +346,7 @@ SYM_FUNC_END(efi32_pe_entry) .org efi32_stub_entry + 0x200 .code64 SYM_FUNC_START_NOALIGN(efi64_stub_entry) - jmp efi_stub_entry + jmp efi_handover_entry SYM_FUNC_END(efi64_stub_entry) #endif diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index d6a376e52cbe1399..d010448dffb12cb8 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -934,12 +934,21 @@ void __noreturn efi_stub_entry(efi_handle_t handle, } #ifdef CONFIG_EFI_HANDOVER_PROTOCOL +void efi_handover_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, + struct boot_params *boot_params) +{ + extern char _bss[], _ebss[]; + + memset(_bss, 0, _ebss - _bss); + efi_stub_entry(handle, sys_table_arg, boot_params); +} + #ifndef CONFIG_EFI_MIXED -extern __alias(efi_stub_entry) +extern __alias(efi_handover_entry) void efi32_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, struct boot_params *boot_params); -extern __alias(efi_stub_entry) +extern __alias(efi_handover_entry) void efi64_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, struct boot_params *boot_params); #endif From patchwork Mon May 22 07:13:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55D2AC7EE2E for ; Mon, 22 May 2023 07:15:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232198AbjEVHPI (ORCPT ); Mon, 22 May 2023 03:15:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232537AbjEVHO4 (ORCPT ); Mon, 22 May 2023 03:14:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7113BA; Mon, 22 May 2023 00:14:55 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 61DBF61DDD; Mon, 22 May 2023 07:14:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C3BFC4339C; Mon, 22 May 2023 07:14:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739694; bh=ygI81p6SqewQoIn6tCgoZLUH2Qo5tPVgo3BNvwGSS3U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ahdZuLYw1TmYdmQL7lxGHEBiiBSANf79+lkAs61O4z58clTFkmUWH58Fsbdo33R3z f1fMye2kQbMk8gjq+efFzOBdtG5qIi9Ea8qvj08KORTxU9WKJePL9KJleO948dlGdd 1Q2JtizxxQVY0RkrEFWjyvwxepYUgaEaWVEH5w+BwdLHBpLNBkCSAGQ3YOS9BBBXND t1aep4slelOSfi3wf6U0TVJzig67BQfHew9FDQm0MMUJENIwTpNCZq9ILKsv1tu+7O bmRmAg52hOVvDqquTz9yxja6QwMK8OKV/WL0VdvLNQOEFNTWlhyGyXccFgxOoALZpG IlAcnhCBhUjBg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 05/21] x86/decompressor: Use proper sequence to take the address of the GOT Date: Mon, 22 May 2023 09:13:59 +0200 Message-Id: <20230522071415.501717-6-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1770; i=ardb@kernel.org; h=from:subject; bh=ygI81p6SqewQoIn6tCgoZLUH2Qo5tPVgo3BNvwGSS3U=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzHhfjGz12WInw7ydqfPDJ7Ukb5i9b7FGVn2Z7oyqH cdfVGp0lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIm0KTIyrF17+9OtoPo3ZXV+ 0yM+ap06MX1Rjqd/vtzOzMsersb9IYwMX2Pv6k44LDOnZ8bi5ZeDUs/EP/19p29RxDaZCRYi8/q LmAE= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org The 32-bit decompressor does not actually use a global offset table (GOT), but as is common for 32-bit position independent code, it uses the magic symbol _GLOBAL_OFFSET_TABLE_ as an anchor from which to derive the actual runtime addresses of other symbols, using special @GOTOFF symbol references that are resolved at link time, and populated with the distance between the address of the magic _GLOBAL_OFFSET_TABLE_ anchor and the address of the symbol in question. This means _GLOBAL_OFFSET_TABLE_ is the only symbol whose actual runtime address needs to be determined explicitly, which is one of the first things that happens in startup_32. However, it does so by taking the absolute address via the immediate field of an ADD instruction (plus a small offset), which seems to defeat the point. Fortunately, the assembler knows that _GLOBAL_OFFSET_TABLE_ is magic, and emits a special relative relocation instead, and so the resulting code works as expected. However, this is not obvious for someone reading the code, and the use of LEA with an explicit relative addend is more idiomatic so use that instead. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_32.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 8876ffe30e9a4819..3530465b5b85ccf3 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -58,7 +58,7 @@ SYM_FUNC_START(startup_32) leal (BP_scratch+4)(%esi), %esp call 1f 1: popl %edx - addl $_GLOBAL_OFFSET_TABLE_+(.-1b), %edx + leal (_GLOBAL_OFFSET_TABLE_ - 1b)(%edx), %edx /* Load new GDT */ leal gdt@GOTOFF(%edx), %eax From patchwork Mon May 22 07:14:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A04FC7EE2F for ; Mon, 22 May 2023 07:15:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231928AbjEVHPH (ORCPT ); Mon, 22 May 2023 03:15:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232545AbjEVHPA (ORCPT ); Mon, 22 May 2023 03:15:00 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF9ECA1; Mon, 22 May 2023 00:14:59 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8671261DD4; Mon, 22 May 2023 07:14:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41F97C433D2; Mon, 22 May 2023 07:14:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739699; bh=XH7auhXZovyO0ZjhIGuK6rXf321+XRWDmJGikLNf51o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a7KdTq8PPZcLUMpaFnJwt9owjwLIw/qIfHVEF5sSQogAuB/WRsIxrAn774VuPIUJx N+FSlb/Akd4n2Us5eUEVyTfgY8jJChhAa7cRZWByVDu8c6ITJzdfi43CIiuM6/M1cs m00dGG46N2zKKp8HsTzugxFheiOff5FDccKpsUy2aqnG5Q1eoc5LSPnek94UTWhMmp 9VKR9uoZd59smBJfAndQiWoU5VfTupBjJ0IL6/buDPYmBOkayLh3NNaRtJr0/yj/ON FK1Z9YPDHdQjueJlsRk4sP9wHGA/L/SbfXFGUZ62SJoZsCvacM8D5SsQJbADTMeWeu cx6Xd2E5oTOAw== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 06/21] x86/decompressor: Store boot_params pointer in callee save register Date: Mon, 22 May 2023 09:14:00 +0200 Message-Id: <20230522071415.501717-7-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3619; i=ardb@kernel.org; h=from:subject; bh=XH7auhXZovyO0ZjhIGuK6rXf321+XRWDmJGikLNf51o=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzGRRpJHd1RnWG0UnzPsfEPSq6/POZO+LSS9tdY8IO KjNVzfuKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABPZns3wzzbrXmbQ4X5Z9d5l m5nnXFj09m1EGWfxTRVXHwEJ9ax57owMPTt+Bh8LX35oLePmrOvs7veDBQ5ubGK1rRRLPtwsIbu FBwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Instead of pushing and popping %RSI several times to preserve the struct boot_params pointer across the execution of the startup code, move it into a callee save register before the first call into C, and copy it back when needed. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 34 +++++++------------- 1 file changed, 11 insertions(+), 23 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 9f90661744741210..2d1b0ee94929f7ec 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -405,10 +405,14 @@ SYM_CODE_START(startup_64) lretq .Lon_kernel_cs: + /* + * RSI holds a pointer to a boot_params structure provided by the + * loader, and this needs to be preserved across C function calls. So + * move it into a callee saved register. + */ + movq %rsi, %r15 - pushq %rsi call load_stage1_idt - popq %rsi #ifdef CONFIG_AMD_MEM_ENCRYPT /* @@ -421,10 +425,8 @@ SYM_CODE_START(startup_64) * detection/setup to ensure that has been done in advance of any dependent * code. */ - pushq %rsi - movq %rsi, %rdi /* real mode address */ + movq %r15, %rdi /* pass struct boot_params pointer */ call sev_enable - popq %rsi #endif /* @@ -437,13 +439,9 @@ SYM_CODE_START(startup_64) * - Non zero RDX means trampoline needs to enable 5-level * paging. * - * RSI holds real mode data and needs to be preserved across - * this function call. */ - pushq %rsi - movq %rsi, %rdi /* real mode address */ + movq %r15, %rdi /* pass struct boot_params pointer */ call paging_prepare - popq %rsi /* Save the trampoline address in RCX */ movq %rax, %rcx @@ -468,14 +466,9 @@ trampoline_return: * * RDI is address of the page table to use instead of page table * in trampoline memory (if required). - * - * RSI holds real mode data and needs to be preserved across - * this function call. */ - pushq %rsi leaq rva(top_pgtable)(%rbx), %rdi call cleanup_trampoline - popq %rsi /* Zero EFLAGS */ pushq $0 @@ -485,7 +478,6 @@ trampoline_return: * Copy the compressed kernel to the end of our buffer * where decompression in place becomes safe. */ - pushq %rsi leaq (_bss-8)(%rip), %rsi leaq rva(_bss-8)(%rbx), %rdi movl $(_bss - startup_32), %ecx @@ -493,7 +485,6 @@ trampoline_return: std rep movsq cld - popq %rsi /* * The GDT may get overwritten either during the copy we just did or @@ -525,30 +516,27 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) shrq $3, %rcx rep stosq - pushq %rsi call load_stage2_idt /* Pass boot_params to initialize_identity_maps() */ - movq (%rsp), %rdi + movq %r15, %rdi /* pass struct boot_params pointer */ call initialize_identity_maps - popq %rsi /* * Do the extraction, and jump to the new kernel.. */ - pushq %rsi /* Save the real mode argument */ - movq %rsi, %rdi /* real mode address */ + movq %r15, %rdi /* pass struct boot_params pointer */ leaq boot_heap(%rip), %rsi /* malloc area for uncompression */ leaq input_data(%rip), %rdx /* input_data */ movl input_len(%rip), %ecx /* input_len */ movq %rbp, %r8 /* output target address */ movl output_len(%rip), %r9d /* decompressed length, end of relocs */ call extract_kernel /* returns kernel entry point in %rax */ - popq %rsi /* * Jump to the decompressed kernel. */ + movq %r15, %rsi jmp *%rax SYM_FUNC_END(.Lrelocated) From patchwork Mon May 22 07:14:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EFE4C77B73 for ; Mon, 22 May 2023 07:15:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232343AbjEVHPS (ORCPT ); Mon, 22 May 2023 03:15:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232055AbjEVHPJ (ORCPT ); Mon, 22 May 2023 03:15:09 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D9D1C1; Mon, 22 May 2023 00:15:08 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D5FE961DD5; Mon, 22 May 2023 07:15:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DA42C4339E; Mon, 22 May 2023 07:15:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739707; bh=6LI2p3tMGpqgjf5Vh98U5/dlrPhBIFsp0XryDg0+x/4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q3m4hzTgh/bSksj7igaVybxKUaNE+hjOnVev/TCYk+HHZ507owq2NKfHGRoiNZK/l QfbcI9Ma3XpBQLOZHtNHyie3dGlmatDuZqAaMRDlF0wWDcyp6ow2m5eYSZMJEI2FBP yq/J57wA294JWZU6JyPyQnDgmsAj0rZvNF8n2+Kp0nvXreFuw2m1OxmPCXLMaWNqre kpeXldvteNljTO/ta8iKS1WeF5xUiFnDU7ga9Jn3JkeK6Xs1yb6zH+MUBa+dZiqoEz doP6jLyjVHoKSYYYRnJHYKKux5wrU+2cNWfesPqXEBuu5wjounLFCubCVTo1RzHrmt S71izQlFfOGxQ== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 08/21] x86/decompressor: Use standard calling convention for trampoline Date: Mon, 22 May 2023 09:14:02 +0200 Message-Id: <20230522071415.501717-9-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4143; i=ardb@kernel.org; h=from:subject; bh=6LI2p3tMGpqgjf5Vh98U5/dlrPhBIFsp0XryDg0+x/4=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzHxl6kWJ8zVOTq1sXadOmfbaxrX+bM1U2TXTp8ROI vgNn21HKQuDGAeDrJgii8Dsv+92np4oVes8SxZmDisTyBAGLk4BmEiPLyNDuw+Dff7Guc1PFHnn RRze9nLSf8nwF6Ifb13/ELn4cq4oPyPDiqVFEuUPjR8JiKZ07vJn71Z+12W017XkTZQr+75VlUc YAQ== X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Update the trampoline code so its arguments are passed via RDI and RSI, which matches the ordinary SysV calling convention for x86_64. This will allow this code to be called directly from C. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 30 +++++++++----------- arch/x86/boot/compressed/pgtable.h | 2 +- 2 files changed, 14 insertions(+), 18 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index af45ddd8297a4a07..a387cd80964e1a1e 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -443,9 +443,9 @@ SYM_CODE_START(startup_64) movq %r15, %rdi /* pass struct boot_params pointer */ call paging_prepare - /* Save the trampoline address in RCX */ - movq %rax, %rcx - + /* Pass the trampoline address and boolean flag as args #1 and #2 */ + movq %rax, %rdi + movq %rdx, %rsi leaq TRAMPOLINE_32BIT_CODE_OFFSET(%rax), %rax call *%rax @@ -534,11 +534,11 @@ SYM_FUNC_END(.Lrelocated) /* * This is the 32-bit trampoline that will be copied over to low memory. * - * ECX contains the base address of the trampoline memory. - * Non zero RDX means trampoline needs to enable 5-level paging. + * EDI contains the base address of the trampoline memory. + * Non-zero ESI means trampoline needs to enable 5-level paging. */ SYM_CODE_START(trampoline_32bit_src) - popq %rdi + popq %r8 /* Switch to compatibility mode (CS.L = 0 CS.D = 1) via far return */ pushq $__KERNEL32_CS leaq 0f(%rip), %rax @@ -552,7 +552,7 @@ SYM_CODE_START(trampoline_32bit_src) movl %eax, %ss /* Set up new stack */ - leal TRAMPOLINE_32BIT_STACK_END(%ecx), %esp + leal TRAMPOLINE_32BIT_STACK_END(%edi), %esp /* Disable paging */ movl %cr0, %eax @@ -560,7 +560,7 @@ SYM_CODE_START(trampoline_32bit_src) movl %eax, %cr0 /* Check what paging mode we want to be in after the trampoline */ - testl %edx, %edx + testl %esi, %esi jz 1f /* We want 5-level paging: don't touch CR3 if it already points to 5-level page tables */ @@ -575,21 +575,17 @@ SYM_CODE_START(trampoline_32bit_src) jz 3f 2: /* Point CR3 to the trampoline's new top level page table */ - leal TRAMPOLINE_32BIT_PGTABLE_OFFSET(%ecx), %eax + leal TRAMPOLINE_32BIT_PGTABLE_OFFSET(%edi), %eax movl %eax, %cr3 3: /* Set EFER.LME=1 as a precaution in case hypervsior pulls the rug */ - pushl %ecx - pushl %edx movl $MSR_EFER, %ecx rdmsr btsl $_EFER_LME, %eax /* Avoid writing EFER if no change was made (for TDX guest) */ jc 1f wrmsr -1: popl %edx - popl %ecx - +1: #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. @@ -606,14 +602,14 @@ SYM_CODE_START(trampoline_32bit_src) /* Enable PAE and LA57 (if required) paging modes */ orl $X86_CR4_PAE, %eax - testl %edx, %edx + testl %esi, %esi jz 1f orl $X86_CR4_LA57, %eax 1: movl %eax, %cr4 /* Calculate address of paging_enabled() once we are executing in the trampoline */ - leal .Lpaging_enabled - trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_OFFSET(%ecx), %eax + leal .Lpaging_enabled - trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_OFFSET(%edi), %eax /* Prepare the stack for far return to Long Mode */ pushl $__KERNEL_CS @@ -630,7 +626,7 @@ SYM_CODE_END(trampoline_32bit_src) .code64 SYM_FUNC_START_LOCAL_NOALIGN(.Lpaging_enabled) /* Return from the trampoline */ - jmp *%rdi + jmp *%r8 SYM_FUNC_END(.Lpaging_enabled) /* diff --git a/arch/x86/boot/compressed/pgtable.h b/arch/x86/boot/compressed/pgtable.h index 91dbb99203fbce2d..4e8cef135226bcbb 100644 --- a/arch/x86/boot/compressed/pgtable.h +++ b/arch/x86/boot/compressed/pgtable.h @@ -14,7 +14,7 @@ extern unsigned long *trampoline_32bit; -extern void trampoline_32bit_src(void *return_ptr); +extern void trampoline_32bit_src(void *trampoline, bool enable_5lvl); #endif /* __ASSEMBLER__ */ #endif /* BOOT_COMPRESSED_PAGETABLE_H */ From patchwork Mon May 22 07:14:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C2AEC77B75 for ; Mon, 22 May 2023 07:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232246AbjEVHQE (ORCPT ); Mon, 22 May 2023 03:16:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232097AbjEVHPd (ORCPT ); Mon, 22 May 2023 03:15:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A75D5CA; Mon, 22 May 2023 00:15:16 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2CBB061DC4; Mon, 22 May 2023 07:15:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DBAEFC433EF; Mon, 22 May 2023 07:15:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739715; bh=hFZ/SCviIEc1YZbiLBLlEDyweQ6PGSxYwFHksamIzHs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hz/XLYpwvrB7eaE67rIp6oVZcPncDEeoJ1HK+tnyQVcUCtXSHsbtsqPkd8b+q6RAT ZKfyjLbFV7cR2whGTuHdlUIvZT8cdvhBpe3lhbO6U9lffuWe2Tj3qFECI2hjnIyYpT uMkW92CnW/NWGG+IJ4qnp9ow11jZtgz7G6CdiN6OjrQ8dK9z8+LtMJQm8WuxcRJy5P 7QpuutM9ZljLjE83bqk3d/qSZPksgzgMO7Wf/FG5bfUEgPYezQ8xwYzWbRwLeEODq5 Xu4TCQTn8mY3nybucf17nyxw6gg3XkiWZpZElvymw03DzBd2SAmMdcjhcng0/0bvyv iWtsyc3P1mXfg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 10/21] x86/decompressor: Call trampoline directly from C code Date: Mon, 22 May 2023 09:14:04 +0200 Message-Id: <20230522071415.501717-11-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4966; i=ardb@kernel.org; h=from:subject; bh=hFZ/SCviIEc1YZbiLBLlEDyweQ6PGSxYwFHksamIzHs=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzPL8sq2/X/qHu2RslN1qcKxny6qQhYuW/epuYZguW MS46tj8jlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRoyGMDId17vRVz82v/sNW 3LB74ubOr+6SR34XTX9+UuHX+vULoh4x/OGZ2Gx+NysqlrmRZ573db4utlXfpc6IrTXQzc8zuWX xkgsA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Instead of returning to the asm calling code to invoke the trampoline, call it straight from the C code that sets the scene. That way, the struct return type is no longer needed for returning two values, and the call can be made conditional more cleanly in a subsequent patch. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 20 +++----------- arch/x86/boot/compressed/pgtable_64.c | 28 ++++++++------------ 2 files changed, 15 insertions(+), 33 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 741b4e8fefc915ea..a60ec9283bd760e3 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -430,24 +430,12 @@ SYM_CODE_START(startup_64) #endif /* - * paging_prepare() sets up the trampoline and checks if we need to - * enable 5-level paging. - * - * paging_prepare() returns a two-quadword structure which lands - * into RDX:RAX: - * - Address of the trampoline is returned in RAX. - * - Non zero RDX means trampoline needs to enable 5-level - * paging. - * + * set_paging_levels() updates the number of paging levels using a + * trampoline in 32-bit addressable memory if the current number does + * not match the desired number. */ movq %r15, %rdi /* pass struct boot_params pointer */ - call paging_prepare - - /* Pass the trampoline address and boolean flag as args #1 and #2 */ - movq %rax, %rdi - movq %rdx, %rsi - leaq TRAMPOLINE_32BIT_CODE_OFFSET(%rax), %rax - call *%rax + call set_paging_levels /* * cleanup_trampoline() would restore trampoline memory. diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index 09fc18180929fab3..b62b6819dcdd01be 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -16,11 +16,6 @@ unsigned int __section(".data") pgdir_shift = 39; unsigned int __section(".data") ptrs_per_p4d = 1; #endif -struct paging_config { - unsigned long trampoline_start; - unsigned long l5_required; -}; - /* Buffer to preserve trampoline memory */ static char trampoline_save[TRAMPOLINE_32BIT_SIZE]; @@ -106,10 +101,10 @@ static unsigned long find_trampoline_placement(void) return bios_start - TRAMPOLINE_32BIT_SIZE; } -struct paging_config paging_prepare(void *rmode) +asmlinkage void set_paging_levels(void *rmode) { - struct paging_config paging_config = {}; - void *tramp_code; + void (*toggle_la57)(void *trampoline, bool enable_5lvl); + bool l5_required = false; /* Initialize boot_params. Required for cmdline_find_option_bool(). */ boot_params = rmode; @@ -130,12 +125,10 @@ struct paging_config paging_prepare(void *rmode) !cmdline_find_option_bool("no5lvl") && native_cpuid_eax(0) >= 7 && (native_cpuid_ecx(7) & (1 << (X86_FEATURE_LA57 & 31)))) { - paging_config.l5_required = 1; + l5_required = true; } - paging_config.trampoline_start = find_trampoline_placement(); - - trampoline_32bit = (unsigned long *)paging_config.trampoline_start; + trampoline_32bit = (unsigned long *)find_trampoline_placement(); /* Preserve trampoline memory */ memcpy(trampoline_save, trampoline_32bit, TRAMPOLINE_32BIT_SIZE); @@ -144,7 +137,7 @@ struct paging_config paging_prepare(void *rmode) memset(trampoline_32bit, 0, TRAMPOLINE_32BIT_SIZE); /* Copy trampoline code in place */ - tramp_code = memcpy(trampoline_32bit + + toggle_la57 = memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE); @@ -154,7 +147,8 @@ struct paging_config paging_prepare(void *rmode) * immediate absolute address, so we have to adjust that based on the * placement of the trampoline. */ - *(u32 *)(tramp_code + trampoline_ljmp_imm_offset) += (unsigned long)tramp_code; + *(u32 *)((u8 *)toggle_la57 + trampoline_ljmp_imm_offset) += + (unsigned long)toggle_la57; /* * The code below prepares page table in trampoline memory. @@ -170,10 +164,10 @@ struct paging_config paging_prepare(void *rmode) * We are not going to use the page table in trampoline memory if we * are already in the desired paging mode. */ - if (paging_config.l5_required == !!(native_read_cr4() & X86_CR4_LA57)) + if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) goto out; - if (paging_config.l5_required) { + if (l5_required) { /* * For 4- to 5-level paging transition, set up current CR3 as * the first and the only entry in a new top-level page table. @@ -196,7 +190,7 @@ struct paging_config paging_prepare(void *rmode) } out: - return paging_config; + toggle_la57(trampoline_32bit, l5_required); } void cleanup_trampoline(void *pgtable) From patchwork Mon May 22 07:14:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6E01C77B75 for ; Mon, 22 May 2023 07:16:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232448AbjEVHQR (ORCPT ); Mon, 22 May 2023 03:16:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232504AbjEVHQD (ORCPT ); Mon, 22 May 2023 03:16:03 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FCAD1BB; Mon, 22 May 2023 00:15:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7ED8A61DE7; Mon, 22 May 2023 07:15:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3320DC433D2; Mon, 22 May 2023 07:15:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739723; bh=CnFv45/Vh35jv2ecaVpF4w7431pbCKxy7DMiPDdcENI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LYl06j+J8ieYZZca2k1PJDH91lGO+30SLJSBq06nguarkjbfMDpUecxsAnuYX0y+e j1phsGY8+nx4lIU611QIJ80L1lZFACYxOLOu7IXGc2t+9++aqhyOiyCjSyosdQlXPp jN9WBZRv4BcsgoHZgWDUZ0ijcx76sqNBlneVNv2yT05n15v4fhnHC7LzoUXH/anGg9 IuBSC8v/+PZ1uzkkKKqIawtbnM1Y4sDWqnsHDhgn86UhItg2W/bnWqu5i/GJpY7a4X uhfFYP62wf2i+zdTt7JrkrGy68+ZbyLwxdbZCGaYPlGJpxapQX1TjKNZmART5vzTqi 7Jf5leHkURz4A== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 12/21] x86/decompressor: Merge trampoline cleanup with switching code Date: Mon, 22 May 2023 09:14:06 +0200 Message-Id: <20230522071415.501717-13-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4166; i=ardb@kernel.org; h=from:subject; bh=CnFv45/Vh35jv2ecaVpF4w7431pbCKxy7DMiPDdcENI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzLpZ32dibazdja1Rn9b22qXov+fefvkk913jmvsPj pSsXHu3o5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEzkoQPD/zwWY4mVUy4VsMvt jI2IfJU8PXCJlfixy8de2z44onBukRjDP6M66dpYdy6l77pmR7/Y/v/S9CB37XK3HfcSZ3ILMD/ kZQcA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Now that the trampoline setup code and the actual invocation of it are all done from the C routine, the trampoline cleanup can be merged into it as well, instead of returning to asm just to call another C function. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 13 +++------ arch/x86/boot/compressed/pgtable_64.c | 28 ++++++++------------ 2 files changed, 15 insertions(+), 26 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 403c96dae34d9c6d..b5bd6be035a7b7ec 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -429,19 +429,14 @@ SYM_CODE_START(startup_64) * set_paging_levels() updates the number of paging levels using a * trampoline in 32-bit addressable memory if the current number does * not match the desired number. + * + * RSI is the relocated address of the page table to use instead of + * page table in trampoline memory (if required). */ movq %r15, %rdi /* pass struct boot_params pointer */ + leaq rva(top_pgtable)(%rbx), %rsi call set_paging_levels - /* - * cleanup_trampoline() would restore trampoline memory. - * - * RDI is address of the page table to use instead of page table - * in trampoline memory (if required). - */ - leaq rva(top_pgtable)(%rbx), %rdi - call cleanup_trampoline - /* Zero EFLAGS */ pushq $0 popfq diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index b92cf1d6e156d5f6..eeddad8c8335655e 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -101,9 +101,10 @@ static unsigned long find_trampoline_placement(void) return bios_start - TRAMPOLINE_32BIT_SIZE; } -asmlinkage void set_paging_levels(void *rmode) +asmlinkage void set_paging_levels(void *rmode, void *pgtable) { void (*toggle_la57)(void *trampoline, bool enable_5lvl); + void *trampoline_pgtable; bool l5_required = false; /* Initialize boot_params. Required for cmdline_find_option_bool(). */ @@ -133,7 +134,7 @@ asmlinkage void set_paging_levels(void *rmode) * are already in the desired paging mode. */ if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) - return; + goto out; trampoline_32bit = (unsigned long *)find_trampoline_placement(); @@ -163,6 +164,8 @@ asmlinkage void set_paging_levels(void *rmode) * The new page table will be used by trampoline code for switching * from 4- to 5-level paging or vice versa. */ + trampoline_pgtable = trampoline_32bit + + TRAMPOLINE_32BIT_PGTABLE_OFFSET / sizeof(unsigned long); if (l5_required) { /* @@ -182,31 +185,21 @@ asmlinkage void set_paging_levels(void *rmode) * may be above 4G. */ src = *(unsigned long *)__native_read_cr3() & PAGE_MASK; - memcpy(trampoline_32bit + TRAMPOLINE_32BIT_PGTABLE_OFFSET / sizeof(unsigned long), - (void *)src, PAGE_SIZE); + memcpy(trampoline_pgtable, (void *)src, PAGE_SIZE); } toggle_la57(trampoline_32bit, l5_required); -} - -void cleanup_trampoline(void *pgtable) -{ - void *trampoline_pgtable; - - trampoline_pgtable = trampoline_32bit + TRAMPOLINE_32BIT_PGTABLE_OFFSET / sizeof(unsigned long); /* - * Move the top level page table out of trampoline memory, - * if it's there. + * Move the top level page table out of trampoline memory. */ - if ((void *)__native_read_cr3() == trampoline_pgtable) { - memcpy(pgtable, trampoline_pgtable, PAGE_SIZE); - native_write_cr3((unsigned long)pgtable); - } + memcpy(pgtable, trampoline_pgtable, PAGE_SIZE); + native_write_cr3((unsigned long)pgtable); /* Restore trampoline memory */ memcpy(trampoline_32bit, trampoline_save, TRAMPOLINE_32BIT_SIZE); +out: /* Initialize variables for 5-level paging */ #ifdef CONFIG_X86_5LEVEL if (__read_cr4() & X86_CR4_LA57) { @@ -215,4 +208,5 @@ void cleanup_trampoline(void *pgtable) ptrs_per_p4d = 512; } #endif + return; } From patchwork Mon May 22 07:14:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3772C7EE23 for ; Mon, 22 May 2023 07:16:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232533AbjEVHQl (ORCPT ); Mon, 22 May 2023 03:16:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232483AbjEVHQI (ORCPT ); Mon, 22 May 2023 03:16:08 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DD8EE7E; Mon, 22 May 2023 00:15:33 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C96BB61DF3; Mon, 22 May 2023 07:15:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E058C433A1; Mon, 22 May 2023 07:15:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739732; bh=9rSHT6To50AeTePxnLLsddpAkNAPINJ4d8fPXZY5ZkI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M1Cqkfr0Z9+BTOytdvTGksJEHot4TmJ+SSmPbxA8uA0H0JIyxxXr7OrYxAThdfSBG iYflMl6nlU4kJa7YVVsFDDxSo3Ra8bxKCcoKC7q/k4BCvsnzX634ypUn0cHqaTHkVR dI8osqfR4xcpswOqzBhqtXf7Z3VV/+yL8Pq95Fd/dZAp/RtNNI/v7Q4/F8hNh140vO JhXdsG4ssDy37iY2/g3PC3rhfg/y3tgvz2LrhycqSPk3rkUIDJHCamrhbej2mKssnL /Kv+9i6uKQQRAvYrPNR4cLqIiJY3Ui1msoBE317GRBcyFoty923VZjR8uI0UIONO3f 3SIKPG1mpFOog== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 14/21] x86/efistub: Prefer EFI memory attributes protocol over DXE services Date: Mon, 22 May 2023 09:14:08 +0200 Message-Id: <20230522071415.501717-15-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3293; i=ardb@kernel.org; h=from:subject; bh=9rSHT6To50AeTePxnLLsddpAkNAPINJ4d8fPXZY5ZkI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzG71mWflt6IOMnrNKp69L/qBw85rtQ8dUnoi11ycK SDGfnJ9RykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZiISh3DP7Up868GXGqfeiyx ft+nFzm7Gn/4NHqoKKilLbRZ58AT1sTwT2npF6eSaXWLZ/KqbN+UImqlvE30U35g2+EJXo4L5mQ 4cQAA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Currently, the EFI stub relies on DXE services in some cases to clear non-execute restrictions from page allocations that need to be executable. This is dodgy, because DXE services are not specified by UEFI but by PI, and they are not intended for consumption by OS loaders. However, no alternative existed at the time. Now, there is a new UEFI protocol that should be used instead, so if it exists, prefer it over the DXE services calls. Signed-off-by: Ard Biesheuvel --- drivers/firmware/efi/libstub/x86-stub.c | 29 ++++++++++++++------ 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 229def7d9b028ceb..fcdae5db0c63c7e5 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -26,6 +26,7 @@ const efi_system_table_t *efi_system_table; const efi_dxe_services_table_t *efi_dxe_table; u32 image_offset __section(".data"); static efi_loaded_image_t *image = NULL; +static efi_memory_attribute_protocol_t *memattr; static efi_status_t preserve_pci_rom_image(efi_pci_io_protocol_t *pci, struct pci_setup_rom **__rom) @@ -222,12 +223,18 @@ void efi_adjust_memory_range_protection(unsigned long start, unsigned long rounded_start, rounded_end; unsigned long unprotect_start, unprotect_size; - if (efi_dxe_table == NULL) - return; - rounded_start = rounddown(start, EFI_PAGE_SIZE); rounded_end = roundup(start + size, EFI_PAGE_SIZE); + if (memattr != NULL) { + efi_call_proto(memattr, clear_memory_attributes, rounded_start, + rounded_end - rounded_start, EFI_MEMORY_XP); + return; + } + + if (efi_dxe_table == NULL) + return; + /* * Don't modify memory region attributes, they are * already suitable, to lower the possibility to @@ -766,6 +773,7 @@ void __noreturn efi_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, struct boot_params *boot_params) { + efi_guid_t guid = EFI_MEMORY_ATTRIBUTE_PROTOCOL_GUID; unsigned long bzimage_addr = (unsigned long)startup_32; unsigned long buffer_start, buffer_end; struct setup_header *hdr = &boot_params->hdr; @@ -777,13 +785,18 @@ void __noreturn efi_stub_entry(efi_handle_t handle, if (efi_system_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE) efi_exit(handle, EFI_INVALID_PARAMETER); - efi_dxe_table = get_efi_config_table(EFI_DXE_SERVICES_TABLE_GUID); - if (efi_dxe_table && - efi_dxe_table->hdr.signature != EFI_DXE_SERVICES_TABLE_SIGNATURE) { - efi_warn("Ignoring DXE services table: invalid signature\n"); - efi_dxe_table = NULL; + if (IS_ENABLED(CONFIG_EFI_DXE_MEM_ATTRIBUTES)) { + efi_dxe_table = get_efi_config_table(EFI_DXE_SERVICES_TABLE_GUID); + if (efi_dxe_table && + efi_dxe_table->hdr.signature != EFI_DXE_SERVICES_TABLE_SIGNATURE) { + efi_warn("Ignoring DXE services table: invalid signature\n"); + efi_dxe_table = NULL; + } } + /* grab the memory attributes protocol if it exists */ + efi_bs_call(locate_protocol, &guid, NULL, (void **)&memattr); + if (IS_ENABLED(CONFIG_X86_64)) { status = efi_setup_5level_paging(); if (status != EFI_SUCCESS) { From patchwork Mon May 22 07:14:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A172C77B75 for ; Mon, 22 May 2023 07:17:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232097AbjEVHRK (ORCPT ); Mon, 22 May 2023 03:17:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232476AbjEVHQj (ORCPT ); Mon, 22 May 2023 03:16:39 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF9E6170E; Mon, 22 May 2023 00:15:41 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1908961DF1; Mon, 22 May 2023 07:15:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CB936C4339E; Mon, 22 May 2023 07:15:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739740; bh=g3nSTumb/KYsp4GqRL0zJ1a7x2cI2HwxP02u+R5/mbI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mirx97l6GWeBA0OhV7TsvAblRauu2aWVvH38JpIG2zKIul+qU7f7MRIgo0GUnbemS qfLTjHv2YeUlFQHRaerfKF5pmzi1/odSwseeo2YmczUaGASU3y8null0bQphft+8xx RdBO6CCwWqblZD0t5wobdOgyDF888fx+hGQ5tnIWAp5PPktseCAsqL3xKFnPV3+57m d+jGNmqd001KMdQrQrfPAC4DnaqZFr5C6NEJ7xbB6LnPpyCICPdC5zIgppJ7eZgFco QcqnjtXwrECw8JDEUv/xXJYl+iJLuyktvL4R5QCSYl/EmnKO7pMPudypMuzhB6kWNT U5kBpXJjGNTNQ== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 16/21] x86/decompressor: Move global symbol references to C code Date: Mon, 22 May 2023 09:14:10 +0200 Message-Id: <20230522071415.501717-17-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5078; i=ardb@kernel.org; h=from:subject; bh=g3nSTumb/KYsp4GqRL0zJ1a7x2cI2HwxP02u+R5/mbI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzKFo2l9xTa26d75v9rwIzo5Ydy/OZve//yZNFamuE 06pdfJ0lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgInUSjAy7D6ZnzFzt9KtU3FW 0bFpX5a6CqtMvfh37t56xS9f0684NDD802tN7i2ZHW0muOyybCmDTICE6MUzu6Y/mSkjH1J/zE2 CAwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org It is no longer necessary to be cautious when referring to global variables in the position independent decompressor code, now that it is built using PIE codegen and makes an assertion in the linker script that no GOT entries exist (which would require adjustment for the actual runtime load address of the decompressor binary). This means global variables can be referenced directly from C code, instead of having to pass their runtime addresses into C routines from asm code, which needs to happen at each call site. Do so for the code that will be called directly from the EFI stub after a subsequent patch, and avoid the need to duplicate this logic a third time. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_32.S | 8 -------- arch/x86/boot/compressed/head_64.S | 8 +------- arch/x86/boot/compressed/misc.c | 16 +++++++++------- 3 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 3530465b5b85ccf3..beee858058df4403 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -168,13 +168,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) */ /* push arguments for extract_kernel: */ - pushl output_len@GOTOFF(%ebx) /* decompressed length, end of relocs */ pushl %ebp /* output address */ - pushl input_len@GOTOFF(%ebx) /* input_len */ - leal input_data@GOTOFF(%ebx), %eax - pushl %eax /* input_data */ - leal boot_heap@GOTOFF(%ebx), %eax - pushl %eax /* heap area */ pushl %esi /* real mode pointer */ call extract_kernel /* returns kernel entry point in %eax */ addl $24, %esp @@ -202,8 +196,6 @@ SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end) */ .bss .balign 4 -boot_heap: - .fill BOOT_HEAP_SIZE, 1, 0 boot_stack: .fill BOOT_STACK_SIZE, 1, 0 boot_stack_end: diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index b5bd6be035a7b7ec..3074d278c7e665d8 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -493,11 +493,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) * Do the extraction, and jump to the new kernel.. */ movq %r15, %rdi /* pass struct boot_params pointer */ - leaq boot_heap(%rip), %rsi /* malloc area for uncompression */ - leaq input_data(%rip), %rdx /* input_data */ - movl input_len(%rip), %ecx /* input_len */ - movq %rbp, %r8 /* output target address */ - movl output_len(%rip), %r9d /* decompressed length, end of relocs */ + movq %rbp, %rsi /* output target address */ call extract_kernel /* returns kernel entry point in %rax */ /* @@ -636,8 +632,6 @@ SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) */ .bss .balign 4 -SYM_DATA_LOCAL(boot_heap, .fill BOOT_HEAP_SIZE, 1, 0) - SYM_DATA_START_LOCAL(boot_stack) .fill BOOT_STACK_SIZE, 1, 0 .balign 16 diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index 36535a3753f5d5fa..ad7a2297c9e186df 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -330,6 +330,11 @@ static size_t parse_elf(void *output) return ehdr.e_entry - LOAD_PHYSICAL_ADDR; } +static u8 boot_heap[BOOT_HEAP_SIZE] __aligned(4); + +extern unsigned char input_data[]; +extern unsigned int input_len, output_len; + /* * The compressed kernel image (ZO), has been moved so that its position * is against the end of the buffer used to hold the uncompressed kernel @@ -347,14 +352,11 @@ static size_t parse_elf(void *output) * |-------uncompressed kernel image---------| * */ -asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, - unsigned char *input_data, - unsigned long input_len, - unsigned char *output, - unsigned long output_len) +asmlinkage __visible void *extract_kernel(void *rmode, unsigned char *output) { const unsigned long kernel_total_size = VO__end - VO__text; unsigned long virt_addr = LOAD_PHYSICAL_ADDR; + memptr heap = (memptr)boot_heap; unsigned long needed_size; size_t entry_offset; @@ -412,7 +414,7 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, * entries. This ensures the full mapped area is usable RAM * and doesn't include any reserved areas. */ - needed_size = max(output_len, kernel_total_size); + needed_size = max((unsigned long)output_len, kernel_total_size); #ifdef CONFIG_X86_64 needed_size = ALIGN(needed_size, MIN_KERNEL_ALIGN); #endif @@ -443,7 +445,7 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, #ifdef CONFIG_X86_64 if (heap > 0x3fffffffffffUL) error("Destination address too large"); - if (virt_addr + max(output_len, kernel_total_size) > KERNEL_IMAGE_SIZE) + if (virt_addr + needed_size > KERNEL_IMAGE_SIZE) error("Destination virtual address is beyond the kernel mapping area"); #else if (heap > ((-__PAGE_OFFSET-(128<<20)-1) & 0x7fffffff)) From patchwork Mon May 22 07:14:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B9A5C7EE2D for ; Mon, 22 May 2023 07:17:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232553AbjEVHR0 (ORCPT ); Mon, 22 May 2023 03:17:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232554AbjEVHQ7 (ORCPT ); Mon, 22 May 2023 03:16:59 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA313198E; Mon, 22 May 2023 00:15:52 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 66F3F61DFD; Mon, 22 May 2023 07:15:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 219F7C433A1; Mon, 22 May 2023 07:15:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739748; bh=XRnDiVEqXvkUypxOdLrzhfuiI1tjnMjnlxv76SViP7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MeHO95+fyW7ojbM1paHarGzzSqOBIfjxJ+yuheV73DAwvKeV08JYidpNzHaWZwuEg JsSeNEaKKmsXAw4HKwtOf5/AbOoeBxeRgkYcrkagdrmYxA8BFQyONaXX9ccphwv/s+ 6Ve8Wrvb5VjOKUh854UpOzm3GlqZavvhxINXZek4X+Aj6O2fKo05MXnihbwHMp/LDW 8EzhuHUU3XFVvk8yDQ6eMa12fYUToQbz9Xy3WDR+7Su3ITXzfPxAwR0DtgBKBA4gxP nJakjQDwZPHQ2RGHwTQvUGfTuf83W4djYeQkiSVqFtAeuDruz1hRuiwhE7+TkmIn47 MLycnjwgkG3Pg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 18/21] x86/head_64: Store boot_params pointer in callee-preserved register Date: Mon, 22 May 2023 09:14:12 +0200 Message-Id: <20230522071415.501717-19-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2440; i=ardb@kernel.org; h=from:subject; bh=XRnDiVEqXvkUypxOdLrzhfuiI1tjnMjnlxv76SViP7E=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzLlJ7ljnYiGh8BOsHcWd/55eUpsgaWUqviFc6vzTO wrbz1/oKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABNpYmb4p5rbsdXfZndEyfJ/ kzQzpFoTKtpDvRQETDetu66UVX7lPMM/g/ezCpJcgz48fdXGI9W4aKn/5k1HdnQf6T4nrar5ZY4 FMwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Instead of pushing/popping %RSI to/from the stack every time a function is called from startup_64(), store it in a callee preserved register and grab it from there when its value is actualled needed. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index a5df3e994f04f10f..95b12fdae10e1dc9 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -60,6 +60,7 @@ SYM_CODE_START_NOALIGN(startup_64) * compiled to run at we first fixup the physical addresses in our page * tables and then reload them. */ + mov %rsi, %r15 /* Preserve boot_params pointer */ /* Set up the stack for verify_cpu() */ leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp @@ -73,9 +74,7 @@ SYM_CODE_START_NOALIGN(startup_64) shrq $32, %rdx wrmsr - pushq %rsi call startup_64_setup_env - popq %rsi #ifdef CONFIG_AMD_MEM_ENCRYPT /* @@ -84,10 +83,8 @@ SYM_CODE_START_NOALIGN(startup_64) * which needs to be done before any CPUID instructions are executed in * subsequent code. */ - movq %rsi, %rdi - pushq %rsi + movq %r15, %rdi call sme_enable - popq %rsi #endif /* Now switch to __KERNEL_CS so IRET works reliably */ @@ -109,9 +106,7 @@ SYM_CODE_START_NOALIGN(startup_64) * programmed into CR3. */ leaq _text(%rip), %rdi - pushq %rsi call __startup_64 - popq %rsi /* Form the CR3 value being sure to include the CR3 modifier */ addq $(early_top_pgt - __START_KERNEL_map), %rax @@ -200,10 +195,8 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) * %rsi carries pointer to realmode data and is callee-clobbered. Save * and restore it. */ - pushq %rsi movq %rax, %rdi call sev_verify_cbit - popq %rsi /* * Switch to new page-table @@ -294,9 +287,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) wrmsr /* Setup and Load IDT */ - pushq %rsi call early_setup_idt - popq %rsi /* Check if nx is implemented */ movl $0x80000001, %eax @@ -334,7 +325,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) /* rsi is pointer to real mode structure with interesting info. pass it to C */ - movq %rsi, %rdi + movq %r15, %rdi .Ljump_to_C_code: /* From patchwork Mon May 22 07:14:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 685477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B67AC77B73 for ; Mon, 22 May 2023 07:17:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232589AbjEVHRr (ORCPT ); Mon, 22 May 2023 03:17:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232331AbjEVHRN (ORCPT ); Mon, 22 May 2023 03:17:13 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAA35C1; Mon, 22 May 2023 00:16:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B99CC61DD4; Mon, 22 May 2023 07:15:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 732C5C4339E; Mon, 22 May 2023 07:15:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739757; bh=6oUMrt+Q90a10bD/8IHJeLA9pGO90K+akqZ6KAUmlwE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lXnVT3BBGvOGBrGEKonh+CX6QVme5/hVLSlw2yXeQXl2TLB09u+M3bh0UJGbN7e3F o8U8YhxkGjydM1bYGVigVs/MF0KAzwZLi5LYKcr1o5XY3UnGB2ww+7hjypc/bpEeqe izLcMA6n31A6gToJtRiUFZxrk7VJg7Odjtp+/9+fJg4kty9m2+PsdXmxw6vFWcLZOF rfl2lBhzbnM5V5Bo83XyeOR7f/BzdTfoI96wdGn21m6cXrIBFlVv1FQn8pL5cV1VOt RUKgSvqCDSyo3ss+WbaGlRvPs8PxDlIvcldFKFzdCDOWAcLG76Dktm2+sb9RtS0tvi Pydsneaa0xVqw== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 20/21] x86/efistub: Check SEV/SNP support while running in the firmware Date: Mon, 22 May 2023 09:14:14 +0200 Message-Id: <20230522071415.501717-21-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5088; i=ardb@kernel.org; h=from:subject; bh=6oUMrt+Q90a10bD/8IHJeLA9pGO90K+akqZ6KAUmlwE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzNXusDBrtcaNl5M7/t9ojDjNn1TBPrvD9+Knc1r97 rd8nz7oKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOJaGf4nzbt4eUn1RdK08// X7SmYN975qIrAQVKX1i/8k/dHaCXuIHhf8TFRb+qSy8cfizF/+dYeVBLi8XlrmdXvFfZ/fsur5j 3lgkA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org Before refactoring the EFI stub boot flow to avoid the legacy bare metal decompressor, duplicate the SEV initialization and SNP feature check in the EFI stub before handing over to the kernel proper. This must be done after calling ExitBootServices(), to ensure that the SEV initialization does not corrupt any state that the firmware itself still relies on. This means that, unfortunately, the only recourse available when the SNP feature mask contains unsupported features is to terminate the virtual machine, which is what the bare metal decompressor does as well. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/sev.c | 19 ++++++++++++++----- arch/x86/include/asm/sev.h | 6 ++++++ drivers/firmware/efi/libstub/x86-stub.c | 17 +++++++++++++++++ 3 files changed, 37 insertions(+), 5 deletions(-) diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 014b89c890887b9a..d33d48359d09bfb5 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -315,23 +315,32 @@ static void enforce_vmpl0(void) */ #define SNP_FEATURES_PRESENT (0) +u64 snp_get_unsupported_features(void) +{ + if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED)) + return 0; + return sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT; +} + +void sev_es_terminate_snp_unsupported(void) +{ + sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED); +} + void snp_check_features(void) { u64 unsupported; - if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED)) - return; - /* * Terminate the boot if hypervisor has enabled any feature lacking * guest side implementation. Pass on the unsupported features mask through * EXIT_INFO_2 of the GHCB protocol so that those features can be reported * as part of the guest boot failure. */ - unsupported = sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT; + unsupported = snp_get_unsupported_features(); if (unsupported) { if (ghcb_version < 2 || (!boot_ghcb && !early_setup_ghcb())) - sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED); + sev_es_terminate_snp_unsupported(); sev_es_ghcb_terminate(boot_ghcb, SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED, unsupported); diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index 13dc2a9d23c1eb25..084a91aa5a6c708f 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -157,6 +157,7 @@ static __always_inline void sev_es_nmi_complete(void) __sev_es_nmi_complete(); } extern int __init sev_es_efi_map_ghcbs(pgd_t *pgd); +extern void sev_enable(struct boot_params *bp); static inline int rmpadjust(unsigned long vaddr, bool rmp_psize, unsigned long attrs) { @@ -202,12 +203,15 @@ void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio); +u64 snp_get_unsupported_features(void); +void sev_es_terminate_snp_unsupported(void); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } static inline int sev_es_setup_ap_jump_table(struct real_mode_header *rmh) { return 0; } static inline void sev_es_nmi_complete(void) { } static inline int sev_es_efi_map_ghcbs(pgd_t *pgd) { return 0; } +static inline void sev_enable(struct boot_params *bp) { } static inline int pvalidate(unsigned long vaddr, bool rmp_psize, bool validate) { return 0; } static inline int rmpadjust(unsigned long vaddr, bool rmp_psize, unsigned long attrs) { return 0; } static inline void setup_ghcb(void) { } @@ -225,6 +229,8 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in { return -ENOTTY; } +static inline u64 snp_get_unsupported_features(void) { return 0; } +static inline void sev_es_terminate_snp_unsupported(void) {} #endif #endif diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index fcdae5db0c63c7e5..02633199a8502b71 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -15,6 +15,7 @@ #include #include #include +#include #include "efistub.h" #include "x86-stub.h" @@ -923,6 +924,22 @@ void __noreturn efi_stub_entry(efi_handle_t handle, goto fail; } + /* + * Call the SEV init code while still running with the firmware's + * GDT/IDT, so #VC exceptions will be handled by EFI. + */ + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { + u64 unsupported; + + sev_enable(boot_params); + unsupported = snp_get_unsupported_features(); + if (unsupported) { + efi_err("Unsupported SEV-SNP features detected: 0x%llx\n", + unsupported); + sev_es_terminate_snp_unsupported(); + } + } + if (IS_ENABLED(CONFIG_X86_64)) { efi_5level_switch();