From patchwork Mon Mar 12 08:44:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 131376 Delivered-To: patch@linaro.org Received: by 10.46.84.17 with SMTP id i17csp478025ljb; Mon, 12 Mar 2018 01:46:35 -0700 (PDT) X-Google-Smtp-Source: AG47ELtbsDOuzrsHS4CQECksI+Cz89bma7sfLHZyqdpb2GnTyQ7H0DdcQlbvT/QWfRDWvcXp2dKo X-Received: by 2002:a17:902:6e0f:: with SMTP id u15-v6mr7408388plk.78.1520844395350; Mon, 12 Mar 2018 01:46:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1520844395; cv=none; d=google.com; s=arc-20160816; b=WdJZ+Rs3AkEpaTTDi7wwmUVXstQICvUdbegomoYw7leXhBTpAEcmUHGubAC50jYR42 r3HBN0pBAXk7JaRrgliP5nRiUIeA8FMKm5mxbr57zLBhQiHvJ+NarwOSMnjjcQ8MckI5 lZT/OSq4Zyp8/S8MV2SZUHWesgg1Skiq8Tkps6V+eirQ9i6PD5YkcJjz38x6/Wee8TVk 57Zy5UrArAlPrbhJKWHEoHGu+EuT+FcWfoNsgwxCKhVycwYJA4ZYyqvBJ+lgshQns/wx zt57OCr0KrjqvG9fBvPWxzywca0jK/5CaKT8EenMOhah0iop9dTr4zqYIiBpUoZn/Wfa 6ZjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=qa5AutG4QCRbyp1jPz/06C3tNLkJdxW216XmUztKtsQ=; b=on/fAr6u5+IPPgi/P6K33ra34414fEnLWQDjJa3SfQvH8DYnjMxx44ItARzzsHinAS ZHa7bU+yj3IlRxXWj1jU5nsgDsZjaYPGmd0NLhH73y6FzbJRzqghOcs7FHQl/Y/ZQOS+ JiCp8iSDylmAB4OQHbpYmpUeW5jPdpF+JRjd3tu2f2v2G+xejDzY3qhcxCMpWXYAwQAQ IM7458Ho24om4rm1NUAJcMLlXoU77ABY4ZJaV8lqOsVHGtfNbL4zjUoHE6EjyI0uBLDN eaRWhqyQbSl6wuYz0RtzqhkK8Gvw2cc9G+AC+VYw18P7opkztCi/sPwvfFvI1hEXz5vY yFIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QKXuvTUx; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d37-v6si5613988plb.231.2018.03.12.01.46.35; Mon, 12 Mar 2018 01:46:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QKXuvTUx; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752489AbeCLIqb (ORCPT + 28 others); Mon, 12 Mar 2018 04:46:31 -0400 Received: from mail-wr0-f196.google.com ([209.85.128.196]:37853 "EHLO mail-wr0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752516AbeCLIpf (ORCPT ); Mon, 12 Mar 2018 04:45:35 -0400 Received: by mail-wr0-f196.google.com with SMTP id z12so14757512wrg.4 for ; Mon, 12 Mar 2018 01:45:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qa5AutG4QCRbyp1jPz/06C3tNLkJdxW216XmUztKtsQ=; b=QKXuvTUxOj2kO56WWj/rT2AlNWYhetZSYwZC2i8xMfJLzNN8W3J1cSK7DYZwjV14wo p8w63NMRW74onL8/Be0XCkMRDnPH/l4ePtL5J+by+cu39luXJklbH9fI7g4TrmhTChH/ T6d0SEWW+aJrRjyYNTdvgkEomr140gyf6Whyg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qa5AutG4QCRbyp1jPz/06C3tNLkJdxW216XmUztKtsQ=; b=cDi57l7zCz1Rm4nyOwDRMPykngj7jjOlIRdW69qEZiLSC+jXOS7h8vYfPybD00bG9x q/hUedo/lLwr/DRlmpI4/KztAjo4SDV0yBDFP4umrIBafPHOzhGj6TC49v9rVlcRLgbw 3CVKiW8MGRv79d1tS1WTarKD/r3tSy1uq/m1lqWQ5q2c2nVfjNx9AJT12sodr9aIb72K 767zH3fQYviAAQ/9MegANrpnMSlUC/AKYLG0Lc6JVKxRzw/9bNJD+ogfJEIq/oMLa4UT FENQlFGXsP14HRbTSh96uuHJg9pkYQz0d0KDO91zpxpD/lLkYbLXeAR9EVGj+rI3R3Tk XomA== X-Gm-Message-State: AElRT7H29zvifz0ttFj7QzgFmfQ+86GNs/TyJsOGeZlTtF/9IpuoM3kq o3Talg7qaCo8Kw6Mwz/JQ0uoxQ== X-Received: by 10.223.153.215 with SMTP id y81mr5449948wrb.144.1520844334274; Mon, 12 Mar 2018 01:45:34 -0700 (PDT) Received: from localhost.localdomain ([105.148.128.186]) by smtp.gmail.com with ESMTPSA id c1sm6544788wre.27.2018.03.12.01.45.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Mar 2018 01:45:33 -0700 (PDT) From: Ard Biesheuvel To: linux-efi@vger.kernel.org, Ingo Molnar , Thomas Gleixner Cc: Sai Praneeth , Ard Biesheuvel , linux-kernel@vger.kernel.org, "Lee, Chun-Yi" , Borislav Petkov , Tony Luck , Andy Lutomirski , "Michael S . Tsirkin" , Bhupesh Sharma , Ricardo Neri , Ravi Shankar Subject: [PATCH 3/5] x86/efi: Use efi_switch_mm() rather than manually twiddling with %cr3 Date: Mon, 12 Mar 2018 08:44:58 +0000 Message-Id: <20180312084500.10764-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180312084500.10764-1-ard.biesheuvel@linaro.org> References: <20180312084500.10764-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Sai Praneeth Use helper function efi_switch_mm() to switch to/from efi_mm when invoking any UEFI runtime services. Likewise, we need to switch back to previous mm (mm context stolen by efi_mm) after the above calls return successfully. We can use efi_switch_mm() helper function only with x86_64 kernel and "efi=old_map" disabled because, x86_32 and efi=old_map do not use efi_pgd, rather they use swapper_pg_dir. Signed-off-by: Sai Praneeth Prakhya Cc: "Lee, Chun-Yi" Cc: Borislav Petkov Cc: Tony Luck Cc: Andy Lutomirski Cc: Michael S. Tsirkin Cc: Bhupesh Sharma Cc: Ricardo Neri Cc: Ravi Shankar Tested-by: Bhupesh Sharma Reviewed-by: Matt Fleming [ardb: add #include of sched/task.h for task_lock/_unlock] Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/efi.h | 25 +++++++++------------- arch/x86/platform/efi/efi_64.c | 41 +++++++++++++++++++----------------- arch/x86/platform/efi/efi_thunk_64.S | 2 +- 3 files changed, 33 insertions(+), 35 deletions(-) -- 2.15.1 diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h index c62443fa7d0a..cec5fae23eb3 100644 --- a/arch/x86/include/asm/efi.h +++ b/arch/x86/include/asm/efi.h @@ -70,14 +70,13 @@ extern asmlinkage u64 efi_call(void *fp, ...); #define efi_call_phys(f, args...) efi_call((f), args) /* - * Scratch space used for switching the pagetable in the EFI stub + * struct efi_scratch - Scratch space used while switching to/from efi_mm + * @phys_stack: stack used during EFI Mixed Mode + * @prev_mm: store/restore stolen mm_struct while switching to/from efi_mm */ struct efi_scratch { - u64 r15; - u64 prev_cr3; - pgd_t *efi_pgt; - bool use_pgd; - u64 phys_stack; + u64 phys_stack; + struct mm_struct *prev_mm; } __packed; #define arch_efi_call_virt_setup() \ @@ -87,11 +86,8 @@ struct efi_scratch { __kernel_fpu_begin(); \ firmware_restrict_branch_speculation_start(); \ \ - if (efi_scratch.use_pgd) { \ - efi_scratch.prev_cr3 = __read_cr3(); \ - write_cr3((unsigned long)efi_scratch.efi_pgt); \ - __flush_tlb_all(); \ - } \ + if (!efi_enabled(EFI_OLD_MEMMAP)) \ + efi_switch_mm(&efi_mm); \ }) #define arch_efi_call_virt(p, f, args...) \ @@ -99,10 +95,8 @@ struct efi_scratch { #define arch_efi_call_virt_teardown() \ ({ \ - if (efi_scratch.use_pgd) { \ - write_cr3(efi_scratch.prev_cr3); \ - __flush_tlb_all(); \ - } \ + if (!efi_enabled(EFI_OLD_MEMMAP)) \ + efi_switch_mm(efi_scratch.prev_mm); \ \ firmware_restrict_branch_speculation_end(); \ __kernel_fpu_end(); \ @@ -145,6 +139,7 @@ extern void __init efi_dump_pagetable(void); extern void __init efi_apply_memmap_quirks(void); extern int __init efi_reuse_config(u64 tables, int nr_tables); extern void efi_delete_dummy_variable(void); +extern void efi_switch_mm(struct mm_struct *mm); struct efi_setup_data { u64 fw_vendor; diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 8881e601c32d..229af5bbecb8 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -81,9 +82,8 @@ pgd_t * __init efi_call_phys_prolog(void) int n_pgds, i, j; if (!efi_enabled(EFI_OLD_MEMMAP)) { - save_pgd = (pgd_t *)__read_cr3(); - write_cr3((unsigned long)efi_scratch.efi_pgt); - goto out; + efi_switch_mm(&efi_mm); + return NULL; } early_code_mapping_set_exec(1); @@ -155,8 +155,7 @@ void __init efi_call_phys_epilog(pgd_t *save_pgd) pud_t *pud; if (!efi_enabled(EFI_OLD_MEMMAP)) { - write_cr3((unsigned long)save_pgd); - __flush_tlb_all(); + efi_switch_mm(efi_scratch.prev_mm); return; } @@ -344,13 +343,6 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) if (efi_enabled(EFI_OLD_MEMMAP)) return 0; - /* - * Since the PGD is encrypted, set the encryption mask so that when - * this value is loaded into cr3 the PGD will be decrypted during - * the pagetable walk. - */ - efi_scratch.efi_pgt = (pgd_t *)__sme_pa(pgd); - /* * It can happen that the physical address of new_memmap lands in memory * which is not mapped in the EFI page table. Therefore we need to go @@ -364,8 +356,6 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) return 1; } - efi_scratch.use_pgd = true; - /* * Certain firmware versions are way too sentimential and still believe * they are exclusive and unquestionable owners of the first physical page, @@ -624,6 +614,22 @@ void __init efi_dump_pagetable(void) #endif } +/* + * Makes the calling thread switch to/from efi_mm context. Can be used + * for SetVirtualAddressMap() i.e. current->active_mm == init_mm as well + * as during efi runtime calls i.e current->active_mm == current_mm. + * We are not mm_dropping()/mm_grabbing() any mm, because we are not + * losing/creating any references. + */ +void efi_switch_mm(struct mm_struct *mm) +{ + task_lock(current); + efi_scratch.prev_mm = current->active_mm; + current->active_mm = mm; + switch_mm(efi_scratch.prev_mm, mm, NULL); + task_unlock(current); +} + #ifdef CONFIG_EFI_MIXED extern efi_status_t efi64_thunk(u32, ...); @@ -677,16 +683,13 @@ efi_status_t efi_thunk_set_virtual_address_map( efi_sync_low_kernel_mappings(); local_irq_save(flags); - efi_scratch.prev_cr3 = __read_cr3(); - write_cr3((unsigned long)efi_scratch.efi_pgt); - __flush_tlb_all(); + efi_switch_mm(&efi_mm); func = (u32)(unsigned long)phys_set_virtual_address_map; status = efi64_thunk(func, memory_map_size, descriptor_size, descriptor_version, virtual_map); - write_cr3(efi_scratch.prev_cr3); - __flush_tlb_all(); + efi_switch_mm(efi_scratch.prev_mm); local_irq_restore(flags); return status; diff --git a/arch/x86/platform/efi/efi_thunk_64.S b/arch/x86/platform/efi/efi_thunk_64.S index 189b218da87c..46c58b08739c 100644 --- a/arch/x86/platform/efi/efi_thunk_64.S +++ b/arch/x86/platform/efi/efi_thunk_64.S @@ -33,7 +33,7 @@ ENTRY(efi64_thunk) * Switch to 1:1 mapped 32-bit stack pointer. */ movq %rsp, efi_saved_sp(%rip) - movq efi_scratch+25(%rip), %rsp + movq efi_scratch(%rip), %rsp /* * Calculate the physical address of the kernel text.