From patchwork Mon Mar 12 09:43:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 131378 Delivered-To: patch@linaro.org Received: by 10.46.84.17 with SMTP id i17csp523670ljb; Mon, 12 Mar 2018 02:44:13 -0700 (PDT) X-Google-Smtp-Source: AG47ELtgdJhKYGnDkh78Q1BAfjJ7w62CxqqwGzLRp1mzqNy+a+xeP1XZZzKJcoF3iHg3TIYEkyNw X-Received: by 10.98.21.200 with SMTP id 191mr7361110pfv.166.1520847852998; Mon, 12 Mar 2018 02:44:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1520847852; cv=none; d=google.com; s=arc-20160816; b=wMnxVYNX9sWqBRj4EARHIyey58D2K+xOKLPHm0kQcZmbXbCSOxa8ul3Gx+3Z5wIQuS pYBFVg74A0I9562BmkQrLpUoj1MYAmPb5jdeqPDZOhn85yRypMiPwjk899Z4vPmd4jxT /2+ijyMCkrGcGoShBOGxwNKLXMOrO5O8Nxbhpta6dzkrhJ5v63JYBFNmYIEZX64j0F5N UjhKpE/JpempgNo+F1eyVj6fq0bNKCoVdDxI95cR7Fu4MmVc37+gHl3kVs1KL/a/O9i6 YhFhgwiGTwz/USbEm7i3U6K7RfJHMcRutmj/ofjTjzVHirWy7pvoHjB7ZKQ0a4aavhJl plyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=DhcUollVkKFMA1+W0HQT0U146L2H377f94AfRgzyNLk=; b=Vd6yLj/32uiI3p6PnJaU1Zjr+3xh/AIx02lBh8jzFUXQr91l8e5w7fmQIVbubosi8F rlkduwm7+ENP/TzUB7r6028em/Vpd7wakAqDb96SyT7aPQNmc6+CBmP5dPweMNo8CypW g/weFeBZlLOOXSIu0BqAum5abSHnItxngLkc7aRfAx/GCpilnbvoyn670IeAhW4pp2lW UoXTnQHbZM7ZKG1aE5qxfDC/8kESrk9yXko06Gel+DsLscgG2Ulf6568fRzSwLNvT9tK Wj1CxAtiomNviEnvQiYZFD9wQcGYlz2Ot7v/YdjMAOHlrSlqFU1GZStLrZCjvEZyGdKp bjGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=XQM8daMP; spf=pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-efi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v3-v6si5698094plo.32.2018.03.12.02.44.12; Mon, 12 Mar 2018 02:44:12 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=XQM8daMP; spf=pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-efi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752422AbeCLJoK (ORCPT + 2 others); Mon, 12 Mar 2018 05:44:10 -0400 Received: from mail-wm0-f65.google.com ([74.125.82.65]:51559 "EHLO mail-wm0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751543AbeCLJoJ (ORCPT ); Mon, 12 Mar 2018 05:44:09 -0400 Received: by mail-wm0-f65.google.com with SMTP id h21so15054630wmd.1 for ; Mon, 12 Mar 2018 02:44:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=a1LPf1UUraSY8M8D44wAx1U2UfTVB+UNhvxilUqi8co=; b=XQM8daMPvuJKNF9lXCjEG/asdB2LfKJiWkfYdxyCo8ftDUNBW8bhtTKEUoyhxufebg /fYORv1RZklPt5oTUnxHrfN7+IVi848yZ+7JeDI4nZHc2vNNr/BhQvqEKEzz1KB/P0JE aaR0149Cm68YmnA38os9U+hWaWaG8Rcgibcr4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=a1LPf1UUraSY8M8D44wAx1U2UfTVB+UNhvxilUqi8co=; b=bb9bNlitwwVqgIFAUKg/7SdoGAFdNS3rdFUc7Nz0v11NSp1GEHkvZNT0XrjpOdqJVv DC5+0Dok8lq6jBYzm+OkJZxhvmgahgyxsnT3waAJKt1L3RWW7XNZy8z2Za0A87ZSHMK4 rDnuy6kWeFW/Quj5prgERQ+a4nqtfXdf6huOoRKM6kVjH3osN/HkMblKbKfwhYCRPMck 4IoaBlzMMAl0H9h6APolHk9VuOQ6iUmaetm2StpZhfnvLJoha4xwQt3Hi1Ak5T0WN3p/ 6gFkH1EuVVRep0mdeJg1DX+UySducX9VeddMgv6z1+lpWYtY7y+VxQI/o6k23UnyzOQS DohA== X-Gm-Message-State: AElRT7Gfivm0YVF3ZvjJmFV8YBUR1DxbHD98SPgPMzeclcKwZDtiD8g/ FajQ7xZWBsjYqr9P3WKfCDqbMgjkoC0= X-Received: by 10.28.14.135 with SMTP id 129mr4994360wmo.53.1520847847441; Mon, 12 Mar 2018 02:44:07 -0700 (PDT) Received: from localhost.localdomain ([105.148.128.186]) by smtp.gmail.com with ESMTPSA id d8sm3972947wmd.20.2018.03.12.02.44.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Mar 2018 02:44:06 -0700 (PDT) From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: mingo@kernel.org, Sai Praneeth , Ard Biesheuvel Subject: [PATCH rebased 1/2] x86/efi: Replace efi_pgd with efi_mm.pgd Date: Mon, 12 Mar 2018 09:43:54 +0000 Message-Id: <20180312094355.5371-1-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.15.1 Sender: linux-efi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Sai Praneeth Since the previous patch added support for efi_mm, let's handle efi_pgd through efi_mm and remove global variable efi_pgd. Signed-off-by: Sai Praneeth Prakhya Cc: "Lee, Chun-Yi" Cc: Borislav Petkov Cc: Tony Luck Cc: Andy Lutomirski Cc: Michael S. Tsirkin Cc: Bhupesh Sharma Cc: Ricardo Neri Cc: Ravi Shankar Tested-by: Bhupesh Sharma Reviewed-by: Matt Fleming Signed-off-by: Ard Biesheuvel --- arch/x86/mm/debug_pagetables.c | 6 +++--- arch/x86/platform/efi/efi_64.c | 18 +++++++++--------- 2 files changed, 12 insertions(+), 12 deletions(-) -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-efi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/x86/mm/debug_pagetables.c b/arch/x86/mm/debug_pagetables.c index 51a6f92da2bf..225fe2f0bfec 100644 --- a/arch/x86/mm/debug_pagetables.c +++ b/arch/x86/mm/debug_pagetables.c @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -73,13 +74,12 @@ static const struct file_operations ptdump_curusr_fops = { #endif #if defined(CONFIG_EFI) && defined(CONFIG_X86_64) -extern pgd_t *efi_pgd; static struct dentry *pe_efi; static int ptdump_show_efi(struct seq_file *m, void *v) { - if (efi_pgd) - ptdump_walk_pgd_level_debugfs(m, efi_pgd, false); + if (efi_mm.pgd) + ptdump_walk_pgd_level_debugfs(m, efi_mm.pgd, false); return 0; } diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 29425b6c98a7..29b267b8cb63 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -191,8 +191,7 @@ void __init efi_call_phys_epilog(pgd_t *save_pgd) early_code_mapping_set_exec(0); } -pgd_t *efi_pgd; -EXPORT_SYMBOL_GPL(efi_pgd); +EXPORT_SYMBOL_GPL(efi_mm); /* * We need our own copy of the higher levels of the page tables @@ -205,7 +204,7 @@ EXPORT_SYMBOL_GPL(efi_pgd); */ int __init efi_alloc_page_tables(void) { - pgd_t *pgd; + pgd_t *pgd, *efi_pgd; p4d_t *p4d; pud_t *pud; gfp_t gfp_mask; @@ -233,6 +232,7 @@ int __init efi_alloc_page_tables(void) return -ENOMEM; } + efi_mm.pgd = efi_pgd; mm_init_cpumask(&efi_mm); init_new_context(NULL, &efi_mm); @@ -248,6 +248,7 @@ void efi_sync_low_kernel_mappings(void) pgd_t *pgd_k, *pgd_efi; p4d_t *p4d_k, *p4d_efi; pud_t *pud_k, *pud_efi; + pgd_t *efi_pgd = efi_mm.pgd; if (efi_enabled(EFI_OLD_MEMMAP)) return; @@ -341,7 +342,7 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) unsigned long pfn, text, pf; struct page *page; unsigned npages; - pgd_t *pgd; + pgd_t *pgd = efi_mm.pgd; if (efi_enabled(EFI_OLD_MEMMAP)) return 0; @@ -351,8 +352,7 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) * this value is loaded into cr3 the PGD will be decrypted during * the pagetable walk. */ - efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd); - pgd = efi_pgd; + efi_scratch.efi_pgt = (pgd_t *)__sme_pa(pgd); /* * It can happen that the physical address of new_memmap lands in memory @@ -422,7 +422,7 @@ static void __init __map_region(efi_memory_desc_t *md, u64 va) { unsigned long flags = _PAGE_RW; unsigned long pfn; - pgd_t *pgd = efi_pgd; + pgd_t *pgd = efi_mm.pgd; if (!(md->attribute & EFI_MEMORY_WB)) flags |= _PAGE_PCD; @@ -526,7 +526,7 @@ void __init parse_efi_setup(u64 phys_addr, u32 data_len) static int __init efi_update_mappings(efi_memory_desc_t *md, unsigned long pf) { unsigned long pfn; - pgd_t *pgd = efi_pgd; + pgd_t *pgd = efi_mm.pgd; int err1, err2; /* Update the 1:1 mapping */ @@ -623,7 +623,7 @@ void __init efi_dump_pagetable(void) if (efi_enabled(EFI_OLD_MEMMAP)) ptdump_walk_pgd_level(NULL, swapper_pg_dir); else - ptdump_walk_pgd_level(NULL, efi_pgd); + ptdump_walk_pgd_level(NULL, efi_mm.pgd); #endif } From patchwork Mon Mar 12 09:43:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 131379 Delivered-To: patch@linaro.org Received: by 10.46.84.17 with SMTP id i17csp523680ljb; Mon, 12 Mar 2018 02:44:13 -0700 (PDT) X-Google-Smtp-Source: AG47ELt477K14Q0ywL7R73rk9xJbR0fSSkk0+xMa/ziKiblsz1P26Bw840n/mWz4bDUzuOvkf+oB X-Received: by 2002:a17:902:6b0c:: with SMTP id o12-v6mr219657plk.295.1520847853529; Mon, 12 Mar 2018 02:44:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1520847853; cv=none; d=google.com; s=arc-20160816; b=hR2SbvBOy4eeo/DnKFUQaZcxn83R0UJ27tCcQccjA8GoA0VPl4P02tAGr4XGmnURC+ ES30MtdVE3NdrazGcL1TCngKgHYjmM7NT02sWRaw3Hn5xNv0NcVbJLNAf25bZXYgws5K nHFSIpDnoAhi80K1PuxWxb8QJUISSikuGcfcGQc3Pxe/tVnIeIz00Sbh4HDZWHSI3Hc5 pcMV4HxJkJ0Qd2Wc4G/XdeNr7W7G+iNnsJtNHaFKSzPjR8UHe/4Qa2fGZxYUEv3hxzDT MeXpGXzxZ+ObaZGrlDVFVUeVjb8FOTarlZg4p0+efpHnlmly9v8wNWlziCubuPNWrlmM 5GRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=X9oZcHOwSiHiR22AawFoqeMP4La0n4fxr9XLZj8CP/I=; b=f4GEGn6oHrZ5Lga2/L3qDNnJqERwuSbT7okLn+pn5ZcjqamYIPHHSSkazFRsXhzc0j 6t6iPIWhaaT2StoouWiInZLI/rkucuPT6x02GgKqXaEONyrf3CUhc6WTbudaKqVgUekj Ipvq9TowKmEBxjz1S4x7PtWCbf6HPKAyU+oCAwa03quI3lCY27XvF8Tn5DnUf27G3E0o x7mLapaw09ynuf772Ew2nEO4mC1DaRzcfj4GiD7NkKIYin+yHbQJv9d6UgMC1nSr1onw t0uLW4WSiInl8hEewbEl2FKjou2hNu9Ls5YXHPpzBTSsEDsvd+ZmpQ9zwA7ve0thkkJN J+ig== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=aQcFyP+j; spf=pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-efi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v3-v6si5698094plo.32.2018.03.12.02.44.13; Mon, 12 Mar 2018 02:44:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=aQcFyP+j; spf=pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-efi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752438AbeCLJoM (ORCPT + 2 others); Mon, 12 Mar 2018 05:44:12 -0400 Received: from mail-wr0-f196.google.com ([209.85.128.196]:37770 "EHLO mail-wr0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752045AbeCLJoK (ORCPT ); Mon, 12 Mar 2018 05:44:10 -0400 Received: by mail-wr0-f196.google.com with SMTP id z12so14944750wrg.4 for ; Mon, 12 Mar 2018 02:44:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NncxohmhXveDEJQa/lHUcnh0h9ximHgVXz+Fs0XBOvg=; b=aQcFyP+jlAe1HZNTiUnbUfDADgoCmZJ+YOrKk4nmZqgD4tnqardfbqbYcCLqchGVj/ apJ6HTFLFBXXVFNFilMjVcmwXy51ZfkJmjex+p6B43JObdt920XwjPruI9Mhdp/RjSP7 J/9IOMwukCOKShAmgeYqfRPIh9WrtGIt81BTs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NncxohmhXveDEJQa/lHUcnh0h9ximHgVXz+Fs0XBOvg=; b=em6iQnlABdGJDF7q+lTJc+CEiOUUX41M7t1n2U+t+wx6VNgoAIREd80XVlx5r2Amoz E49e1NFunxCUFsYsqPEeqGYFQI0RP0LlSyF7qU7bHOQTl9ANahA9nkiFLk7YOdAcff9z rWulFi/xIqh0BD5yt/PYxF08da7oyFca+ir0ZQ7Be/vYrGa7LMicV2IWrJVi6iOr8LwR X6XartRdQEajy4c9mMvYwn/bE4PITqXvo+TnltATM6RBk1g5wsLK13DjXSBBebujwIlW ETvvtVKhbdCXAlGfuO3Y7IQbTvzRrCBu4g0RFZWMEt0mzqPHnt3ckAS5KsSXioAdhl3U HG5Q== X-Gm-Message-State: AElRT7GsoB2YxbCIUjDS4E8LipUQfEcAqpM1BTrBhjvSxnqynyCNYwpD qLc5C7a/vPH0gE7qRKCuvAir6zpGNEw= X-Received: by 10.223.160.157 with SMTP id m29mr5620320wrm.119.1520847849225; Mon, 12 Mar 2018 02:44:09 -0700 (PDT) Received: from localhost.localdomain ([105.148.128.186]) by smtp.gmail.com with ESMTPSA id d8sm3972947wmd.20.2018.03.12.02.44.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Mar 2018 02:44:08 -0700 (PDT) From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: mingo@kernel.org, Sai Praneeth , Ard Biesheuvel Subject: [PATCH rebased 2/2] x86/efi: Use efi_switch_mm() rather than manually twiddling with %cr3 Date: Mon, 12 Mar 2018 09:43:55 +0000 Message-Id: <20180312094355.5371-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180312094355.5371-1-ard.biesheuvel@linaro.org> References: <20180312094355.5371-1-ard.biesheuvel@linaro.org> Sender: linux-efi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Sai Praneeth Use helper function efi_switch_mm() to switch to/from efi_mm when invoking any UEFI runtime services. Likewise, we need to switch back to previous mm (mm context stolen by efi_mm) after the above calls return successfully. We can use efi_switch_mm() helper function only with x86_64 kernel and "efi=old_map" disabled because, x86_32 and efi=old_map do not use efi_pgd, rather they use swapper_pg_dir. Signed-off-by: Sai Praneeth Prakhya Cc: "Lee, Chun-Yi" Cc: Borislav Petkov Cc: Tony Luck Cc: Andy Lutomirski Cc: Michael S. Tsirkin Cc: Bhupesh Sharma Cc: Ricardo Neri Cc: Ravi Shankar Tested-by: Bhupesh Sharma Reviewed-by: Matt Fleming [ardb: add #include of sched/task.h for task_lock/_unlock] Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/efi.h | 25 +++++------- arch/x86/platform/efi/efi_64.c | 41 +++++++++++--------- arch/x86/platform/efi/efi_thunk_64.S | 2 +- 3 files changed, 33 insertions(+), 35 deletions(-) -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-efi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h index c62443fa7d0a..cec5fae23eb3 100644 --- a/arch/x86/include/asm/efi.h +++ b/arch/x86/include/asm/efi.h @@ -70,14 +70,13 @@ extern asmlinkage u64 efi_call(void *fp, ...); #define efi_call_phys(f, args...) efi_call((f), args) /* - * Scratch space used for switching the pagetable in the EFI stub + * struct efi_scratch - Scratch space used while switching to/from efi_mm + * @phys_stack: stack used during EFI Mixed Mode + * @prev_mm: store/restore stolen mm_struct while switching to/from efi_mm */ struct efi_scratch { - u64 r15; - u64 prev_cr3; - pgd_t *efi_pgt; - bool use_pgd; - u64 phys_stack; + u64 phys_stack; + struct mm_struct *prev_mm; } __packed; #define arch_efi_call_virt_setup() \ @@ -87,11 +86,8 @@ struct efi_scratch { __kernel_fpu_begin(); \ firmware_restrict_branch_speculation_start(); \ \ - if (efi_scratch.use_pgd) { \ - efi_scratch.prev_cr3 = __read_cr3(); \ - write_cr3((unsigned long)efi_scratch.efi_pgt); \ - __flush_tlb_all(); \ - } \ + if (!efi_enabled(EFI_OLD_MEMMAP)) \ + efi_switch_mm(&efi_mm); \ }) #define arch_efi_call_virt(p, f, args...) \ @@ -99,10 +95,8 @@ struct efi_scratch { #define arch_efi_call_virt_teardown() \ ({ \ - if (efi_scratch.use_pgd) { \ - write_cr3(efi_scratch.prev_cr3); \ - __flush_tlb_all(); \ - } \ + if (!efi_enabled(EFI_OLD_MEMMAP)) \ + efi_switch_mm(efi_scratch.prev_mm); \ \ firmware_restrict_branch_speculation_end(); \ __kernel_fpu_end(); \ @@ -145,6 +139,7 @@ extern void __init efi_dump_pagetable(void); extern void __init efi_apply_memmap_quirks(void); extern int __init efi_reuse_config(u64 tables, int nr_tables); extern void efi_delete_dummy_variable(void); +extern void efi_switch_mm(struct mm_struct *mm); struct efi_setup_data { u64 fw_vendor; diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 29b267b8cb63..4f8a9c963fbb 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -82,9 +83,8 @@ pgd_t * __init efi_call_phys_prolog(void) int n_pgds, i, j; if (!efi_enabled(EFI_OLD_MEMMAP)) { - save_pgd = (pgd_t *)__read_cr3(); - write_cr3((unsigned long)efi_scratch.efi_pgt); - goto out; + efi_switch_mm(&efi_mm); + return NULL; } early_code_mapping_set_exec(1); @@ -156,8 +156,7 @@ void __init efi_call_phys_epilog(pgd_t *save_pgd) pud_t *pud; if (!efi_enabled(EFI_OLD_MEMMAP)) { - write_cr3((unsigned long)save_pgd); - __flush_tlb_all(); + efi_switch_mm(efi_scratch.prev_mm); return; } @@ -347,13 +346,6 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) if (efi_enabled(EFI_OLD_MEMMAP)) return 0; - /* - * Since the PGD is encrypted, set the encryption mask so that when - * this value is loaded into cr3 the PGD will be decrypted during - * the pagetable walk. - */ - efi_scratch.efi_pgt = (pgd_t *)__sme_pa(pgd); - /* * It can happen that the physical address of new_memmap lands in memory * which is not mapped in the EFI page table. Therefore we need to go @@ -367,8 +359,6 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) return 1; } - efi_scratch.use_pgd = true; - /* * Certain firmware versions are way too sentimential and still believe * they are exclusive and unquestionable owners of the first physical page, @@ -627,6 +617,22 @@ void __init efi_dump_pagetable(void) #endif } +/* + * Makes the calling thread switch to/from efi_mm context. Can be used + * for SetVirtualAddressMap() i.e. current->active_mm == init_mm as well + * as during efi runtime calls i.e current->active_mm == current_mm. + * We are not mm_dropping()/mm_grabbing() any mm, because we are not + * losing/creating any references. + */ +void efi_switch_mm(struct mm_struct *mm) +{ + task_lock(current); + efi_scratch.prev_mm = current->active_mm; + current->active_mm = mm; + switch_mm(efi_scratch.prev_mm, mm, NULL); + task_unlock(current); +} + #ifdef CONFIG_EFI_MIXED extern efi_status_t efi64_thunk(u32, ...); @@ -680,16 +686,13 @@ efi_status_t efi_thunk_set_virtual_address_map( efi_sync_low_kernel_mappings(); local_irq_save(flags); - efi_scratch.prev_cr3 = __read_cr3(); - write_cr3((unsigned long)efi_scratch.efi_pgt); - __flush_tlb_all(); + efi_switch_mm(&efi_mm); func = (u32)(unsigned long)phys_set_virtual_address_map; status = efi64_thunk(func, memory_map_size, descriptor_size, descriptor_version, virtual_map); - write_cr3(efi_scratch.prev_cr3); - __flush_tlb_all(); + efi_switch_mm(efi_scratch.prev_mm); local_irq_restore(flags); return status; diff --git a/arch/x86/platform/efi/efi_thunk_64.S b/arch/x86/platform/efi/efi_thunk_64.S index 189b218da87c..46c58b08739c 100644 --- a/arch/x86/platform/efi/efi_thunk_64.S +++ b/arch/x86/platform/efi/efi_thunk_64.S @@ -33,7 +33,7 @@ ENTRY(efi64_thunk) * Switch to 1:1 mapped 32-bit stack pointer. */ movq %rsp, efi_saved_sp(%rip) - movq efi_scratch+25(%rip), %rsp + movq efi_scratch(%rip), %rsp /* * Calculate the physical address of the kernel text.