From patchwork Tue Nov 18 12:57:06 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 41022 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f200.google.com (mail-lb0-f200.google.com [209.85.217.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 3D562241C9 for ; Tue, 18 Nov 2014 13:01:39 +0000 (UTC) Received: by mail-lb0-f200.google.com with SMTP id f15sf12913608lbj.7 for ; Tue, 18 Nov 2014 05:01:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id :in-reply-to:references:cc:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:mime-version:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:content-type:content-transfer-encoding; bh=EkqqEraZjCU6Aj+3cpErd3WX0NFv4EnhoKZd7mYQCDg=; b=h1brfYcabBnSRYw6Yd6CPk29vGU6EbkBA2tzzvySIV53Xs0S7HHvifuqUwdCRtWuAA EACE3Qrruq1EfOtQ7yaqIu9az6WMkUOdzjYexNAEKb69lR9jpbUFB8mYxvjG/okz+EKJ VPgC385dHsuTmQ7+1ikQeQZ36TYhRSDFdW7/Z5QI/izu+gaNcnAH4a/AcLqY3e5JYxWj +iNPb+e30EDE9tBgDSR5cOafr5fFPw7sTgueoR7ECbqSQ29Krp3UvlV7vwNjLXpY5Dtm yStp9z0cw1Ga9o7VHxWKRXb31VIBZ9iefALmcZ34o/XJnxLXC/pmuvU4tIvrksHeWfTJ jUCw== X-Gm-Message-State: ALoCoQkcJiShZvOw18CcaMqCbbB4cfzvnhlvGB2VHIV7jdKvcQN/YXIuco/LIRNmEPdOLQVNRIYk X-Received: by 10.180.8.9 with SMTP id n9mr5669562wia.6.1416315698151; Tue, 18 Nov 2014 05:01:38 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.36.67 with SMTP id o3ls800823laj.86.gmail; Tue, 18 Nov 2014 05:01:38 -0800 (PST) X-Received: by 10.112.239.12 with SMTP id vo12mr18832347lbc.81.1416315697992; Tue, 18 Nov 2014 05:01:37 -0800 (PST) Received: from mail-la0-f42.google.com (mail-la0-f42.google.com. [209.85.215.42]) by mx.google.com with ESMTPS id t9si55643330lag.56.2014.11.18.05.01.37 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Nov 2014 05:01:37 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.42 as permitted sender) client-ip=209.85.215.42; Received: by mail-la0-f42.google.com with SMTP id s18so6262658lam.29 for ; Tue, 18 Nov 2014 05:01:37 -0800 (PST) X-Received: by 10.112.45.102 with SMTP id l6mr6798699lbm.46.1416315697841; Tue, 18 Nov 2014 05:01:37 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.184.201 with SMTP id ew9csp1323165lbc; Tue, 18 Nov 2014 05:01:36 -0800 (PST) X-Received: by 10.68.247.168 with SMTP id yf8mr16548533pbc.40.1416315695252; Tue, 18 Nov 2014 05:01:35 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id x3si37995765pda.177.2014.11.18.05.01.34 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Nov 2014 05:01:35 -0800 (PST) Received-SPF: none (google.com: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org does not designate permitted sender hosts) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XqiNX-0003aL-8E; Tue, 18 Nov 2014 12:59:31 +0000 Received: from mail-wg0-f52.google.com ([74.125.82.52]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XqiLs-0002T4-SF for linux-arm-kernel@lists.infradead.org; Tue, 18 Nov 2014 12:57:50 +0000 Received: by mail-wg0-f52.google.com with SMTP id a1so3938846wgh.39 for ; Tue, 18 Nov 2014 04:57:26 -0800 (PST) X-Received: by 10.195.11.6 with SMTP id ee6mr48277230wjd.95.1416315446692; Tue, 18 Nov 2014 04:57:26 -0800 (PST) Received: from ards-macbook-pro.local (cag06-7-83-153-85-71.fbx.proxad.net. [83.153.85.71]) by mx.google.com with ESMTPSA id cz3sm55726799wjb.23.2014.11.18.04.57.24 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 18 Nov 2014 04:57:25 -0800 (PST) From: Ard Biesheuvel To: leif.lindholm@linaro.org, roy.franz@linaro.org, linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, msalter@redhat.com, dyoung@redhat.com, linux-efi@vger.kernel.org, matt.fleming@intel.com, will.deacon@arm.com, catalin.marinas@arm.com, grant.likely@linaro.org, geoff.levand@linaro.org Subject: [PATCH v3 07/13] arm64/efi: move SetVirtualAddressMap() to UEFI stub Date: Tue, 18 Nov 2014 13:57:06 +0100 Message-Id: <1416315432-8534-8-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1416315432-8534-1-git-send-email-ard.biesheuvel@linaro.org> References: <1416315432-8534-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20141118_045749_263008_96336F9D X-CRM114-Status: GOOD ( 31.75 ) X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [74.125.82.52 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.52 listed in wl.mailspike.net] -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders Cc: Ard Biesheuvel X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.42 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 In order to support kexec, the kernel needs to be able to deal with the state of the UEFI firmware after SetVirtualAddressMap() has been called. To avoid having separate code paths for non-kexec and kexec, let's move the call to SetVirtualAddressMap() to the stub: this will guarantee us that it will only be called once (since the stub is not executed during kexec), and ensures that the UEFI state is identical between kexec and normal boot. This implies that the layout of the virtual mapping needs to be created by the stub as well. All regions are rounded up to a naturally aligned multiple of 64 KB (for compatibility with 64k pages kernels) and recorded in the UEFI memory map. The kernel proper reads those values and installs the mappings in a dedicated set of page tables that are swapped in during UEFI Runtime Services calls. Signed-off-by: Ard Biesheuvel --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/efi.h | 16 +++-- arch/arm64/kernel/efi.c | 128 ++++++++----------------------------- arch/arm64/kernel/setup.c | 1 + drivers/firmware/efi/libstub/fdt.c | 107 ++++++++++++++++++++++++++++++- 5 files changed, 144 insertions(+), 109 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 2c3c2ca6f8bc..a6d00b7cf60b 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -393,6 +393,7 @@ config EFI select EFI_RUNTIME_WRAPPERS select EFI_STUB select EFI_ARMSTUB + select EFI_VIRTMAP default y help This option provides support for runtime services provided diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index a34fd3b12e2b..e2cea16f3bd7 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -14,21 +14,27 @@ extern void efi_idmap_init(void); #define efi_call_virt(f, ...) \ ({ \ - efi_##f##_t *__f = efi.systab->runtime->f; \ + efi_##f##_t *__f; \ efi_status_t __s; \ \ - kernel_neon_begin(); \ + kernel_neon_begin(); /* disables preemption */ \ + efi_virtmap_load(); \ + __f = efi.systab->runtime->f; \ __s = __f(__VA_ARGS__); \ + efi_virtmap_unload(); \ kernel_neon_end(); \ __s; \ }) #define __efi_call_virt(f, ...) \ ({ \ - efi_##f##_t *__f = efi.systab->runtime->f; \ + efi_##f##_t *__f; \ \ - kernel_neon_begin(); \ + kernel_neon_begin(); /* disables preemption */ \ + efi_virtmap_load(); \ + __f = efi.systab->runtime->f; \ __f(__VA_ARGS__); \ + efi_virtmap_unload(); \ kernel_neon_end(); \ }) @@ -44,4 +50,6 @@ extern void efi_idmap_init(void); #define efi_call_early(f, ...) sys_table_arg->boottime->f(__VA_ARGS__) +void efi_set_pgd(struct mm_struct *mm); + #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index 6fac253bc783..beb5a79d32c3 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -25,11 +25,10 @@ #include #include #include +#include struct efi_memory_map memmap; -static efi_runtime_services_t *runtime; - static u64 efi_system_table; static int uefi_debug __initdata; @@ -72,6 +71,8 @@ static void __init efi_setup_idmap(void) static int __init uefi_init(void) { efi_char16_t *c16; + void *config_tables; + u64 table_size; char vendor[100] = "unknown"; int i, retval; @@ -99,7 +100,7 @@ static int __init uefi_init(void) efi.systab->hdr.revision & 0xffff); /* Show what we know for posterity */ - c16 = early_memremap(efi.systab->fw_vendor, + c16 = early_memremap(efi_to_phys(efi.systab->fw_vendor), sizeof(vendor)); if (c16) { for (i = 0; i < (int) sizeof(vendor) - 1 && *c16; ++i) @@ -112,8 +113,14 @@ static int __init uefi_init(void) efi.systab->hdr.revision >> 16, efi.systab->hdr.revision & 0xffff, vendor); - retval = efi_config_init(NULL); + table_size = sizeof(efi_config_table_64_t) * efi.systab->nr_tables; + config_tables = early_memremap(efi_to_phys(efi.systab->tables), + table_size); + + retval = efi_config_parse_tables(config_tables, + efi.systab->nr_tables, NULL); + early_memunmap(config_tables, table_size); out: early_memunmap(efi.systab, sizeof(efi_system_table_t)); return retval; @@ -328,51 +335,9 @@ void __init efi_idmap_init(void) efi_setup_idmap(); } -static int __init remap_region(efi_memory_desc_t *md, void **new) -{ - u64 paddr, vaddr, npages, size; - - paddr = md->phys_addr; - npages = md->num_pages; - memrange_efi_to_native(&paddr, &npages); - size = npages << PAGE_SHIFT; - - if (is_normal_ram(md)) - vaddr = (__force u64)ioremap_cache(paddr, size); - else - vaddr = (__force u64)ioremap(paddr, size); - - if (!vaddr) { - pr_err("Unable to remap 0x%llx pages @ %p\n", - npages, (void *)paddr); - return 0; - } - - /* adjust for any rounding when EFI and system pagesize differs */ - md->virt_addr = vaddr + (md->phys_addr - paddr); - - if (uefi_debug) - pr_info(" EFI remap 0x%012llx => %p\n", - md->phys_addr, (void *)md->virt_addr); - - memcpy(*new, md, memmap.desc_size); - *new += memmap.desc_size; - - return 1; -} - -/* - * Switch UEFI from an identity map to a kernel virtual map - */ static int __init arm64_enter_virtual_mode(void) { - efi_memory_desc_t *md; - phys_addr_t virtmap_phys; - void *virtmap, *virt_md; - efi_status_t status; u64 mapsize; - int count = 0; - unsigned long flags; if (!efi_enabled(EFI_BOOT)) { pr_info("EFI services will not be available.\n"); @@ -395,79 +360,28 @@ static int __init arm64_enter_virtual_mode(void) efi.memmap = &memmap; - /* Map the runtime regions */ - virtmap = kmalloc(mapsize, GFP_KERNEL); - if (!virtmap) { - pr_err("Failed to allocate EFI virtual memmap\n"); - return -1; - } - virtmap_phys = virt_to_phys(virtmap); - virt_md = virtmap; - - for_each_efi_memory_desc(&memmap, md) { - if (!(md->attribute & EFI_MEMORY_RUNTIME)) - continue; - if (!remap_region(md, &virt_md)) - goto err_unmap; - ++count; - } - - efi.systab = (__force void *)efi_lookup_mapped_addr(efi_system_table); + efi.systab = (__force void *)ioremap_cache(efi_system_table, + sizeof(efi_system_table_t)); if (!efi.systab) { - /* - * If we have no virtual mapping for the System Table at this - * point, the memory map doesn't cover the physical offset where - * it resides. This means the System Table will be inaccessible - * to Runtime Services themselves once the virtual mapping is - * installed. - */ - pr_err("Failed to remap EFI System Table -- buggy firmware?\n"); - goto err_unmap; + pr_err("Failed to remap EFI System Table\n"); + return -1; } set_bit(EFI_SYSTEM_TABLES, &efi.flags); - local_irq_save(flags); - cpu_switch_mm(idmap_pg_dir, &init_mm); - - /* Call SetVirtualAddressMap with the physical address of the map */ - runtime = efi.systab->runtime; - efi.set_virtual_address_map = runtime->set_virtual_address_map; - - status = efi.set_virtual_address_map(count * memmap.desc_size, - memmap.desc_size, - memmap.desc_version, - (efi_memory_desc_t *)virtmap_phys); - cpu_set_reserved_ttbr0(); - flush_tlb_all(); - local_irq_restore(flags); - - kfree(virtmap); - free_boot_services(); - if (status != EFI_SUCCESS) { - pr_err("Failed to set EFI virtual address map! [%lx]\n", - status); + if (!efi_enabled(EFI_VIRTMAP)) { + pr_err("No UEFI virtual mapping was installed -- runtime services will not be available\n"); return -1; } /* Set up runtime services function pointers */ - runtime = efi.systab->runtime; efi_native_runtime_setup(); set_bit(EFI_RUNTIME_SERVICES, &efi.flags); efi.runtime_version = efi.systab->hdr.revision; return 0; - -err_unmap: - /* unmap all mappings that succeeded: there are 'count' of those */ - for (virt_md = virtmap; count--; virt_md += memmap.desc_size) { - md = virt_md; - iounmap((__force void __iomem *)md->virt_addr); - } - kfree(virtmap); - return -1; } early_initcall(arm64_enter_virtual_mode); @@ -484,3 +398,11 @@ static int __init arm64_dmi_init(void) return 0; } core_initcall(arm64_dmi_init); + +void efi_set_pgd(struct mm_struct *mm) +{ + cpu_switch_mm(mm->pgd, mm); + flush_tlb_all(); + if (icache_is_aivivt()) + __flush_icache_all(); +} diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 2437196cc5d4..ac19b2d6a3fc 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -393,6 +393,7 @@ void __init setup_arch(char **cmdline_p) request_standard_resources(); efi_idmap_init(); + efi_virtmap_init(); unflatten_device_tree(); diff --git a/drivers/firmware/efi/libstub/fdt.c b/drivers/firmware/efi/libstub/fdt.c index c846a9608cbd..7129ed55e4bb 100644 --- a/drivers/firmware/efi/libstub/fdt.c +++ b/drivers/firmware/efi/libstub/fdt.c @@ -168,6 +168,68 @@ fdt_set_fail: #endif /* + * This is the base address at which to start allocating virtual memory ranges + * for UEFI Runtime Services. This is a userland range so that we can use any + * allocation we choose, and eliminate the risk of a conflict after kexec. + */ +#define EFI_RT_VIRTUAL_BASE 0x40000000 + +static void update_memory_map(efi_memory_desc_t *memory_map, + unsigned long map_size, unsigned long desc_size, + int *count) +{ + u64 efi_virt_base = EFI_RT_VIRTUAL_BASE; + union { + efi_memory_desc_t entry; + u8 pad[desc_size]; + } *p, *q, tmp; + int i = map_size / desc_size; + + p = (void *)memory_map; + for (q = p; i >= 0; i--, q++) { + u64 paddr, size; + + if (!(q->entry.attribute & EFI_MEMORY_RUNTIME)) + continue; + + /* + * Swap the entries around so that all EFI_MEMORY_RUNTIME + * entries bubble to the top. This will allow us to reuse the + * table as input to SetVirtualAddressMap(). + */ + if (q != p) { + tmp = *p; + *p = *q; + *q = tmp; + } + + /* + * Make the mapping compatible with 64k pages: this allows + * a 4k page size kernel to kexec a 64k page size kernel and + * vice versa. + */ + paddr = round_down(p->entry.phys_addr, SZ_64K); + size = round_up(p->entry.num_pages * EFI_PAGE_SIZE + + p->entry.phys_addr - paddr, SZ_64K); + + /* + * Avoid wasting memory on PTEs by choosing a virtual base that + * is compatible with section mappings if this region has the + * appropriate size and physical alignment. (Sections are 2 MB + * on 4k granule kernels) + */ + if (IS_ALIGNED(p->entry.phys_addr, SZ_2M) && size >= SZ_2M) + efi_virt_base = round_up(efi_virt_base, SZ_2M); + + p->entry.virt_addr = efi_virt_base + p->entry.phys_addr - paddr; + efi_virt_base += size; + + ++p; + ++*count; + } +} + +/* * Allocate memory for a new FDT, then add EFI, commandline, and * initrd related fields to the FDT. This routine increases the * FDT allocation size until the allocated memory is large @@ -196,6 +258,7 @@ efi_status_t allocate_new_fdt_and_exit_boot(efi_system_table_t *sys_table, efi_memory_desc_t *memory_map; unsigned long new_fdt_size; efi_status_t status; + int runtime_entry_count = 0; /* * Estimate size of new FDT, and allocate memory for it. We @@ -248,12 +311,52 @@ efi_status_t allocate_new_fdt_and_exit_boot(efi_system_table_t *sys_table, } } + /* + * Update the memory map with virtual addresses, and reorder the entries + * so that we can pass it straight into SetVirtualAddressMap() + */ + update_memory_map(memory_map, map_size, desc_size, + &runtime_entry_count); + + pr_efi(sys_table, + "Exiting boot services and installing virtual address map...\n"); + /* Now we are ready to exit_boot_services.*/ status = sys_table->boottime->exit_boot_services(handle, mmap_key); + if (status == EFI_SUCCESS) { + efi_set_virtual_address_map_t *svam; + + /* Install the new virtual address map */ + svam = sys_table->runtime->set_virtual_address_map; + status = svam(runtime_entry_count * desc_size, desc_size, + desc_ver, memory_map); - if (status == EFI_SUCCESS) - return status; + /* + * We are beyond the point of no return here, so if the call to + * SetVirtualAddressMap() failed, we need to signal that to the + * incoming kernel but proceed normally otherwise. + */ + if (status != EFI_SUCCESS) { + int i; + + /* + * Set the virtual address field of all + * EFI_MEMORY_RUNTIME entries to 0. This will signal + * the incoming kernel that no virtual translation has + * been installed. + */ + for (i = 0; i < map_size; i += desc_size) { + efi_memory_desc_t *p; + + p = (efi_memory_desc_t *)((u8 *)memory_map + i); + if (!(p->attribute & EFI_MEMORY_RUNTIME)) + break; + p->virt_addr = 0; + } + } + return EFI_SUCCESS; + } pr_efi_err(sys_table, "Exit boot services failed.\n");