From patchwork Wed Dec 30 15:26:03 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 59082 Delivered-To: patch@linaro.org Received: by 10.112.130.2 with SMTP id oa2csp2909512lbb; Wed, 30 Dec 2015 07:26:55 -0800 (PST) X-Received: by 10.98.13.14 with SMTP id v14mr94607572pfi.21.1451489211792; Wed, 30 Dec 2015 07:26:51 -0800 (PST) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p88si49100770pfi.246.2015.12.30.07.26.51; Wed, 30 Dec 2015 07:26:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dkim=neutral (body hash did not verify) header.i=@linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754893AbbL3P0q (ORCPT + 29 others); Wed, 30 Dec 2015 10:26:46 -0500 Received: from mail-wm0-f42.google.com ([74.125.82.42]:34596 "EHLO mail-wm0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754051AbbL3P02 (ORCPT ); Wed, 30 Dec 2015 10:26:28 -0500 Received: by mail-wm0-f42.google.com with SMTP id u188so42478381wmu.1 for ; Wed, 30 Dec 2015 07:26:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dowfedk/Inpk9DlsW44NEm/51N6lBvPtHcXfQm6XdoM=; b=hktyS0cAKikaZJJht+rPqN7eRNZTZzm9QZT2nmsUAgCoa5XlSYwe4B4k04MIXrKKPP V44fmdVuUeIIvjInwJ8qQlwolYeWeJSN6VmzVmkkOdB0Rjk/oiya3/Tp2o1kVARhykBz rEZjw7q4vA0PN7dwPvtsWzBbC2T0IVvE7MAUo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dowfedk/Inpk9DlsW44NEm/51N6lBvPtHcXfQm6XdoM=; b=GXYtz9gLwYv8/wgMhbyDDOiYSx+7r5dR7JVHvT3uISP9ygNtwPXPuVt5CouJI7luSh pM3kQJwOkoSa/C/Py+VlRXdaGGTgeW7FxChlWuG8nTQMe98QZhn8raqfaqEq3ldK1Mxj brsv2kXp3DufY0I/1Unw5HEC31tUb2QRPJQ/e11JNTXepAoD/7nIzAbkb/EyljFttHeM lp1kLk2T4JqObIXywAUUBz4IGtBaiTXLONN+eQcmkNbrobEF1fjGba3VQTdwm76lDqF3 SStOvduwPJiUmOkSH2M1sUxsv65a7mapH/9fp22L643l/SNKWClVEQ8EWisepVJ1AlhH 8FaQ== X-Gm-Message-State: ALoCoQlzeiYc7tM030nfYnwqOhYdcPjt7DPAIyAHt5dZWpPY2rMMERwGdgcpeNpY7yXjWQY2cWySbvFKJbO0nRsZKIxh+4ApiA== X-Received: by 10.28.183.132 with SMTP id h126mr49888236wmf.6.1451489187622; Wed, 30 Dec 2015 07:26:27 -0800 (PST) Received: from localhost.localdomain (cag06-7-83-153-85-71.fbx.proxad.net. [83.153.85.71]) by smtp.gmail.com with ESMTPSA id z137sm31510865wmc.8.2015.12.30.07.26.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 30 Dec 2015 07:26:27 -0800 (PST) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, kernel-hardening@lists.openwall.com, will.deacon@arm.com, catalin.marinas@arm.com, mark.rutland@arm.com, leif.lindholm@linaro.org, keescook@chromium.org, linux-kernel@vger.kernel.org Cc: stuart.yoder@freescale.com, bhupesh.sharma@freescale.com, arnd@arndb.de, marc.zyngier@arm.com, christoffer.dall@linaro.org, Ard Biesheuvel Subject: [PATCH v2 04/13] arm64: decouple early fixmap init from linear mapping Date: Wed, 30 Dec 2015 16:26:03 +0100 Message-Id: <1451489172-17420-5-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1451489172-17420-1-git-send-email-ard.biesheuvel@linaro.org> References: <1451489172-17420-1-git-send-email-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Since the early fixmap page tables are populated using pages that are part of the static footprint of the kernel, they are covered by the initial kernel mapping, and we can refer to them without using __va/__pa translations, which are tied to the linear mapping. Instead, let's introduce __phys_to_kimg, which will be tied to the kernel virtual mapping, regardless of whether or not it intersects with the linear mapping. This will allow us to move the kernel out of the linear mapping in a subsequent patch. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/compiler.h | 2 + arch/arm64/kernel/vmlinux.lds.S | 12 +-- arch/arm64/mm/mmu.c | 85 ++++++++------------ 3 files changed, 42 insertions(+), 57 deletions(-) -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ diff --git a/arch/arm64/include/asm/compiler.h b/arch/arm64/include/asm/compiler.h index ee35fd0f2236..dd342af63673 100644 --- a/arch/arm64/include/asm/compiler.h +++ b/arch/arm64/include/asm/compiler.h @@ -27,4 +27,6 @@ */ #define __asmeq(x, y) ".ifnc " x "," y " ; .err ; .endif\n\t" +#define __pgdir __attribute__((section(".pgdir"),aligned(PAGE_SIZE))) + #endif /* __ASM_COMPILER_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index ced0dedcabcc..363c2f529951 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -160,11 +160,13 @@ SECTIONS BSS_SECTION(0, 0, 0) - . = ALIGN(PAGE_SIZE); - idmap_pg_dir = .; - . += IDMAP_DIR_SIZE; - swapper_pg_dir = .; - . += SWAPPER_DIR_SIZE; + .pgdir (NOLOAD) : ALIGN(PAGE_SIZE) { + idmap_pg_dir = .; + . += IDMAP_DIR_SIZE; + swapper_pg_dir = .; + . += SWAPPER_DIR_SIZE; + *(.pgdir) + } _end = .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 50b1de8e127b..a78fc5a882da 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -540,39 +540,11 @@ void vmemmap_free(unsigned long start, unsigned long end) } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ -static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss; -#if CONFIG_PGTABLE_LEVELS > 2 -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss; -#endif -#if CONFIG_PGTABLE_LEVELS > 3 -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss; -#endif - -static inline pud_t * fixmap_pud(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd)); - - return pud_offset(pgd, addr); -} - -static inline pmd_t * fixmap_pmd(unsigned long addr) -{ - pud_t *pud = fixmap_pud(addr); - - BUG_ON(pud_none(*pud) || pud_bad(*pud)); +static pte_t *__fixmap_pte; - return pmd_offset(pud, addr); -} - -static inline pte_t * fixmap_pte(unsigned long addr) +static inline pte_t *fixmap_pte(unsigned long addr) { - pmd_t *pmd = fixmap_pmd(addr); - - BUG_ON(pmd_none(*pmd) || pmd_bad(*pmd)); - - return pte_offset_kernel(pmd, addr); + return __fixmap_pte + pte_index(addr); } void __init early_fixmap_init(void) @@ -583,33 +555,42 @@ void __init early_fixmap_init(void) unsigned long addr = FIXADDR_START; pgd = pgd_offset_k(addr); - pgd_populate(&init_mm, pgd, bm_pud); - pud = pud_offset(pgd, addr); - pud_populate(&init_mm, pud, bm_pmd); - pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, bm_pte); +#if CONFIG_PGTABLE_LEVELS > 3 + if (pgd_none(*pgd)) { + static pud_t bm_pud[PTRS_PER_PUD] __pgdir; + + pgd_populate(&init_mm, pgd, bm_pud); + memblock_reserve(__pa(bm_pud), sizeof(bm_pud)); + } + pud = (pud_t *)__phys_to_kimg(pud_offset_phys(pgd, addr)); +#else + pud = (pud_t *)pgd; +#endif +#if CONFIG_PGTABLE_LEVELS > 2 + if (pud_none(*pud)) { + static pmd_t bm_pmd[PTRS_PER_PMD] __pgdir; + + pud_populate(&init_mm, pud, bm_pmd); + memblock_reserve(__pa(bm_pmd), sizeof(bm_pmd)); + } + pmd = (pmd_t *)__phys_to_kimg(pmd_offset_phys(pud, addr)); +#else + pmd = (pmd_t *)pud; +#endif + if (pmd_none(*pmd)) { + static pte_t bm_pte[PTRS_PER_PTE] __pgdir; + + pmd_populate_kernel(&init_mm, pmd, bm_pte); + memblock_reserve(__pa(bm_pte), sizeof(bm_pte)); + } + __fixmap_pte = (pte_t *)__phys_to_kimg(pmd_page_paddr(*pmd)); /* * The boot-ioremap range spans multiple pmds, for which - * we are not preparted: + * we are not prepared: */ BUILD_BUG_ON((__fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT) != (__fix_to_virt(FIX_BTMAP_END) >> PMD_SHIFT)); - - if ((pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_BEGIN))) - || pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_END))) { - WARN_ON(1); - pr_warn("pmd %p != %p, %p\n", - pmd, fixmap_pmd(fix_to_virt(FIX_BTMAP_BEGIN)), - fixmap_pmd(fix_to_virt(FIX_BTMAP_END))); - pr_warn("fix_to_virt(FIX_BTMAP_BEGIN): %08lx\n", - fix_to_virt(FIX_BTMAP_BEGIN)); - pr_warn("fix_to_virt(FIX_BTMAP_END): %08lx\n", - fix_to_virt(FIX_BTMAP_END)); - - pr_warn("FIX_BTMAP_END: %d\n", FIX_BTMAP_END); - pr_warn("FIX_BTMAP_BEGIN: %d\n", FIX_BTMAP_BEGIN); - } } void __set_fixmap(enum fixed_addresses idx,