From patchwork Mon Nov 16 11:23:14 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 56582 Delivered-To: patch@linaro.org Received: by 10.112.155.196 with SMTP id vy4csp1257827lbb; Mon, 16 Nov 2015 03:26:52 -0800 (PST) X-Received: by 10.68.203.132 with SMTP id kq4mr53861688pbc.82.1447673212623; Mon, 16 Nov 2015 03:26:52 -0800 (PST) Return-Path: Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id ut4si50131238pbc.128.2015.11.16.03.26.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Nov 2015 03:26:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) smtp.mailfrom=linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org; dkim=neutral (body hash did not verify) header.i=@linaro_org.20150623.gappssmtp.com Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZyHuI-0003LT-Pq; Mon, 16 Nov 2015 11:25:10 +0000 Received: from mail-wm0-x233.google.com ([2a00:1450:400c:c09::233]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZyHt9-00028F-I3 for linux-arm-kernel@lists.infradead.org; Mon, 16 Nov 2015 11:24:02 +0000 Received: by wmec201 with SMTP id c201so170934221wme.0 for ; Mon, 16 Nov 2015 03:23:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro_org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NRfRBsxSqSibIqiuAwlHQnaff/WBzuxhL2aJf3odk30=; b=LJYEasPrvy2hWJEM+Y3o0Ad+LUC7sakhghlOxWhI8VztWlS5rOspTUmSBKubJDIYen GcQwMmNynsjnNap9Aj1dbtlfbNxUJ53BEX48PpMWa3Qkmf99lP/58YRLaYiDoE/3LKHM iP9skelUDyA8NiLEg4QwEoKo8dbY1iJ3ZquYOUu4Z4O/Z2n+juj8pjWuUVv0HW1j0Tmt jb3aMgUiCUZtfhNEq+jXpByr2/YywPBJQensfhSvzvTUwPbkILMrrH6bcrXrjDmzaayq d+FRW9vfBc3s1snF51DWDDa84NlH+QajGkXJohS2+bAVDzsy2wg8Wrh6HhBMKhcFKpMC VHYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NRfRBsxSqSibIqiuAwlHQnaff/WBzuxhL2aJf3odk30=; b=J5DBnjOxeWuZhsCmgJt+L0xQkTXlZy49SmoVKy1IogWbX1L9BcG3vB//mkWv1hp7+y tBaPgGJ+rdg4+Jjz4JGcUjEBEYaZmhnX1NEn8kMh9xQr4k0BqW/RGCr52y7zCiKzS4Pd +My6I4a5/Cqilhz5yP9n6l3M/yGhvel97sctrJ+bPt2cSfbMLsZ8Lj5XKHYcXFHVzRYU hUZgQsjDZQrYzZ/TQzKjjj2sfbTTCERqUoe93JCKKbvS5e4e/J2Wu+fe+EGW0kn3L3Vl CxSGBcmVo3ZfjBWsdLg45z5ANSRNtZAa7moHtJqeVCNc/DJ2cYnMmDXU4jV0RNngm6EK munw== X-Gm-Message-State: ALoCoQnqIS36+7Yv9PbS9FEGLZ4QZjiI5pRdFKUI7XBetT+mm7KSrQSYEuiaFopc7FIlXKl39KS7 X-Received: by 10.28.46.137 with SMTP id u131mr17466144wmu.61.1447673017212; Mon, 16 Nov 2015 03:23:37 -0800 (PST) Received: from localhost.localdomain ([47.53.155.123]) by smtp.gmail.com with ESMTPSA id t126sm18062422wmd.18.2015.11.16.03.23.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 16 Nov 2015 03:23:36 -0800 (PST) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, catalin.marinas@arm.com, will.deacon@arm.com Subject: [PATCH v3 3/7] arm64: split off early mapping code from early_fixmap_init() Date: Mon, 16 Nov 2015 12:23:14 +0100 Message-Id: <1447672998-20981-4-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1447672998-20981-1-git-send-email-ard.biesheuvel@linaro.org> References: <1447672998-20981-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20151116_032359_934517_F21B4075 X-CRM114-Status: GOOD ( 15.05 ) X-Spam-Score: -2.6 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.6 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [2a00:1450:400c:c09:0:0:0:233 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ard Biesheuvel , suzuki.poulose@arm.com, james.morse@arm.com, labbott@fedoraproject.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org This splits off and generalises the population of the statically allocated fixmap page tables so that we may reuse it later for the linear mapping once we move the kernel text mapping out of it. This also involves taking into account that table entries at any of the levels we are populating may have been populated already, since the fixmap mapping might not be disjoint up to the pgd level anymore from other early mappings. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/compiler.h | 2 + arch/arm64/kernel/vmlinux.lds.S | 12 ++-- arch/arm64/mm/mmu.c | 60 ++++++++++++++------ 3 files changed, 51 insertions(+), 23 deletions(-) -- 1.9.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel diff --git a/arch/arm64/include/asm/compiler.h b/arch/arm64/include/asm/compiler.h index ee35fd0f2236..dd342af63673 100644 --- a/arch/arm64/include/asm/compiler.h +++ b/arch/arm64/include/asm/compiler.h @@ -27,4 +27,6 @@ */ #define __asmeq(x, y) ".ifnc " x "," y " ; .err ; .endif\n\t" +#define __pgdir __attribute__((section(".pgdir"),aligned(PAGE_SIZE))) + #endif /* __ASM_COMPILER_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 1ee2c3937d4e..87a596246ec7 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -164,11 +164,13 @@ SECTIONS BSS_SECTION(0, 0, 0) - . = ALIGN(PAGE_SIZE); - idmap_pg_dir = .; - . += IDMAP_DIR_SIZE; - swapper_pg_dir = .; - . += SWAPPER_DIR_SIZE; + .pgdir (NOLOAD) : ALIGN(PAGE_SIZE) { + idmap_pg_dir = .; + . += IDMAP_DIR_SIZE; + swapper_pg_dir = .; + . += SWAPPER_DIR_SIZE; + *(.pgdir) + } _end = .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 32ddd893da9a..4f397a87c2be 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -396,6 +396,44 @@ static void __init __map_memblock(phys_addr_t start, phys_addr_t end) } #endif +struct bootstrap_pgtables { + pte_t pte[PTRS_PER_PTE]; + pmd_t pmd[PTRS_PER_PMD > 1 ? PTRS_PER_PMD : 0]; + pud_t pud[PTRS_PER_PUD > 1 ? PTRS_PER_PUD : 0]; +}; + +static void __init bootstrap_early_mapping(unsigned long addr, + struct bootstrap_pgtables *reg, + bool pte_level) +{ + pgd_t *pgd; + pud_t *pud; + pmd_t *pmd; + + pgd = pgd_offset_k(addr); + if (pgd_none(*pgd)) { + clear_page(reg->pud); + memblock_reserve(__pa(reg->pud), PAGE_SIZE); + pgd_populate(&init_mm, pgd, reg->pud); + } + pud = pud_offset(pgd, addr); + if (pud_none(*pud)) { + clear_page(reg->pmd); + memblock_reserve(__pa(reg->pmd), PAGE_SIZE); + pud_populate(&init_mm, pud, reg->pmd); + } + + if (!pte_level) + return; + + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) { + clear_page(reg->pte); + memblock_reserve(__pa(reg->pte), PAGE_SIZE); + pmd_populate_kernel(&init_mm, pmd, reg->pte); + } +} + static void __init map_mem(void) { struct memblock_region *reg; @@ -598,14 +636,6 @@ void vmemmap_free(unsigned long start, unsigned long end) } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ -static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss; -#if CONFIG_PGTABLE_LEVELS > 2 -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss; -#endif -#if CONFIG_PGTABLE_LEVELS > 3 -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss; -#endif - static inline pud_t * fixmap_pud(unsigned long addr) { pgd_t *pgd = pgd_offset_k(addr); @@ -635,21 +665,15 @@ static inline pte_t * fixmap_pte(unsigned long addr) void __init early_fixmap_init(void) { - pgd_t *pgd; - pud_t *pud; + static struct bootstrap_pgtables fixmap_bs_pgtables __pgdir; pmd_t *pmd; - unsigned long addr = FIXADDR_START; - pgd = pgd_offset_k(addr); - pgd_populate(&init_mm, pgd, bm_pud); - pud = pud_offset(pgd, addr); - pud_populate(&init_mm, pud, bm_pmd); - pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, bm_pte); + bootstrap_early_mapping(FIXADDR_START, &fixmap_bs_pgtables, true); + pmd = fixmap_pmd(FIXADDR_START); /* * The boot-ioremap range spans multiple pmds, for which - * we are not preparted: + * we are not prepared: */ BUILD_BUG_ON((__fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT) != (__fix_to_virt(FIX_BTMAP_END) >> PMD_SHIFT));