From patchwork Mon May 11 07:13:03 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 48212 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f198.google.com (mail-lb0-f198.google.com [209.85.217.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id A13E12121F for ; Mon, 11 May 2015 07:17:45 +0000 (UTC) Received: by lbcne10 with SMTP id ne10sf36280703lbc.1 for ; Mon, 11 May 2015 00:17:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id :in-reply-to:references:cc:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:mime-version :content-type:content-transfer-encoding:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list; bh=yhp892xFwjd0h9W2oi7CQp00eRMwlnPyTTOkCSH1WEc=; b=Pkkrq1CtYnE4qIsvO5A0WjfUeN9BmaSpHvKVm+V/gzktfiWKd1vPeJpKJ9PTI81xhI Yx37f2w8AurZfjIXZE8M5UxrnMn08Alm4XeWYqRNfkkImARMw+BakkYRabdNrP14hoOJ U4Nm3OcJFAwM/ogVJQOHCRuL1/ACIEIag0Kr+Rxov/XvsrndC1IpkAkfgIkX3uuycDqv H7T2QUKoIB5SETEnaMbTZkFC3QOTSKR8jT6uFOHHAd967jNwPv/tTBavUPGWQZ8EyW2Y PslwREmiSJIbeo0tRfint9X7Y5idxNq77cx40p6uum0fU7/N+R9DfO7n6FIo6Qwl8T5B oUxw== X-Gm-Message-State: ALoCoQnom/a7vHqtmKW8O0B+1qdZ5vIBWL6zzf2yGOVXBfl+shWj9qROXNkv/B0Jv3gSAcb9YhWv X-Received: by 10.152.26.134 with SMTP id l6mr6689067lag.10.1431328664610; Mon, 11 May 2015 00:17:44 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.2.9 with SMTP id 9ls670122laq.89.gmail; Mon, 11 May 2015 00:17:44 -0700 (PDT) X-Received: by 10.153.6.36 with SMTP id cr4mr7255093lad.56.1431328664438; Mon, 11 May 2015 00:17:44 -0700 (PDT) Received: from mail-la0-f50.google.com (mail-la0-f50.google.com. [209.85.215.50]) by mx.google.com with ESMTPS id e7si7870465lbc.62.2015.05.11.00.17.44 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 11 May 2015 00:17:44 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) client-ip=209.85.215.50; Received: by labbd9 with SMTP id bd9so87049595lab.2 for ; Mon, 11 May 2015 00:17:44 -0700 (PDT) X-Received: by 10.112.140.231 with SMTP id rj7mr6878336lbb.76.1431328664154; Mon, 11 May 2015 00:17:44 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.108.230 with SMTP id hn6csp1346279lbb; Mon, 11 May 2015 00:17:43 -0700 (PDT) X-Received: by 10.68.139.103 with SMTP id qx7mr16874520pbb.133.1431328662376; Mon, 11 May 2015 00:17:42 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id d4si16860983pdg.116.2015.05.11.00.17.41 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 11 May 2015 00:17:42 -0700 (PDT) Received-SPF: none (google.com: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org does not designate permitted sender hosts) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YrhwI-0003c4-N9; Mon, 11 May 2015 07:15:46 +0000 Received: from mail-wg0-f48.google.com ([74.125.82.48]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1YrhuL-0000jR-Fc for linux-arm-kernel@lists.infradead.org; Mon, 11 May 2015 07:13:46 +0000 Received: by wgbhc8 with SMTP id hc8so18737998wgb.2 for ; Mon, 11 May 2015 00:13:23 -0700 (PDT) X-Received: by 10.180.78.199 with SMTP id d7mr17757825wix.94.1431328403276; Mon, 11 May 2015 00:13:23 -0700 (PDT) Received: from localhost.localdomain (cag06-7-83-153-85-71.fbx.proxad.net. [83.153.85.71]) by mx.google.com with ESMTPSA id o5sm10728933wia.0.2015.05.11.00.13.21 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 11 May 2015 00:13:22 -0700 (PDT) From: Ard Biesheuvel To: catalin.marinas@arm.com, mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org Subject: [PATCH 05/10] arm64: split off early mapping code from early_fixmap_init() Date: Mon, 11 May 2015 09:13:03 +0200 Message-Id: <1431328388-3051-6-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1431328388-3051-1-git-send-email-ard.biesheuvel@linaro.org> References: <1431328388-3051-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150511_001345_722813_E72713B1 X-CRM114-Status: GOOD ( 14.22 ) X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [74.125.82.48 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.48 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders Cc: Ard Biesheuvel X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 This splits off and generalises the population of the statically allocated fixmap page tables so that we may reuse it later for the linear mapping once we move the kernel text mapping out of it. This also involves taking into account that table entries at any of the levels we are populating may have been populated already, since the fixmap mapping might not be disjoint up to the pgd level anymore from other early mappings. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/compiler.h | 2 ++ arch/arm64/kernel/vmlinux.lds.S | 12 ++++---- arch/arm64/mm/mmu.c | 60 +++++++++++++++++++++++++++------------ 3 files changed, 51 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/compiler.h b/arch/arm64/include/asm/compiler.h index ee35fd0f2236..dd342af63673 100644 --- a/arch/arm64/include/asm/compiler.h +++ b/arch/arm64/include/asm/compiler.h @@ -27,4 +27,6 @@ */ #define __asmeq(x, y) ".ifnc " x "," y " ; .err ; .endif\n\t" +#define __pgdir __attribute__((section(".pgdir"),aligned(PAGE_SIZE))) + #endif /* __ASM_COMPILER_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 98073332e2d0..ceec4def354b 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -160,11 +160,13 @@ SECTIONS BSS_SECTION(0, 0, 0) - . = ALIGN(PAGE_SIZE); - idmap_pg_dir = .; - . += IDMAP_DIR_SIZE; - swapper_pg_dir = .; - . += SWAPPER_DIR_SIZE; + .pgdir (NOLOAD) : ALIGN(PAGE_SIZE) { + idmap_pg_dir = .; + . += IDMAP_DIR_SIZE; + swapper_pg_dir = .; + . += SWAPPER_DIR_SIZE; + *(.pgdir) + } _end = .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 82d3435bf14f..f64a817af469 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -342,6 +342,44 @@ static void __init __map_memblock(phys_addr_t start, phys_addr_t end) } #endif +struct bootstrap_pgtables { + pte_t pte[PTRS_PER_PTE]; + pmd_t pmd[PTRS_PER_PMD > 1 ? PTRS_PER_PMD : 0]; + pud_t pud[PTRS_PER_PUD > 1 ? PTRS_PER_PUD : 0]; +}; + +static void __init bootstrap_early_mapping(unsigned long addr, + struct bootstrap_pgtables *reg, + bool pte_level) +{ + pgd_t *pgd; + pud_t *pud; + pmd_t *pmd; + + pgd = pgd_offset_k(addr); + if (pgd_none(*pgd)) { + clear_page(reg->pud); + memblock_reserve(__pa(reg->pud), PAGE_SIZE); + pgd_populate(&init_mm, pgd, reg->pud); + } + pud = pud_offset(pgd, addr); + if (pud_none(*pud)) { + clear_page(reg->pmd); + memblock_reserve(__pa(reg->pmd), PAGE_SIZE); + pud_populate(&init_mm, pud, reg->pmd); + } + + if (!pte_level) + return; + + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) { + clear_page(reg->pte); + memblock_reserve(__pa(reg->pte), PAGE_SIZE); + pmd_populate_kernel(&init_mm, pmd, reg->pte); + } +} + static void __init map_mem(void) { struct memblock_region *reg; @@ -555,14 +593,6 @@ void vmemmap_free(unsigned long start, unsigned long end) } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ -static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss; -#if CONFIG_PGTABLE_LEVELS > 2 -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss; -#endif -#if CONFIG_PGTABLE_LEVELS > 3 -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss; -#endif - static inline pud_t * fixmap_pud(unsigned long addr) { pgd_t *pgd = pgd_offset_k(addr); @@ -592,21 +622,15 @@ static inline pte_t * fixmap_pte(unsigned long addr) void __init early_fixmap_init(void) { - pgd_t *pgd; - pud_t *pud; + static struct bootstrap_pgtables fixmap_bs_pgtables __pgdir; pmd_t *pmd; - unsigned long addr = FIXADDR_START; - pgd = pgd_offset_k(addr); - pgd_populate(&init_mm, pgd, bm_pud); - pud = pud_offset(pgd, addr); - pud_populate(&init_mm, pud, bm_pmd); - pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, bm_pte); + bootstrap_early_mapping(FIXADDR_START, &fixmap_bs_pgtables, true); + pmd = fixmap_pmd(FIXADDR_START); /* * The boot-ioremap range spans multiple pmds, for which - * we are not preparted: + * we are not prepared: */ BUILD_BUG_ON((__fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT) != (__fix_to_virt(FIX_BTMAP_END) >> PMD_SHIFT));