From patchwork Wed Sep 23 00:37:39 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 54017 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f69.google.com (mail-la0-f69.google.com [209.85.215.69]) by patches.linaro.org (Postfix) with ESMTPS id E274C22D91 for ; Wed, 23 Sep 2015 00:41:33 +0000 (UTC) Received: by lacao8 with SMTP id ao8sf4024187lac.3 for ; Tue, 22 Sep 2015 17:41:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id :in-reply-to:references:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:cc:mime-version :content-type:content-transfer-encoding:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list; bh=o9mNzODg6XSbRc28txXyh9wiLFk4rbagqMEAd9JBfXQ=; b=DA7IP5MaZvMzI6p6lCoSXIhx5hbbG9dtK+kQIyfayDXumZ1wEp8/9Nd+AdCjZiUtMC S/KklcbxKS01QVs/d8LWUjyKubCYvwlaaFe0WtzpjIeYCJBz5a24GZWFE82kAKme5ocU tTUyMoM/CXUcle56kRDRow6UDqvcTQRPVPX/KKMWphdlRfxaSLTsMezQDaIXSa6YAJLP 7HawiR7QHo2XLS9cKv3oDJe4lIop96XNWpMrEPqW4Y53yW+Sba290tY2oxkezeVmKKBU cb/2DUTmzvB8CO4JSbSQDoHtBSpzXtOMavpNP2aj8PaVbNGPloPtOw2DqkaZ1UNn7SGf E5gw== X-Gm-Message-State: ALoCoQnjuIeGdQvbP78o4w9qsEtJWGXX05SsTxHQsQNlEsvgp/ScvVx14Qr2R4couJysil0zZVPc X-Received: by 10.194.109.233 with SMTP id hv9mr4792832wjb.1.1442968892643; Tue, 22 Sep 2015 17:41:32 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.25.166.200 with SMTP id p191ls90049lfe.37.gmail; Tue, 22 Sep 2015 17:41:32 -0700 (PDT) X-Received: by 10.152.43.138 with SMTP id w10mr3942548lal.104.1442968892494; Tue, 22 Sep 2015 17:41:32 -0700 (PDT) Received: from mail-la0-f42.google.com (mail-la0-f42.google.com. [209.85.215.42]) by mx.google.com with ESMTPS id h15si1596814lfb.126.2015.09.22.17.41.32 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Sep 2015 17:41:32 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.42 as permitted sender) client-ip=209.85.215.42; Received: by lahh2 with SMTP id h2so8565122lah.0 for ; Tue, 22 Sep 2015 17:41:32 -0700 (PDT) X-Received: by 10.112.64.72 with SMTP id m8mr10339575lbs.41.1442968892366; Tue, 22 Sep 2015 17:41:32 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.59.35 with SMTP id w3csp789863lbq; Tue, 22 Sep 2015 17:41:31 -0700 (PDT) X-Received: by 10.67.15.36 with SMTP id fl4mr33998220pad.152.1442968891311; Tue, 22 Sep 2015 17:41:31 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id pm10si6365033pac.92.2015.09.22.17.41.31 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Sep 2015 17:41:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZeY65-00066k-P5; Wed, 23 Sep 2015 00:39:45 +0000 Received: from mail-pa0-f51.google.com ([209.85.220.51]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZeY5K-0005QW-Q3 for linux-arm-kernel@lists.infradead.org; Wed, 23 Sep 2015 00:39:01 +0000 Received: by padhy16 with SMTP id hy16so23859659pad.1 for ; Tue, 22 Sep 2015 17:38:37 -0700 (PDT) X-Received: by 10.66.155.9 with SMTP id vs9mr33839341pab.63.1442968717180; Tue, 22 Sep 2015 17:38:37 -0700 (PDT) Received: from localhost.localdomain ([70.35.39.2]) by smtp.gmail.com with ESMTPSA id ja4sm1927162pbb.19.2015.09.22.17.38.36 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 22 Sep 2015 17:38:36 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, will.deacon@arm.com, catalin.marinas@arm.com Subject: [PATCH v2 3/7] arm64: split off early mapping code from early_fixmap_init() Date: Tue, 22 Sep 2015 17:37:39 -0700 Message-Id: <1442968663-31843-4-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1442968663-31843-1-git-send-email-ard.biesheuvel@linaro.org> References: <1442968663-31843-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150922_173858_998859_A6A402ED X-CRM114-Status: GOOD ( 15.92 ) X-Spam-Score: -2.6 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.6 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [209.85.220.51 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.220.51 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Cc: Ard Biesheuvel MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.42 as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 This splits off and generalises the population of the statically allocated fixmap page tables so that we may reuse it later for the linear mapping once we move the kernel text mapping out of it. This also involves taking into account that table entries at any of the levels we are populating may have been populated already, since the fixmap mapping might not be disjoint up to the pgd level anymore from other early mappings. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/compiler.h | 2 + arch/arm64/kernel/vmlinux.lds.S | 12 ++-- arch/arm64/mm/mmu.c | 60 ++++++++++++++------ 3 files changed, 51 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/compiler.h b/arch/arm64/include/asm/compiler.h index ee35fd0f2236..dd342af63673 100644 --- a/arch/arm64/include/asm/compiler.h +++ b/arch/arm64/include/asm/compiler.h @@ -27,4 +27,6 @@ */ #define __asmeq(x, y) ".ifnc " x "," y " ; .err ; .endif\n\t" +#define __pgdir __attribute__((section(".pgdir"),aligned(PAGE_SIZE))) + #endif /* __ASM_COMPILER_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 98073332e2d0..ceec4def354b 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -160,11 +160,13 @@ SECTIONS BSS_SECTION(0, 0, 0) - . = ALIGN(PAGE_SIZE); - idmap_pg_dir = .; - . += IDMAP_DIR_SIZE; - swapper_pg_dir = .; - . += SWAPPER_DIR_SIZE; + .pgdir (NOLOAD) : ALIGN(PAGE_SIZE) { + idmap_pg_dir = .; + . += IDMAP_DIR_SIZE; + swapper_pg_dir = .; + . += SWAPPER_DIR_SIZE; + *(.pgdir) + } _end = .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 9211b8527f25..5af804334697 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -342,6 +342,44 @@ static void __init __map_memblock(phys_addr_t start, phys_addr_t end) } #endif +struct bootstrap_pgtables { + pte_t pte[PTRS_PER_PTE]; + pmd_t pmd[PTRS_PER_PMD > 1 ? PTRS_PER_PMD : 0]; + pud_t pud[PTRS_PER_PUD > 1 ? PTRS_PER_PUD : 0]; +}; + +static void __init bootstrap_early_mapping(unsigned long addr, + struct bootstrap_pgtables *reg, + bool pte_level) +{ + pgd_t *pgd; + pud_t *pud; + pmd_t *pmd; + + pgd = pgd_offset_k(addr); + if (pgd_none(*pgd)) { + clear_page(reg->pud); + memblock_reserve(__pa(reg->pud), PAGE_SIZE); + pgd_populate(&init_mm, pgd, reg->pud); + } + pud = pud_offset(pgd, addr); + if (pud_none(*pud)) { + clear_page(reg->pmd); + memblock_reserve(__pa(reg->pmd), PAGE_SIZE); + pud_populate(&init_mm, pud, reg->pmd); + } + + if (!pte_level) + return; + + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) { + clear_page(reg->pte); + memblock_reserve(__pa(reg->pte), PAGE_SIZE); + pmd_populate_kernel(&init_mm, pmd, reg->pte); + } +} + static void __init map_mem(void) { struct memblock_region *reg; @@ -544,14 +582,6 @@ void vmemmap_free(unsigned long start, unsigned long end) } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ -static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss; -#if CONFIG_PGTABLE_LEVELS > 2 -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss; -#endif -#if CONFIG_PGTABLE_LEVELS > 3 -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss; -#endif - static inline pud_t * fixmap_pud(unsigned long addr) { pgd_t *pgd = pgd_offset_k(addr); @@ -581,21 +611,15 @@ static inline pte_t * fixmap_pte(unsigned long addr) void __init early_fixmap_init(void) { - pgd_t *pgd; - pud_t *pud; + static struct bootstrap_pgtables fixmap_bs_pgtables __pgdir; pmd_t *pmd; - unsigned long addr = FIXADDR_START; - pgd = pgd_offset_k(addr); - pgd_populate(&init_mm, pgd, bm_pud); - pud = pud_offset(pgd, addr); - pud_populate(&init_mm, pud, bm_pmd); - pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, bm_pte); + bootstrap_early_mapping(FIXADDR_START, &fixmap_bs_pgtables, true); + pmd = fixmap_pmd(FIXADDR_START); /* * The boot-ioremap range spans multiple pmds, for which - * we are not preparted: + * we are not prepared: */ BUILD_BUG_ON((__fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT) != (__fix_to_virt(FIX_BTMAP_END) >> PMD_SHIFT));