From patchwork Mon Feb 29 14:44:40 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 63219 Delivered-To: patch@linaro.org Received: by 10.112.199.169 with SMTP id jl9csp1292144lbc; Mon, 29 Feb 2016 06:48:35 -0800 (PST) X-Received: by 10.98.13.216 with SMTP id 85mr22327784pfn.143.1456757315666; Mon, 29 Feb 2016 06:48:35 -0800 (PST) Return-Path: Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id bp6si4403524pac.135.2016.02.29.06.48.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Feb 2016 06:48:35 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) smtp.mailfrom=linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org; dkim=neutral (body hash did not verify) header.i=@linaro.org Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aaP6h-0004ax-7v; Mon, 29 Feb 2016 14:47:31 +0000 Received: from mail-wm0-x22d.google.com ([2a00:1450:400c:c09::22d]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1aaP5B-0003QE-9b for linux-arm-kernel@lists.infradead.org; Mon, 29 Feb 2016 14:45:59 +0000 Received: by mail-wm0-x22d.google.com with SMTP id l68so40030320wml.0 for ; Mon, 29 Feb 2016 06:45:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VCFSwUmpTBjHcp8U7JO0B0c0xxU/zWks6Rzq3TS8Yx0=; b=b9xFoxj/JTmL5oq4PGbKqpEIy1fB3uvttVC/+BsGyR/6lpdJaMEqG3PvIJ2W3lUCW5 YmlWTvvrd1JEhUbiJAuhFtVttrWUqeGRNhnPGN0ZRoO1KDGwTvEvsdaoBUEN1msuIBdE mduoDqMOmfV+mTdBlMFzTyFQiFs0UMbS1CEv0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VCFSwUmpTBjHcp8U7JO0B0c0xxU/zWks6Rzq3TS8Yx0=; b=Y81SibfQKU39tjZYbrOeOLrT5HO7ekXBjrXBAdm5NyrjT6YO6YjvS2B+cGzM+Qoniy hZs+L/edCXbXyb/3HtMU625Aj956H5xnTIyqABPAxg165/KtwO+5E/LX5v9wqLELoDU2 2pP66t/J8LJI2+yMCFllE0yAowYastYu0JZA3fCBS5sMMdp15z2S/ZZ+fwpfYA5ra93C DiObPopA81BuSSFDDpXJ3bSD01a5p655CAQ2w/Eqlx9d/TR0Ut6e79hUJHcXS8LiLlHz 8kpC7tXpvkHeEMoZGvZYbe2HT8bA600hrH7I74K3A7cDSpRhED/sUeDAdHjNU66UJYm+ AM9w== X-Gm-Message-State: AD7BkJJg7N5epDp1E4v2e6/cSK4dgtzCBltg1+Z9d/YGlG75B+G/eBZ3/O9+J0E3yCuEpDkl X-Received: by 10.28.101.7 with SMTP id z7mr11387071wmb.92.1456757135689; Mon, 29 Feb 2016 06:45:35 -0800 (PST) Received: from localhost.localdomain ([195.55.142.58]) by smtp.gmail.com with ESMTPSA id di1sm26196693wjc.3.2016.02.29.06.45.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 29 Feb 2016 06:45:34 -0800 (PST) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will.deacon@arm.com, mark.rutland@arm.com Subject: [PATCH v2 5/9] arm64: mm: move vmemmap region right below the linear region Date: Mon, 29 Feb 2016 15:44:40 +0100 Message-Id: <1456757084-1078-6-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1456757084-1078-1-git-send-email-ard.biesheuvel@linaro.org> References: <1456757084-1078-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160229_064557_820670_18DF8B81 X-CRM114-Status: GOOD ( 16.58 ) X-Spam-Score: -2.7 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [2a00:1450:400c:c09:0:0:0:22d listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jonas@southpole.se, Ard Biesheuvel , linux-mm@kvack.org, nios2-dev@lists.rocketboards.org, linux@lists.openrisc.net, lftan@altera.com, akpm@linux-foundation.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org This moves the vmemmap region right below PAGE_OFFSET, aka the start of the linear region, and redefines its size to be a power of two. Due to the placement of PAGE_OFFSET in the middle of the address space, whose size is a power of two as well, this guarantees that virt to page conversions and vice versa can be implemented efficiently, by masking and shifting rather than ordinary arithmetic. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/memory.h | 18 +++++++++++++++++- arch/arm64/include/asm/pgtable.h | 11 +++-------- arch/arm64/mm/dump.c | 16 ++++++++-------- arch/arm64/mm/init.c | 16 +++++++++++----- 4 files changed, 39 insertions(+), 22 deletions(-) -- 2.5.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 12f8a00fb3f1..8a2ab195ca77 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -40,6 +40,21 @@ #define PCI_IO_SIZE SZ_16M /* + * Log2 of the upper bound of the size of a struct page. Used for sizing + * the vmemmap region only, does not affect actual memory footprint. + * We don't use sizeof(struct page) directly since taking its size here + * requires its definition to be available at this point in the inclusion + * chain, and it may not be a power of 2 in the first place. + */ +#define STRUCT_PAGE_MAX_SHIFT 6 + +/* + * VMEMMAP_SIZE - allows the whole linear region to be covered by + * a struct page array + */ +#define VMEMMAP_SIZE (UL(1) << (VA_BITS - PAGE_SHIFT - 1 + STRUCT_PAGE_MAX_SHIFT)) + +/* * PAGE_OFFSET - the virtual address of the start of the kernel image (top * (VA_BITS - 1)) * VA_BITS - the maximum number of bits for virtual addresses. @@ -54,7 +69,8 @@ #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define MODULES_VADDR (VA_START + KASAN_SHADOW_SIZE) #define MODULES_VSIZE (SZ_128M) -#define PCI_IO_END (PAGE_OFFSET - SZ_2M) +#define VMEMMAP_START (PAGE_OFFSET - VMEMMAP_SIZE) +#define PCI_IO_END (VMEMMAP_START - SZ_2M) #define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE) #define FIXADDR_TOP (PCI_IO_START - SZ_2M) #define TASK_SIZE_64 (UL(1) << VA_BITS) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index af80e437d075..fdea729f1fbf 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -24,20 +24,15 @@ #include /* - * VMALLOC and SPARSEMEM_VMEMMAP ranges. + * VMALLOC range. * - * VMEMAP_SIZE: allows the whole linear region to be covered by a struct page array - * (rounded up to PUD_SIZE). * VMALLOC_START: beginning of the kernel vmalloc space - * VMALLOC_END: extends to the available space below vmmemmap, PCI I/O space, - * fixed mappings and modules + * VMALLOC_END: extends to the available space below vmmemmap, PCI I/O space + * and fixed mappings */ -#define VMEMMAP_SIZE ALIGN((1UL << (VA_BITS - PAGE_SHIFT - 1)) * sizeof(struct page), PUD_SIZE) - #define VMALLOC_START (MODULES_END) #define VMALLOC_END (PAGE_OFFSET - PUD_SIZE - VMEMMAP_SIZE - SZ_64K) -#define VMEMMAP_START (VMALLOC_END + SZ_64K) #define vmemmap ((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT)) #define FIRST_USER_ADDRESS 0UL diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c index f9271cb2f5e3..a21f47421b0c 100644 --- a/arch/arm64/mm/dump.c +++ b/arch/arm64/mm/dump.c @@ -37,14 +37,14 @@ enum address_markers_idx { MODULES_END_NR, VMALLOC_START_NR, VMALLOC_END_NR, -#ifdef CONFIG_SPARSEMEM_VMEMMAP - VMEMMAP_START_NR, - VMEMMAP_END_NR, -#endif FIXADDR_START_NR, FIXADDR_END_NR, PCI_START_NR, PCI_END_NR, +#ifdef CONFIG_SPARSEMEM_VMEMMAP + VMEMMAP_START_NR, + VMEMMAP_END_NR, +#endif KERNEL_SPACE_NR, }; @@ -53,14 +53,14 @@ static struct addr_marker address_markers[] = { { MODULES_END, "Modules end" }, { VMALLOC_START, "vmalloc() Area" }, { VMALLOC_END, "vmalloc() End" }, -#ifdef CONFIG_SPARSEMEM_VMEMMAP - { 0, "vmemmap start" }, - { 0, "vmemmap end" }, -#endif { FIXADDR_START, "Fixmap start" }, { FIXADDR_TOP, "Fixmap end" }, { PCI_IO_START, "PCI I/O start" }, { PCI_IO_END, "PCI I/O end" }, +#ifdef CONFIG_SPARSEMEM_VMEMMAP + { 0, "vmemmap start" }, + { 0, "vmemmap end" }, +#endif { PAGE_OFFSET, "Linear Mapping" }, { -1, NULL }, }; diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index cebad6cceda8..9cfe94b41b54 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -365,13 +365,13 @@ void __init mem_init(void) " .text : 0x%p" " - 0x%p" " (%6ld KB)\n" " .rodata : 0x%p" " - 0x%p" " (%6ld KB)\n" " .data : 0x%p" " - 0x%p" " (%6ld KB)\n" + " fixed : 0x%16lx - 0x%16lx (%6ld KB)\n" + " PCI I/O : 0x%16lx - 0x%16lx (%6ld MB)\n" #ifdef CONFIG_SPARSEMEM_VMEMMAP " vmemmap : 0x%16lx - 0x%16lx (%6ld GB maximum)\n" " 0x%16lx - 0x%16lx (%6ld MB actual)\n" #endif - " fixed : 0x%16lx - 0x%16lx (%6ld KB)\n" - " PCI I/O : 0x%16lx - 0x%16lx (%6ld MB)\n" - " memory : 0x%16lx - 0x%16lx (%6ld MB)\n", + " memory : 0x%16lx - 0x%16lx (%6ld MB)\n", #ifdef CONFIG_KASAN MLG(KASAN_SHADOW_START, KASAN_SHADOW_END), #endif @@ -381,14 +381,14 @@ void __init mem_init(void) MLK_ROUNDUP(_text, __start_rodata), MLK_ROUNDUP(__start_rodata, _etext), MLK_ROUNDUP(_sdata, _edata), + MLK(FIXADDR_START, FIXADDR_TOP), + MLM(PCI_IO_START, PCI_IO_END), #ifdef CONFIG_SPARSEMEM_VMEMMAP MLG(VMEMMAP_START, VMEMMAP_START + VMEMMAP_SIZE), MLM((unsigned long)phys_to_page(memblock_start_of_DRAM()), (unsigned long)virt_to_page(high_memory)), #endif - MLK(FIXADDR_START, FIXADDR_TOP), - MLM(PCI_IO_START, PCI_IO_END), MLM(__phys_to_virt(memblock_start_of_DRAM()), (unsigned long)high_memory)); @@ -404,6 +404,12 @@ void __init mem_init(void) BUILD_BUG_ON(TASK_SIZE_32 > TASK_SIZE_64); #endif + /* + * Make sure we chose the upper bound of sizeof(struct page) + * correctly. + */ + BUILD_BUG_ON(sizeof(struct page) > (1 << STRUCT_PAGE_MAX_SHIFT)); + if (PAGE_SIZE >= 16384 && get_num_physpages() <= 128) { extern int sysctl_overcommit_memory; /*