From patchwork Fri Oct 21 11:22:58 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 78645 Delivered-To: patch@linaro.org Received: by 10.140.97.247 with SMTP id m110csp1245905qge; Fri, 21 Oct 2016 04:26:11 -0700 (PDT) X-Received: by 10.98.137.206 with SMTP id n75mr741969pfk.132.1477049171542; Fri, 21 Oct 2016 04:26:11 -0700 (PDT) Return-Path: Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id de8si1942153pad.84.2016.10.21.04.26.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Oct 2016 04:26:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) smtp.mailfrom=linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bxXwn-00006K-F9; Fri, 21 Oct 2016 11:25:13 +0000 Received: from mail-lf0-x236.google.com ([2a00:1450:4010:c07::236]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bxXvl-0007TJ-LL for linux-arm-kernel@lists.infradead.org; Fri, 21 Oct 2016 11:24:11 +0000 Received: by mail-lf0-x236.google.com with SMTP id b75so137912836lfg.3 for ; Fri, 21 Oct 2016 04:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=y4WNQL3lFm/s0YXCJ8BhxtN6TL6NDyT0yOS92QTQVAw=; b=OP8h+IV71kcAL5JtmvyW09pRuYMkrr4kSMXS52h0NqaN1KFUcOVMi0tjg7vQg7yToh O7h1C0/CJEvKpqtFzo6CfKphbYIZs367KXtJ4B5TLCkFeibrkLH53Yk/X/hok/W57+vO uLePqNBNGQFgLd5t0QFV6qa5sjjtKjSgpePmQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=y4WNQL3lFm/s0YXCJ8BhxtN6TL6NDyT0yOS92QTQVAw=; b=g11zxh4qo5oY3DBmACwkyvsYx0om+8JEPbFo0vbasyPFWDpjVvQwySgXLTPmCRtDas rgQkr9SlcdWzdePbi8QeKH8nK3fxQm23D/JhI9rOM6ziHnEcTaUStSMtwz3mjePo/BV4 pf6VwfQI5iXgiaktzYEyGsojWToYee5smZHzq0gcyAiaCUEojd8E2I1pqoCsVt1WUGIJ TgGoOdgEVjdUVBobGewzNg4ScG5Mk/PjJntGKy2l84HQOcXPHCejqPAfjyZRa3Ik42/B OkW76vxm9vMSZyT91eL6XMuK9TuKG7AwkcdonWmuRX22RZ70DfWTHuLLNLKRcsb7RW12 k+bQ== X-Gm-Message-State: AA6/9Rkf5a9QO0SAaFhSkgAD39v1RzUvPETPYNBwbhvRHnlzZM9u3v8oXTnGygflaUg33YrU X-Received: by 10.28.203.202 with SMTP id b193mr10071176wmg.0.1477049027098; Fri, 21 Oct 2016 04:23:47 -0700 (PDT) Received: from localhost.localdomain ([196.66.89.52]) by smtp.gmail.com with ESMTPSA id 130sm3678711wmf.0.2016.10.21.04.23.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 21 Oct 2016 04:23:46 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com Subject: [PATCH v4 3/3] arm64: mm: set the contiguous bit for kernel mappings where appropriate Date: Fri, 21 Oct 2016 12:22:58 +0100 Message-Id: <1477048978-4140-4-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1477048978-4140-1-git-send-email-ard.biesheuvel@linaro.org> References: <1477048978-4140-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20161021_042410_031530_D5C9987C X-CRM114-Status: GOOD ( 15.73 ) X-Spam-Score: -2.7 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [2a00:1450:4010:c07:0:0:0:236 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, steve.capper@linaro.org, will.deacon@arm.com, jeremy.linton@arm.com, Ard Biesheuvel MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org Now that we no longer allow live kernel PMDs to be split, it is safe to start using the contiguous bit for kernel mappings. So set the contiguous bit in the kernel page mappings for regions whose size and alignment are suitable for this. This enables the following contiguous range sizes for the virtual mapping of the kernel image, and for the linear mapping: granule size | cont PTE | cont PMD | -------------+------------+------------+ 4 KB | 64 KB | 32 MB | 16 KB | 2 MB | 1 GB* | 64 KB | 2 MB | 16 GB* | * Only when built for 3 or more levels of translation. This is due to the fact that a 2 level configuration only consists of PGDs and PTEs, and the added complexity of dealing with folded PMDs is not justified considering that 16 GB contiguous ranges are likely to be ignored by the hardware (and 16k/2 levels is a niche configuration) Signed-off-by: Ard Biesheuvel --- arch/arm64/mm/mmu.c | 34 +++++++++++++++++--- 1 file changed, 30 insertions(+), 4 deletions(-) -- 2.7.4 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 7b0dd07212ae..dd5f12d0959e 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -107,8 +107,10 @@ static bool pgattr_change_is_safe(u64 old, u64 new) static void alloc_init_pte(pmd_t *pmd, unsigned long addr, unsigned long end, unsigned long pfn, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(void)) + phys_addr_t (*pgtable_alloc)(void), + bool page_mappings_only) { + pgprot_t __prot = prot; pte_t *pte; BUG_ON(pmd_sect(*pmd)); @@ -126,7 +128,18 @@ static void alloc_init_pte(pmd_t *pmd, unsigned long addr, do { pte_t old_pte = *pte; - set_pte(pte, pfn_pte(pfn, prot)); + /* + * Set the contiguous bit for the subsequent group of PTEs if + * its size and alignment are appropriate. + */ + if (((addr | PFN_PHYS(pfn)) & ~CONT_PTE_MASK) == 0) { + if (end - addr >= CONT_PTE_SIZE && !page_mappings_only) + __prot = __pgprot(pgprot_val(prot) | PTE_CONT); + else + __prot = prot; + } + + set_pte(pte, pfn_pte(pfn, __prot)); pfn++; /* @@ -145,6 +158,7 @@ static void alloc_init_pmd(pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t (*pgtable_alloc)(void), bool page_mappings_only) { + pgprot_t __prot = prot; pmd_t *pmd; unsigned long next; @@ -171,7 +185,18 @@ static void alloc_init_pmd(pud_t *pud, unsigned long addr, unsigned long end, /* try section mapping first */ if (((addr | next | phys) & ~SECTION_MASK) == 0 && !page_mappings_only) { - pmd_set_huge(pmd, phys, prot); + /* + * Set the contiguous bit for the subsequent group of + * PMDs if its size and alignment are appropriate. + */ + if (((addr | phys) & ~CONT_PMD_MASK) == 0) { + if (end - addr >= CONT_PMD_SIZE) + __prot = __pgprot(pgprot_val(prot) | + PTE_CONT); + else + __prot = prot; + } + pmd_set_huge(pmd, phys, __prot); /* * After the PMD entry has been populated once, we @@ -181,7 +206,8 @@ static void alloc_init_pmd(pud_t *pud, unsigned long addr, unsigned long end, pmd_val(*pmd))); } else { alloc_init_pte(pmd, addr, next, __phys_to_pfn(phys), - prot, pgtable_alloc); + prot, pgtable_alloc, + page_mappings_only); BUG_ON(pmd_val(old_pmd) != 0 && pmd_val(old_pmd) != pmd_val(*pmd));