From patchwork Thu Dec 6 18:21:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 153056 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10859959ljp; Thu, 6 Dec 2018 10:21:27 -0800 (PST) X-Google-Smtp-Source: AFSGD/WlkQ6DoO9j2nd+8W6N57GIOGWa+8a2hwRZUsr0R2oW6TpMJaFzSdbR68yy9DdEzg1O0diD X-Received: by 2002:aa7:8608:: with SMTP id p8mr29994067pfn.125.1544120486995; Thu, 06 Dec 2018 10:21:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544120486; cv=none; d=google.com; s=arc-20160816; b=FVsn6wgqzZoLVPmb61+f+mlf9bLMBsPz+CukpNBebUjY4XhjIg6dDrCizGYishk+qr HJH5TnteBhWLDrKUvwSTrmkLfSuvgvxd9FdbpH37ZEa+598srvPJT3O2iM02VFfrGCl8 dVl5xZvLP5clXz7T35fNeiXmDxTILKov3FbXuEFrqxKbihLDnXu7T1ETDSjkGbPzy4D6 Kuqq5YXnRi0Qs8om1NEZV4Bspp9Y8mQsY+AyGEr6M1tGJxslAaBTXNpBg2/S3NNLaIXz lQnWfVXOKsVeAu7UCX2+vqwhbd8kIAJRKho0aK78/PJo8FWUNz8bAaguZjZmZn+NG/Jr TIvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=Ka6oXZu8DJRXSCZkb2TermywvttDv1cohIKM9nr4o2s=; b=jfl+KmYBayd2xmwIbAkmQT5/83bnlyUodAuhF071HvpzX/L5HkpVg0OUZI5/w8B8tl h3UQbr3738SaFPzM73dbLySuKbrlH4Z/D2QQ5hpQyg7B40uhBmsAEN58Qa3xc/odxhcV QJG1c28B6i1lWizC11swFXkVoxeCCLk8KJu4pHxR+bxVoDGYDSrJ6EmvYaP5/Em9V/5c 0X+dLDeqJYaHlVlpBzBesbcdXm6BEfZZxcHH964MsWLKYzrws59GI3Zw/cDOCO2zvz2G +3t6Cp05GIQA0W9PxEoJ/2QIGammTjdHXP3odKS8SpRiViPyO7jFoHQZoqy29pgB93Yi CTMA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t13si756655pgm.175.2018.12.06.10.21.26; Thu, 06 Dec 2018 10:21:26 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726143AbeLFSVU (ORCPT + 31 others); Thu, 6 Dec 2018 13:21:20 -0500 Received: from foss.arm.com ([217.140.101.70]:57826 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725901AbeLFSVQ (ORCPT ); Thu, 6 Dec 2018 13:21:16 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8D060EBD; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5A4E33F71D; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 755961AE0BA8; Thu, 6 Dec 2018 18:21:36 +0000 (GMT) From: Will Deacon To: akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, sean.j.christopherson@intel.com, Will Deacon Subject: [RESEND PATCH v4 1/5] ioremap: Rework pXd_free_pYd_page() API Date: Thu, 6 Dec 2018 18:21:31 +0000 Message-Id: <1544120495-17438-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1544120495-17438-1-git-send-email-will.deacon@arm.com> References: <1544120495-17438-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The recently merged API for ensuring break-before-make on page-table entries when installing huge mappings in the vmalloc/ioremap region is fairly counter-intuitive, resulting in the arch freeing functions (e.g. pmd_free_pte_page()) being called even on entries that aren't present. This resulted in a minor bug in the arm64 implementation, giving rise to spurious VM_WARN messages. This patch moves the pXd_present() checks out into the core code, refactoring the callsites at the same time so that we avoid the complex conjunctions when determining whether or not we can put down a huge mapping. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Suggested-by: Linus Torvalds Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- lib/ioremap.c | 56 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 14 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 517f5853ffed..6c72764af19c 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -76,6 +76,25 @@ static int ioremap_pte_range(pmd_t *pmd, unsigned long addr, return 0; } +static int ioremap_try_huge_pmd(pmd_t *pmd, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pmd_enabled()) + return 0; + + if ((end - addr) != PMD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PMD_SIZE)) + return 0; + + if (pmd_present(*pmd) && !pmd_free_pte_page(pmd, addr)) + return 0; + + return pmd_set_huge(pmd, phys_addr, prot); +} + static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -89,13 +108,8 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, do { next = pmd_addr_end(addr, end); - if (ioremap_pmd_enabled() && - ((next - addr) == PMD_SIZE) && - IS_ALIGNED(phys_addr + addr, PMD_SIZE) && - pmd_free_pte_page(pmd, addr)) { - if (pmd_set_huge(pmd, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) return -ENOMEM; @@ -103,6 +117,25 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, return 0; } +static int ioremap_try_huge_pud(pud_t *pud, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pud_enabled()) + return 0; + + if ((end - addr) != PUD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PUD_SIZE)) + return 0; + + if (pud_present(*pud) && !pud_free_pmd_page(pud, addr)) + return 0; + + return pud_set_huge(pud, phys_addr, prot); +} + static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -116,13 +149,8 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, do { next = pud_addr_end(addr, end); - if (ioremap_pud_enabled() && - ((next - addr) == PUD_SIZE) && - IS_ALIGNED(phys_addr + addr, PUD_SIZE) && - pud_free_pmd_page(pud, addr)) { - if (pud_set_huge(pud, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) return -ENOMEM; From patchwork Thu Dec 6 18:21:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 153053 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10859787ljp; Thu, 6 Dec 2018 10:21:18 -0800 (PST) X-Google-Smtp-Source: AFSGD/VJ+ANYA1O9AoaMOWlqkmv39NPjQcZnczhk7UD4zBa5LAaC4GWpwQLJaQFNw708/kA8uGVH X-Received: by 2002:a63:3703:: with SMTP id e3mr24512473pga.348.1544120478326; Thu, 06 Dec 2018 10:21:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544120478; cv=none; d=google.com; s=arc-20160816; b=FyuL06Jnph1s1fI2yHTR8OyjiwrJgQg5VuX9QUzC1wKpPd5hGZ7P8Ve++MkHHBmCmW D6KVkiDW2pTjT49di8W4kfNOAKbB/iAvD/Jn49/Slq/JaERhr5dFiXctJ1cnsgUt3wcq OVi3lUnHH0QmOlnQjsJsu0hxmLJtZPugYKNHlBAs5e3roEtm3HU48iPtNo1dPnBeOAyW 1zxpezuYiDxKcRVZSW8xheeD//i2CHOxWwJeUy6qUaC0Vm09CcpOxIbQZ0C5SaR31tYT lmng2if+DZa/F8rAoRDKACQQlRQ/TUzApyyTOQnM3GtUsfCywKbOt5Bz7/zA4VdJSzo2 VVWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=/V6Ii5Ay/gyCu4FDeRHUvTmr/uoACgypNN/miYDcTTU=; b=yNVNciNZ08haqLeEQ6HSbZdBuNEzmHmCm3dHf14nlcNusx5QNzfWDL0O0U3fFVQoZe SNu7v7noLsiC8zZJWeMcG0I4U5PItAocJ7BS8mFyyLl33HrmzVHZJ0P6T7tqpjK8/1WT jCbsp3kcVSMnJ6yrtnMEv7sENioZSPQtYXmBWCSeVjElIs6FjDX4D8kD3VPvIosos2cO ZL9scxtphwV2K9ggKrGIxTqwC4imOjqA0vufFz9yStxeRsePFEktKGOCzos6dj4Jj2NM ZJBok7nE5i5k+jC1w9/SuoMomgsQX5bhwbQZxSI0wUK4v71z281jqkk5chOzZH7m5Mm8 ie8w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n8si799809plk.9.2018.12.06.10.21.18; Thu, 06 Dec 2018 10:21:18 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726101AbeLFSVQ (ORCPT + 31 others); Thu, 6 Dec 2018 13:21:16 -0500 Received: from foss.arm.com ([217.140.101.70]:57838 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725916AbeLFSVQ (ORCPT ); Thu, 6 Dec 2018 13:21:16 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9FFF81596; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7098C3F908; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 841DA1AE0BE1; Thu, 6 Dec 2018 18:21:36 +0000 (GMT) From: Will Deacon To: akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, sean.j.christopherson@intel.com, Will Deacon Subject: [RESEND PATCH v4 2/5] arm64: mmu: Drop pXd_present() checks from pXd_free_pYd_table() Date: Thu, 6 Dec 2018 18:21:32 +0000 Message-Id: <1544120495-17438-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1544120495-17438-1-git-send-email-will.deacon@arm.com> References: <1544120495-17438-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The core code already has a check for pXd_none(), so remove it from the architecture implementation. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d1d6601b385d..786cfa6355be 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -989,10 +989,8 @@ int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) pmd = READ_ONCE(*pmdp); - if (!pmd_present(pmd)) - return 1; if (!pmd_table(pmd)) { - VM_WARN_ON(!pmd_table(pmd)); + VM_WARN_ON(1); return 1; } @@ -1012,10 +1010,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pud = READ_ONCE(*pudp); - if (!pud_present(pud)) - return 1; if (!pud_table(pud)) { - VM_WARN_ON(!pud_table(pud)); + VM_WARN_ON(1); return 1; } From patchwork Thu Dec 6 18:21:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 153057 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10860001ljp; Thu, 6 Dec 2018 10:21:30 -0800 (PST) X-Google-Smtp-Source: AFSGD/VCUqO3zweS8u6XiB4GwqZUHmBJyMUvEg+WEPw3OF4lLL4JiAOxx9+ndWzzkgKKlViYWL+N X-Received: by 2002:a17:902:3281:: with SMTP id z1mr29858763plb.296.1544120490485; Thu, 06 Dec 2018 10:21:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544120490; cv=none; d=google.com; s=arc-20160816; b=LiCK2jfAhESrmlxWP1gPicXEg/BvPm2riJvr+XerJ5nLEunp3yiXfaKL0C/q7WRCtc FKV0gyLLdSJKiVAPA5jw815wp9D1Xk/jlC1zuONX0i2wJSgxP3AQYlDy0O2mQMhl0VYr QVJFKvoJp40Xdf/4o8+xHJpa6yaEONUQuf1uCrp/q/f5uPX2ToSg/LVk1cSH/mB7f/9/ Q+R27KG8Jk0LTPVSIOZ9luzgwye7+0vYylPzecYUoYjTy7pL+pKnQ+Rz8I5jdulMv5z5 rVZaQ4nRRBYh3SCey7HseMuL/eU5KtIV+PDVKMU9IQpFMGeAVaSkN3mZ2imV+qRLCZ1G tp7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=S3Pe7jCrFGE7JxF6m5RucqgSGzB7M8lCaWDcweULJ1Y=; b=fGnPBAEl7m2MI+rf80d7O9Y1xFoq46MhRkmBE7g0S53dN3w7VGaREulkRh8/70n+NZ RFivKLel3MbFRHUVXGM43iGshgFp+DCdshiZICbLCpxT5MEw86cHOidMPHOURQ4206+i /401ilDvScOYCdJN0BKhMCg1nXItPJOAEvQrfddpIsXShmNPse17NOoJQ8WCqthm7M1j 2z2VhzP9x80P1qbXBDMx782m9IALBhXCGdU86HsZeivLVolFTHEWatjD5Jv8383GMxjb TKj0sRVQ+KSq0Nd0f2MVt2R0D+9PVAxczgibELjbNy+vPv5htTLSwpE/4ld8M6tFUeag GnsA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h13si700592pgs.17.2018.12.06.10.21.30; Thu, 06 Dec 2018 10:21:30 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726129AbeLFSVU (ORCPT + 31 others); Thu, 6 Dec 2018 13:21:20 -0500 Received: from foss.arm.com ([217.140.101.70]:57854 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725942AbeLFSVQ (ORCPT ); Thu, 6 Dec 2018 13:21:16 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B3DDA15AB; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8483C3F5AF; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 9A6A91AE0BE3; Thu, 6 Dec 2018 18:21:36 +0000 (GMT) From: Will Deacon To: akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, sean.j.christopherson@intel.com, Will Deacon Subject: [RESEND PATCH v4 3/5] x86/pgtable: Drop pXd_none() checks from pXd_free_pYd_table() Date: Thu, 6 Dec 2018 18:21:33 +0000 Message-Id: <1544120495-17438-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1544120495-17438-1-git-send-email-will.deacon@arm.com> References: <1544120495-17438-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The core code already has a check for pXd_none(), so remove it from the architecture implementation. Cc: Chintan Pandya Cc: Toshi Kani Cc: Michal Hocko Cc: Andrew Morton Acked-by: Thomas Gleixner Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/x86/mm/pgtable.c | 6 ------ 1 file changed, 6 deletions(-) -- 2.1.4 diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 59274e2c1ac4..e95a7d6ac8f8 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -811,9 +811,6 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) pte_t *pte; int i; - if (pud_none(*pud)) - return 1; - pmd = (pmd_t *)pud_page_vaddr(*pud); pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); if (!pmd_sv) @@ -855,9 +852,6 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { pte_t *pte; - if (pmd_none(*pmd)) - return 1; - pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); From patchwork Thu Dec 6 18:21:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 153054 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10859796ljp; Thu, 6 Dec 2018 10:21:18 -0800 (PST) X-Google-Smtp-Source: AFSGD/XFJlhFqd9h7W1qHjBh0ohhbtBT1orTHM5+trNKhMvTpgsrd5wJK9CJNUZUOGU3eMdy6JbT X-Received: by 2002:a62:399b:: with SMTP id u27mr30618056pfj.181.1544120478710; Thu, 06 Dec 2018 10:21:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544120478; cv=none; d=google.com; s=arc-20160816; b=B8YYcpMG64uNLfT7AEcBtucpWM0J8Qm9tmTQegc8LmYVUoEGxyvUIWd/JPbk1tgsBQ m/0aeKxxelOhCHnAE8yU8FoPZzXIskJsqp7wdfjS6pW/ZLvApSIk0/NVVB2Ec4XOxdIg yr+xHz74ACc+9EYAtq3/iEs5W3enMcdIee0laU6IxQAQ2V/pckXD4K1Hbhx6h9i8G7GO 4mMFXNT7iVO8zpal792GZDEcXvK4/Jr0kEQkJ94yTe4BlbImuddZhlP3Lafbg+soiwCE T8n8J3Iv/G3CGNvHfkyDRFRAogZbJ8xbGhNY7y7qzEkv1j4ISA6NAJsDeEeD3vMkHCDr 0QVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=1Ou78OtI/MFN3FEOfIqRBplaVjPGG1HlQbCtT73huwU=; b=QIXbBEsQqrbNP0o5aeJQmYe62gfSQVUCaKw8I62U0yCdXpjFf3oEqFd/dzSOhWcEBb HraBYE9YGj1oYvA2ruD6XvloBVQX35porpBznocgNt0dhyr0PWpsAA9Ix0N8CPPP5sZT V0Qf/hRYKslcfHKJLHl8c00zoeV7bY7RXrsl7vrdNsLWMp58PY+BE4nYG0nONDhvDDwn hCUMklOXxgmzvgNyVCVGW7e+9ephvcTjYxZFube51VKkKMdoAk4UGpXmvCOkvcMkq+e6 DkLRyZubLhAiAdEH5WnoPuuaqsRJHEYHEWaDkJ0/uc8lOQN8gPbr9jtfm2DJP+GZnx1z hS1A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n8si799809plk.9.2018.12.06.10.21.18; Thu, 06 Dec 2018 10:21:18 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726115AbeLFSVR (ORCPT + 31 others); Thu, 6 Dec 2018 13:21:17 -0500 Received: from foss.arm.com ([217.140.101.70]:57872 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725945AbeLFSVQ (ORCPT ); Thu, 6 Dec 2018 13:21:16 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C825815AD; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 98AF23F71D; Thu, 6 Dec 2018 10:21:15 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id AEC071AE0BF1; Thu, 6 Dec 2018 18:21:36 +0000 (GMT) From: Will Deacon To: akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, sean.j.christopherson@intel.com, Will Deacon Subject: [RESEND PATCH v4 4/5] lib/ioremap: Ensure phys_addr actually corresponds to a physical address Date: Thu, 6 Dec 2018 18:21:34 +0000 Message-Id: <1544120495-17438-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1544120495-17438-1-git-send-email-will.deacon@arm.com> References: <1544120495-17438-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current ioremap() code uses a phys_addr variable at each level of page table, which is confusingly offset by subtracting the base virtual address being mapped so that adding the current virtual address back on when iterating through the page table entries gives back the corresponding physical address. This is fairly confusing and results in all users of phys_addr having to add the current virtual address back on. Instead, this patch just updates phys_addr when iterating over the page table entries, ensuring that it's always up-to-date and doesn't require explicit offsetting. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Cc: Sean Christopherson Tested-by: Sean Christopherson Reviewed-by: Sean Christopherson Signed-off-by: Will Deacon --- lib/ioremap.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 6c72764af19c..10d7c5485c39 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -101,19 +101,18 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, pmd_t *pmd; unsigned long next; - phys_addr -= addr; pmd = pmd_alloc(&init_mm, pud, addr); if (!pmd) return -ENOMEM; do { next = pmd_addr_end(addr, end); - if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr, prot)) continue; - if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_pte_range(pmd, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pmd++, addr = next, addr != end); + } while (pmd++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -142,19 +141,18 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, pud_t *pud; unsigned long next; - phys_addr -= addr; pud = pud_alloc(&init_mm, p4d, addr); if (!pud) return -ENOMEM; do { next = pud_addr_end(addr, end); - if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pud(pud, addr, next, phys_addr, prot)) continue; - if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_pmd_range(pud, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pud++, addr = next, addr != end); + } while (pud++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -164,7 +162,6 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, p4d_t *p4d; unsigned long next; - phys_addr -= addr; p4d = p4d_alloc(&init_mm, pgd, addr); if (!p4d) return -ENOMEM; @@ -173,14 +170,14 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, if (ioremap_p4d_enabled() && ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr + addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr + addr, prot)) + IS_ALIGNED(phys_addr, P4D_SIZE)) { + if (p4d_set_huge(p4d, phys_addr, prot)) continue; } - if (ioremap_pud_range(p4d, addr, next, phys_addr + addr, prot)) + if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM; - } while (p4d++, addr = next, addr != end); + } while (p4d++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -196,14 +193,13 @@ int ioremap_page_range(unsigned long addr, BUG_ON(addr >= end); start = addr; - phys_addr -= addr; pgd = pgd_offset_k(addr); do { next = pgd_addr_end(addr, end); - err = ioremap_p4d_range(pgd, addr, next, phys_addr+addr, prot); + err = ioremap_p4d_range(pgd, addr, next, phys_addr, prot); if (err) break; - } while (pgd++, addr = next, addr != end); + } while (pgd++, phys_addr += (next - addr), addr = next, addr != end); flush_cache_vmap(start, end); From patchwork Thu Dec 6 18:21:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 153055 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10859877ljp; Thu, 6 Dec 2018 10:21:23 -0800 (PST) X-Google-Smtp-Source: AFSGD/UmE4IaYzytn11lgNbDYXyW/QjROo0cZfBiceJeyaiOajAywHzKbCU1TL19rU7pt6+J3BNF X-Received: by 2002:a17:902:74c1:: with SMTP id f1mr28709867plt.273.1544120483221; Thu, 06 Dec 2018 10:21:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544120483; cv=none; d=google.com; s=arc-20160816; b=uLirbCwTxr+gh4P8nvzKe5DXpF3/InmItXEEncJSSGq3FZqHCWng7EEnpRnIPrIPXK B8OXi4QcF5OvBPNXRYI7ULR8d9Ymw3H1N1eSFFKMT/6a6Raooa1wCIOUGNdhbU/nFbDY yEXKKFcSjyXw5Kbc4G8FSl5b5V8xIxNg+jcb0av4cWGA/D4bsE7oy7YDJOyKAvQSZz0d SH6jtOTGHXLO1gdMQOg7uprn+ZWM7KnKxgLiahc5wpxtJwzhOPRZvmxuf19uWLojQ0UA zOAaOIaDO66D2Qv+zorUxiGT/j5rtffMGOpY/cJgbHWBzf8VVkMyQGYDKGvtiBNMu40b itYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=l44sI+xexd99+rlEiWTnTtPsv1VhQI3OmU75tH27288=; b=x+79sYtWY2jk0AIuJiWuOGG+F9i4er6H3pdJQdmZ11SQOPrv5RIyauLJHeunSrciRU N4cxaNcbBkoiUfpZMojgNUVdI7SiPTuyiWwgPR7zrm4QliOVcyw6Yza5jPFpmMH8tFa7 YMDlr3X4eiwyRRbKlFZJl+YjfvUS0K+K+/FJK5cIJH5MPORCAI0mT5e1IR3uKXNL93pY KNTgGg6FzbcgdAjqkGCJ35cpUNSTCchix5K0ioYstSuxJXQa4oPUN0rplmNotwodoiCn Y2xSeW679VK0Hogxl9dT8mYU7xhVeLVnHzh/IT5Y8IM2f7dpkPIYG+3+AnNJkpw6ymDt M8ZA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t13si756655pgm.175.2018.12.06.10.21.22; Thu, 06 Dec 2018 10:21:23 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726158AbeLFSVV (ORCPT + 31 others); Thu, 6 Dec 2018 13:21:21 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57902 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726090AbeLFSVR (ORCPT ); Thu, 6 Dec 2018 13:21:17 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 70FD71650; Thu, 6 Dec 2018 10:21:16 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 40F623F5AF; Thu, 6 Dec 2018 10:21:16 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id C31A81AE0BF5; Thu, 6 Dec 2018 18:21:36 +0000 (GMT) From: Will Deacon To: akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, sean.j.christopherson@intel.com, Will Deacon Subject: [RESEND PATCH v4 5/5] lib/ioremap: Ensure break-before-make is used for huge p4d mappings Date: Thu, 6 Dec 2018 18:21:35 +0000 Message-Id: <1544120495-17438-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1544120495-17438-1-git-send-email-will.deacon@arm.com> References: <1544120495-17438-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Whilst no architectures actually enable support for huge p4d mappings in the vmap area, the code that is implemented should be using break-before-make, as we do for pud and pmd huge entries. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 5 +++++ arch/x86/mm/pgtable.c | 8 ++++++++ include/asm-generic/pgtable.h | 5 +++++ lib/ioremap.c | 27 +++++++++++++++++++++------ 4 files changed, 39 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 786cfa6355be..cf9a26d3d7f5 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1028,3 +1028,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pmd_free(NULL, table); return 1; } + +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; /* Don't attempt a block mapping */ +} diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index e95a7d6ac8f8..b0284eab14dc 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -794,6 +794,14 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +/* + * Until we support 512GB pages, skip them in the vmap area. + */ +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} + #ifdef CONFIG_X86_64 /** * pud_free_pmd_page - Clear pud entry and free pmd page. diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 359fb935ded6..e0381a4ce7d4 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1019,6 +1019,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ @@ -1046,6 +1047,10 @@ static inline int pmd_clear_huge(pmd_t *pmd) { return 0; } +static inline int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return 0; diff --git a/lib/ioremap.c b/lib/ioremap.c index 10d7c5485c39..063213685563 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -156,6 +156,25 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, return 0; } +static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_p4d_enabled()) + return 0; + + if ((end - addr) != P4D_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, P4D_SIZE)) + return 0; + + if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) + return 0; + + return p4d_set_huge(p4d, phys_addr, prot); +} + static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -168,12 +187,8 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, do { next = p4d_addr_end(addr, end); - if (ioremap_p4d_enabled() && - ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr, prot)) - continue; - } + if (ioremap_try_huge_p4d(p4d, addr, next, phys_addr, prot)) + continue; if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM;