From patchwork Tue Mar 10 12:38:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 229441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7381EC18E5A for ; Tue, 10 Mar 2020 13:37:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4D2BE20675 for ; Tue, 10 Mar 2020 13:37:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1583847422; bh=q0dA++JbjvCA5R5jLB67BZqMzPWHMTL43xyWr3Uu7ck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=RQl8UPV+kJ1bcXPS9D7LM/zPXXrON9jVbjhTGFxpJBkGFPfMDR9ThF+o2ybbt9mWc wzfuj1Em3j+JMLN/vdtOJUv5zfgLqaPqk5Id7tyH9EQ9d8Laa1YTyM9QShEaHjY62s fjpu9fq+MBnoHBIaacuOHuaNNg7TNmK1n0coJwPY= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727154AbgCJMly (ORCPT ); Tue, 10 Mar 2020 08:41:54 -0400 Received: from mail.kernel.org ([198.145.29.99]:41608 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727124AbgCJMly (ORCPT ); Tue, 10 Mar 2020 08:41:54 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A239424686; Tue, 10 Mar 2020 12:41:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1583844113; bh=q0dA++JbjvCA5R5jLB67BZqMzPWHMTL43xyWr3Uu7ck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RtZgkY3nFHSQ1h/laIeOEJyWBgtkFDNy0NkzrQumaciVvgywgyAhPuE1mgSNW6OWP 3CkJaHNQRBWScccfzPAvzeSNlZgRJS/RnOCXyxRDDju2O/Ic4XKV15sRSkoNQozrQQ tP9xYUi/qbBnkL1sdWhaKHIa88BV/ScFy83Mrobw= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Punit Agrawal , Steve Capper , Michal Hocko , "Kirill A. Shutemov" , "Aneesh Kumar K.V" , Catalin Marinas , Will Deacon , Naoya Horiguchi , Mark Rutland , Hillf Danton , Mike Kravetz , Andrew Morton , Linus Torvalds , Ajay Kaher , Vlastimil Babka Subject: [PATCH 4.4 35/72] mm, gup: ensure real head page is ref-counted when using hugepages Date: Tue, 10 Mar 2020 13:38:48 +0100 Message-Id: <20200310123609.865580324@linuxfoundation.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200310123601.053680753@linuxfoundation.org> References: <20200310123601.053680753@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Punit Agrawal commit d63206ee32b6e64b0e12d46e5d6004afd9913713 upstream. When speculatively taking references to a hugepage using page_cache_add_speculative() in gup_huge_pmd(), it is assumed that the page returned by pmd_page() is the head page. Although normally true, this assumption doesn't hold when the hugepage comprises of successive page table entries such as when using contiguous bit on arm64 at PTE or PMD levels. This can be addressed by ensuring that the page passed to page_cache_add_speculative() is the real head or by de-referencing the head page within the function. We take the first approach to keep the usage pattern aligned with page_cache_get_speculative() where users already pass the appropriate page, i.e., the de-referenced head. Apply the same logic to fix gup_huge_[pud|pgd]() as well. [punit.agrawal@arm.com: fix arm64 ltp failure] Link: http://lkml.kernel.org/r/20170619170145.25577-5-punit.agrawal@arm.com Link: http://lkml.kernel.org/r/20170522133604.11392-3-punit.agrawal@arm.com Signed-off-by: Punit Agrawal Acked-by: Steve Capper Cc: Michal Hocko Cc: "Kirill A. Shutemov" Cc: Aneesh Kumar K.V Cc: Catalin Marinas Cc: Will Deacon Cc: Naoya Horiguchi Cc: Mark Rutland Cc: Hillf Danton Cc: Mike Kravetz Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Ajay Kaher Signed-off-by: Vlastimil Babka Signed-off-by: Greg Kroah-Hartman --- mm/gup.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) --- a/mm/gup.c +++ b/mm/gup.c @@ -1130,8 +1130,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_ return 0; refs = 0; - head = pmd_page(orig); - page = head + ((addr & ~PMD_MASK) >> PAGE_SHIFT); + page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); tail = page; do { pages[*nr] = page; @@ -1140,6 +1139,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_ refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pmd_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1176,8 +1176,7 @@ static int gup_huge_pud(pud_t orig, pud_ return 0; refs = 0; - head = pud_page(orig); - page = head + ((addr & ~PUD_MASK) >> PAGE_SHIFT); + page = pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); tail = page; do { pages[*nr] = page; @@ -1186,6 +1185,7 @@ static int gup_huge_pud(pud_t orig, pud_ refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pud_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1218,8 +1218,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_ return 0; refs = 0; - head = pgd_page(orig); - page = head + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); + page = pgd_page(orig) + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); tail = page; do { pages[*nr] = page; @@ -1228,6 +1227,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_ refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(pgd_page(orig)); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0;