From patchwork Thu Aug 20 09:23:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kroah-Hartman X-Patchwork-Id: 265556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFD24C433E1 for ; Thu, 20 Aug 2020 11:29:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CAD59207BB for ; Thu, 20 Aug 2020 11:29:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597922957; bh=gXQKUixbcqk1akb5Ehd3N74M/WishC/eje2yI7weVVs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=1mVmTErHIxmPscpDtJG/gPEpUnrSRdChHTrGT5HHLVVAR39HSlBJr/8lx9ZnB5C/w 0b/uL20VkVtsdewoOTPPW8qS+AIdQggSkb4Lanxy/4NtyC6F3RPENTzpVj25+Gf3TZ sYdlBRwNXHeDxwKyW752cWpVisxAsdVF/SvGVVVc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728515AbgHTL3L (ORCPT ); Thu, 20 Aug 2020 07:29:11 -0400 Received: from mail.kernel.org ([198.145.29.99]:33454 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728227AbgHTKFi (ORCPT ); Thu, 20 Aug 2020 06:05:38 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DC53120738; Thu, 20 Aug 2020 10:05:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597917937; bh=gXQKUixbcqk1akb5Ehd3N74M/WishC/eje2yI7weVVs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=COYcmwVEv78o0jS99oaikSs9kvvIk22gYBfQvuoo5r5r08vI0O9YXKm2Is3HhKVe1 zO5bXKXDvxv/XQUteZBnt0oeLlQm8ZidyMobMT2IX8qWGgclIecnBXNLfk0Ha8zRT8 j3a7njy5WMT12bjrckgHpKdzT4P+5TmQhTbLHydk= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Hugh Dickins , Andrew Morton , "Kirill A. Shutemov" , Andrea Arcangeli , Mike Kravetz , Song Liu , Linus Torvalds Subject: [PATCH 4.9 210/212] khugepaged: retract_page_tables() remember to test exit Date: Thu, 20 Aug 2020 11:23:03 +0200 Message-Id: <20200820091612.950775980@linuxfoundation.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200820091602.251285210@linuxfoundation.org> References: <20200820091602.251285210@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Hugh Dickins commit 18e77600f7a1ed69f8ce46c9e11cad0985712dfa upstream. Only once have I seen this scenario (and forgot even to notice what forced the eventual crash): a sequence of "BUG: Bad page map" alerts from vm_normal_page(), from zap_pte_range() servicing exit_mmap(); pmd:00000000, pte values corresponding to data in physical page 0. The pte mappings being zapped in this case were supposed to be from a huge page of ext4 text (but could as well have been shmem): my belief is that it was racing with collapse_file()'s retract_page_tables(), found *pmd pointing to a page table, locked it, but *pmd had become 0 by the time start_pte was decided. In most cases, that possibility is excluded by holding mmap lock; but exit_mmap() proceeds without mmap lock. Most of what's run by khugepaged checks khugepaged_test_exit() after acquiring mmap lock: khugepaged_collapse_pte_mapped_thps() and hugepage_vma_revalidate() do so, for example. But retract_page_tables() did not: fix that. The fix is for retract_page_tables() to check khugepaged_test_exit(), after acquiring mmap lock, before doing anything to the page table. Getting the mmap lock serializes with __mmput(), which briefly takes and drops it in __khugepaged_exit(); then the khugepaged_test_exit() check on mm_users makes sure we don't touch the page table once exit_mmap() might reach it, since exit_mmap() will be proceeding without mmap lock, not expecting anyone to be racing with it. Fixes: f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Signed-off-by: Andrew Morton Acked-by: Kirill A. Shutemov Cc: Andrea Arcangeli Cc: Mike Kravetz Cc: Song Liu Cc: [4.8+] Link: http://lkml.kernel.org/r/alpine.LSU.2.11.2008021215400.27773@eggly.anvils Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/khugepaged.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1250,6 +1250,7 @@ static void collect_mm_slot(struct mm_sl static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) { struct vm_area_struct *vma; + struct mm_struct *mm; unsigned long addr; pmd_t *pmd, _pmd; @@ -1263,7 +1264,8 @@ static void retract_page_tables(struct a continue; if (vma->vm_end < addr + HPAGE_PMD_SIZE) continue; - pmd = mm_find_pmd(vma->vm_mm, addr); + mm = vma->vm_mm; + pmd = mm_find_pmd(mm, addr); if (!pmd) continue; /* @@ -1272,14 +1274,16 @@ static void retract_page_tables(struct a * re-fault. Not ideal, but it's more important to not disturb * the system too much. */ - if (down_write_trylock(&vma->vm_mm->mmap_sem)) { - spinlock_t *ptl = pmd_lock(vma->vm_mm, pmd); - /* assume page table is clear */ - _pmd = pmdp_collapse_flush(vma, addr, pmd); - spin_unlock(ptl); - up_write(&vma->vm_mm->mmap_sem); - atomic_long_dec(&vma->vm_mm->nr_ptes); - pte_free(vma->vm_mm, pmd_pgtable(_pmd)); + if (down_write_trylock(&mm->mmap_sem)) { + if (!khugepaged_test_exit(mm)) { + spinlock_t *ptl = pmd_lock(mm, pmd); + /* assume page table is clear */ + _pmd = pmdp_collapse_flush(vma, addr, pmd); + spin_unlock(ptl); + atomic_long_dec(&mm->nr_ptes); + pte_free(mm, pmd_pgtable(_pmd)); + } + up_write(&mm->mmap_sem); } } i_mmap_unlock_write(mapping);