From patchwork Thu Apr 28 15:42:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567764 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55CE2C433EF for ; Thu, 28 Apr 2022 15:42:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349437AbiD1Pp6 (ORCPT ); Thu, 28 Apr 2022 11:45:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349380AbiD1Ppr (ORCPT ); Thu, 28 Apr 2022 11:45:47 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F4BEB6E60; Thu, 28 Apr 2022 08:42:32 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C855D61FAE; Thu, 28 Apr 2022 15:42:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6E9D5C385A0; Thu, 28 Apr 2022 15:42:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160551; bh=LWyqadbMX4amHy8ej3ONoYy1N7Vi21lmkuoAe29epNc=; h=From:To:Cc:Subject:Date:From; b=HjkJLc1wNNAyH1cLytrkHl+dQ+0CBbIjEdE4CoeM/h102Ex1eYwk8wRrXQSeKWc5Q Er2YfUzL6pv2U2BFeAzVhnv8mwjuwPadG/2CJH8LCBnG9MTHbTJEUbfW+O2a/eCWqf kvTTExklafsb3xldxza7H3jcP1ubIyb84nW/Ksgc= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Peter Xu , John Hubbard , Claudio Imbrenda , Alex Williamson , Christoph Hellwig , Jan Kara , Andrea Arcangeli , "Kirill A . Shutemov" , Jason Gunthorpe , David Hildenbrand , Lukas Bulwahn , Matthew Wilcox , Jason Gunthorpe , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 01/14] mm: fix invalid page pointer returned with FOLL_PIN gups Date: Thu, 28 Apr 2022 17:42:09 +0200 Message-Id: <20220428154222.1230793-1-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3214; i=gregkh@linuxfoundation.org; h=from:subject; bh=gsjByhgD7RsvTzmoVWRKhBN4eOYCQhhmeLufTdnA7Mc=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW29FJmY1djf5GD/S4WnOtZ+wOlPjvRULg8mm1/8mxfmd OP+nI5aFQZCJQVZMkeXLNp6j+ysOKXoZ2p6GmcPKBDKEgYtTACYyZybDPPOTB5bJZ/+L3KwpG7hKKe JjR7RFM8OC7RVxd3YczHl5+do5YQ2eL78eCQmpAgA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Xu commit 7196040e19ad634293acd3eff7083149d7669031 upstream. Patch series "mm/gup: some cleanups", v5. This patch (of 5): Alex reported invalid page pointer returned with pin_user_pages_remote() from vfio after upstream commit 4b6c33b32296 ("vfio/type1: Prepare for batched pinning with struct vfio_batch"). It turns out that it's not the fault of the vfio commit; however after vfio switches to a full page buffer to store the page pointers it starts to expose the problem easier. The problem is for VM_PFNMAP vmas we should normally fail with an -EFAULT then vfio will carry on to handle the MMIO regions. However when the bug triggered, follow_page_mask() returned -EEXIST for such a page, which will jump over the current page, leaving that entry in **pages untouched. However the caller is not aware of it, hence the caller will reference the page as usual even if the pointer data can be anything. We had that -EEXIST logic since commit 1027e4436b6a ("mm: make GUP handle pfn mapping unless FOLL_GET is requested") which seems very reasonable. It could be that when we reworked GUP with FOLL_PIN we could have overlooked that special path in commit 3faa52c03f44 ("mm/gup: track FOLL_PIN pages"), even if that commit rightfully touched up follow_devmap_pud() on checking FOLL_PIN when it needs to return an -EEXIST. Attaching the Fixes to the FOLL_PIN rework commit, as it happened later than 1027e4436b6a. [jhubbard@nvidia.com: added some tags, removed a reference to an out of tree module.] Link: https://lkml.kernel.org/r/20220207062213.235127-1-jhubbard@nvidia.com Link: https://lkml.kernel.org/r/20220204020010.68930-1-jhubbard@nvidia.com Link: https://lkml.kernel.org/r/20220204020010.68930-2-jhubbard@nvidia.com Fixes: 3faa52c03f44 ("mm/gup: track FOLL_PIN pages") Signed-off-by: Peter Xu Signed-off-by: John Hubbard Reviewed-by: Claudio Imbrenda Reported-by: Alex Williamson Debugged-by: Alex Williamson Tested-by: Alex Williamson Reviewed-by: Christoph Hellwig Reviewed-by: Jan Kara Cc: Andrea Arcangeli Cc: Kirill A. Shutemov Cc: Jason Gunthorpe Cc: David Hildenbrand Cc: Lukas Bulwahn Cc: Matthew Wilcox (Oracle) Cc: Jason Gunthorpe Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/gup.c b/mm/gup.c index 7bc1ba9ce440..41da0bd61bec 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -465,7 +465,7 @@ static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address, pte_t *pte, unsigned int flags) { /* No page to get reference */ - if (flags & FOLL_GET) + if (flags & (FOLL_GET | FOLL_PIN)) return -EFAULT; if (flags & FOLL_TOUCH) { From patchwork Thu Apr 28 15:42:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA2C6C433F5 for ; Thu, 28 Apr 2022 15:42:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349403AbiD1Ppt (ORCPT ); Thu, 28 Apr 2022 11:45:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344055AbiD1Ppo (ORCPT ); Thu, 28 Apr 2022 11:45:44 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06EFAAC04B; Thu, 28 Apr 2022 08:42:29 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 96F8A61F95; Thu, 28 Apr 2022 15:42:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E38AC385A0; Thu, 28 Apr 2022 15:42:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160548; bh=W6JZHURiD6S8sCBWTVgKCEUD2N8YrqtIi8H2lqDIUIE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=2FTCNuEMytEk2OLSJR+am6TFPzLZzc9hAU1K91ZblJPHOU3wXQN8kU6njVxB8Py/l M36IPZ6KvEBBoep7xbkFbx5r+H3i9WqDDOrSesrxA394/eBd4mGWEqQoOCqjrRfn/Q QskWcYrWsaZI4jQM1mJ2aVtC2QaJznoMfzJqN8Qc= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Zi Yan , Axel Rasmussen , David Rientjes , Fam Zheng , "Kirill A . Shutemov" , Lars Persson , Mike Kravetz , Peter Xu , Xiongchun Duan , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 02/14] mm: fix missing cache flush for all tail pages of compound page Date: Thu, 28 Apr 2022 17:42:10 +0200 Message-Id: <20220428154222.1230793-2-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2363; i=gregkh@linuxfoundation.org; h=from:subject; bh=tzQoyaSqlxtKRFY8pVM4ItgoSNdr8h3zMFKbCiWeB+o=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW++4TAoPCs68vr7igM3rJk+lvg4Rixv6869mVp2wv2Cx +2N3RywLgyATg6yYIsuXbTxH91ccUvQytD0NM4eVCWQIAxenAEzk/FeGBWdvSqqovPy9V3b1Oqf2r0 sqt79pk2SYHyd3/rzal4VSM3hYg4XL8hQ/L9jwCAA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Muchun Song commit 2771739a7162782c0aa6424b2e3dd874e884a15d upstream. The D-cache maintenance inside move_to_new_page() only consider one page, there is still D-cache maintenance issue for tail pages of compound page (e.g. THP or HugeTLB). THP migration is only enabled on x86_64, ARM64 and powerpc, while powerpc and arm64 need to maintain the consistency between I-Cache and D-Cache, which depends on flush_dcache_page() to maintain the consistency between I-Cache and D-Cache. But there is no issues on arm64 and powerpc since they already considers the compound page cache flushing in their icache flush function. HugeTLB migration is enabled on arm, arm64, mips, parisc, powerpc, riscv, s390 and sh, while arm has handled the compound page cache flush in flush_dcache_page(), but most others do not. In theory, the issue exists on many architectures. Fix this by not using flush_dcache_folio() since it is not backportable. Link: https://lkml.kernel.org/r/20220210123058.79206-3-songmuchun@bytedance.com Fixes: 290408d4a250 ("hugetlb: hugepage migration core") Signed-off-by: Muchun Song Reviewed-by: Zi Yan Cc: Axel Rasmussen Cc: David Rientjes Cc: Fam Zheng Cc: Kirill A. Shutemov Cc: Lars Persson Cc: Mike Kravetz Cc: Peter Xu Cc: Xiongchun Duan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/migrate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 086a36637467..fc0e14ecd42a 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -916,9 +916,12 @@ static int move_to_new_page(struct page *newpage, struct page *page, if (!PageMappingFlags(page)) page->mapping = NULL; - if (likely(!is_zone_device_page(newpage))) - flush_dcache_page(newpage); + if (likely(!is_zone_device_page(newpage))) { + int i, nr = compound_nr(newpage); + for (i = 0; i < nr; i++) + flush_dcache_page(newpage + i); + } } out: return rc; From patchwork Thu Apr 28 15:42:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46547C43217 for ; Thu, 28 Apr 2022 15:42:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349453AbiD1PqH (ORCPT ); Thu, 28 Apr 2022 11:46:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349432AbiD1PqG (ORCPT ); Thu, 28 Apr 2022 11:46:06 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2F5B97BA4; Thu, 28 Apr 2022 08:42:51 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 9FE92B82919; Thu, 28 Apr 2022 15:42:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E5B9C385AE; Thu, 28 Apr 2022 15:42:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160569; bh=vTiTP7APfnXW+lzB1PYghHO5btI5VUE0Bb+7eSWCJao=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vj5em6zaSmA0fQ4h3Wl+2NIaKChNi94AD96GtWWo210nnOg5ILZLXWP35M5oJoCE1 rsQwqEMcRH0VCQX178rpyK5ad++NlYKQtNRa35M7H1DBpHAeigQS/egrLE5xDiVKy/ bJ2RigzmX2F+nbsUDf6ObfNbYAeFPvCg+wuUbmsc= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Mike Kravetz , Axel Rasmussen , David Rientjes , Fam Zheng , "Kirill A . Shutemov" , Lars Persson , Peter Xu , Xiongchun Duan , Zi Yan , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 03/14] mm: hugetlb: fix missing cache flush in copy_huge_page_from_user() Date: Thu, 28 Apr 2022 17:42:11 +0200 Message-Id: <20220428154222.1230793-3-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1634; i=gregkh@linuxfoundation.org; h=from:subject; bh=RDNUkPqpYNC5Fqd1rPdREkz34APkzrftk+046XZIsXU=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8w9h6q2DrPSUctb/PaO1o2O1PniZ57Iv1iZoRWl0fU /A+fO2JZGASZGGTFFFm+bOM5ur/ikKKXoe1pmDmsTCBDGLg4BWAikXwM8yv97fbsizJcusV7puzR3m 3BfYdvvmJYcKXWgj/77RUB926ODUdS7X7UpRpsBQA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Muchun Song commit e763243cc6cb1fcc720ec58cfd6e7c35ae90a479 upstream. userfaultfd calls copy_huge_page_from_user() which does not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which might have an alias issue with the kernel address used to copy the data from the user to. Fix this issue by flushing dcache in copy_huge_page_from_user(). Link: https://lkml.kernel.org/r/20220210123058.79206-4-songmuchun@bytedance.com Fixes: fa4d75c1de13 ("userfaultfd: hugetlbfs: add copy_huge_page_from_user for hugetlb userfaultfd support") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Cc: Axel Rasmussen Cc: David Rientjes Cc: Fam Zheng Cc: Kirill A. Shutemov Cc: Lars Persson Cc: Peter Xu Cc: Xiongchun Duan Cc: Zi Yan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/memory.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index b69afe3dd597..886925d97759 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5475,6 +5475,8 @@ long copy_huge_page_from_user(struct page *dst_page, if (rc) break; + flush_dcache_page(subpage); + cond_resched(); } return ret_val; From patchwork Thu Apr 28 15:42:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A3CAC433F5 for ; Thu, 28 Apr 2022 15:43:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347971AbiD1PqV (ORCPT ); Thu, 28 Apr 2022 11:46:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349432AbiD1PqU (ORCPT ); Thu, 28 Apr 2022 11:46:20 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C937AB3DCB; Thu, 28 Apr 2022 08:42:54 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 78605B82E5E; Thu, 28 Apr 2022 15:42:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 796F1C385AE; Thu, 28 Apr 2022 15:42:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160572; bh=FigP1rKu5Y706yYbX73vbKh4dLtC3zR1Bbs8LKoNESA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mBgp7CDHDBsvGPmxf48mgvcPTrEKpp/9bsSeDh2RNuSe4Jl7Aje5Su/+kdCxUracj XEpcTr509prF175AmINfaAoIn+uFlPQX7U6p8a/Ts0gIz0noYMl16ZJLu2g2j5VA5j LJEGSqwpNGjD/aXx9OCtWiQKOZP+1twuLtmApyvE= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Mike Kravetz , Axel Rasmussen , David Rientjes , Fam Zheng , "Kirill A . Shutemov" , Lars Persson , Peter Xu , Xiongchun Duan , Zi Yan , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 04/14] mm: hugetlb: fix missing cache flush in hugetlb_mcopy_atomic_pte() Date: Thu, 28 Apr 2022 17:42:12 +0200 Message-Id: <20220428154222.1230793-4-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1942; i=gregkh@linuxfoundation.org; h=from:subject; bh=Bx716YFt3zvnDbB2iimMq8Rzl+VGbS5K+OqBTq22op8=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+/8f37obMjhjuuxE7+eOxbQLHV3U5QhU8jWTDGbtK5k 0bM9HbEsDIJMDLJiiixftvEc3V9xSNHL0PY0zBxWJpAhDFycAjCRj0sY5sq4fmbQK5I9arnhlSKTcf W0vyytLxjmmekePSB2Zvnu+LBFLN2Xuf+aiWbzAAA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Muchun Song commit 348923665a0e50ad9fc0b3bb8127d3cb976691cc upstream. folio_copy() will copy the data from one page to the target page, then the target page will be mapped to the user space address, which might have an alias issue with the kernel address used to copy the data from the page to. There are 2 ways to fix this issue. 1) insert flush_dcache_page() after folio_copy(). 2) replace folio_copy() with copy_user_huge_page() which already considers the cache maintenance. We chose 2) way to fix the issue since architectures can optimize this situation. It is also make backports easier. Link: https://lkml.kernel.org/r/20220210123058.79206-5-songmuchun@bytedance.com Fixes: 8cc5fcbb5be8 ("mm, hugetlb: fix racy resv_huge_pages underflow on UFFDIO_COPY") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Cc: Axel Rasmussen Cc: David Rientjes Cc: Fam Zheng Cc: Kirill A. Shutemov Cc: Lars Persson Cc: Peter Xu Cc: Xiongchun Duan Cc: Zi Yan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/hugetlb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a1da8757cc9c..e2dc190c6725 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5820,7 +5820,8 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, *pagep = NULL; goto out; } - folio_copy(page_folio(page), page_folio(*pagep)); + copy_user_huge_page(page, *pagep, dst_addr, dst_vma, + pages_per_huge_page(h)); put_page(*pagep); *pagep = NULL; } From patchwork Thu Apr 28 15:42:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3ECAC433EF for ; Thu, 28 Apr 2022 15:43:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349519AbiD1PqW (ORCPT ); Thu, 28 Apr 2022 11:46:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349466AbiD1PqU (ORCPT ); Thu, 28 Apr 2022 11:46:20 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 201BEB8207; Thu, 28 Apr 2022 08:42:58 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C12CAB82DAA; Thu, 28 Apr 2022 15:42:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C982EC385BF; Thu, 28 Apr 2022 15:42:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160575; bh=tmemQG01tDxaffG/Pfuw0hVRBBqMxYFA4QXgyJ9KyB4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tNVbFGWj4w+O8XJzSS3u5R9zNz+eNvMzluHnAROJ8PEHSppWkxC/Kb1EazourJU6V 4Pf1K0SgnT2l+azMe/3RAD14dhoQshgKCeVlL0FfDqoA7B4tVNV8aLs7zAkN33q04u +154IftS0pcbXeIt6+h5JRXfRcpSHmYNhcPmr5JA= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Mike Kravetz , Axel Rasmussen , David Rientjes , Fam Zheng , "Kirill A . Shutemov" , Lars Persson , Peter Xu , Xiongchun Duan , Zi Yan , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 05/14] mm: shmem: fix missing cache flush in shmem_mfill_atomic_pte() Date: Thu, 28 Apr 2022 17:42:13 +0200 Message-Id: <20220428154222.1230793-5-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1949; i=gregkh@linuxfoundation.org; h=from:subject; bh=Zpdi/DUYbYcHJZrH6XatoKk6jiOZB35pe/zLcwmmE6k=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8I7V/+7fotz2mnl0RMZRSeptb7+8HX9sc9WQ8Dz9v+ 4Y/m6IhlYRBkYpAVU2T5so3n6P6KQ4pehranYeawMoEMYeDiFICJvH/GME/XkV/0Z52gvsadlwwHQk K/Z987IsIwT1X64LHvwY/O7N29c9sv8zo+sbrDrQA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Muchun Song commit 19b482c29b6f3805f1d8e93015847b89e2f7f3b1 upstream. userfaultfd calls shmem_mfill_atomic_pte() which does not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which might have an alias issue with the kernel address used to copy the data from the user to. Insert flush_dcache_page() in non-zero-page case. And replace clear_highpage() with clear_user_highpage() which already considers the cache maintenance. Link: https://lkml.kernel.org/r/20220210123058.79206-6-songmuchun@bytedance.com Fixes: 8d1039634206 ("userfaultfd: shmem: add shmem_mfill_zeropage_pte for userfaultfd support") Fixes: 4c27fe4c4c84 ("userfaultfd: shmem: add shmem_mcopy_atomic_pte for userfaultfd support") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Cc: Axel Rasmussen Cc: David Rientjes Cc: Fam Zheng Cc: Kirill A. Shutemov Cc: Lars Persson Cc: Peter Xu Cc: Xiongchun Duan Cc: Zi Yan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/shmem.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index a09b29ec2b45..7a46419d331d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2357,8 +2357,10 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, /* don't free the page */ goto out_unacct_blocks; } + + flush_dcache_page(page); } else { /* ZEROPAGE */ - clear_highpage(page); + clear_user_highpage(page, dst_addr); } } else { page = *pagep; From patchwork Thu Apr 28 15:42:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52D82C4332F for ; Thu, 28 Apr 2022 15:43:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349529AbiD1Pq2 (ORCPT ); Thu, 28 Apr 2022 11:46:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349506AbiD1PqU (ORCPT ); Thu, 28 Apr 2022 11:46:20 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37B9EB715C; Thu, 28 Apr 2022 08:43:01 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id AB08EB82DAA; Thu, 28 Apr 2022 15:42:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D529FC385AF; Thu, 28 Apr 2022 15:42:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160578; bh=cUTeQdSbuKp1C1IB6s6cp2R4rK9jLAPr9VFjKwq5OVg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NVbN9lEcnOc/+Jxc3KuGDhg6Dr8dnI5JqV5nzay/g7SJqVoLqShtoT6oewTEOLYwD flLEKLx58kSL9hiOdWdGzg23ZYE1TfoW1gPGUCsbpTcgy2HWR66hQLfLqNuQoFf7my sW0RXQzWXELykHKHBoXm5bo5DKucM8/ujbgHuW0s= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Axel Rasmussen , David Rientjes , Fam Zheng , "Kirill A . Shutemov" , Lars Persson , Mike Kravetz , Peter Xu , Xiongchun Duan , Zi Yan , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 06/14] mm: userfaultfd: fix missing cache flush in mcopy_atomic_pte() and __mcopy_atomic() Date: Thu, 28 Apr 2022 17:42:14 +0200 Message-Id: <20220428154222.1230793-6-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1990; i=gregkh@linuxfoundation.org; h=from:subject; bh=w/4yshSSx6bYlycE+CXvFti1EqgLl8fqTCv8PDJp7+o=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8emM35cFrCfpYrsaulchb4ctgd5eWKLZG7/95q8YWr nNd3dcSyMAgyMciKKbJ82cZzdH/FIUUvQ9vTMHNYmUCGMHBxCsBErtQyLJh7/oDIuZBg9RvHvypanU rzKWsu+8swP+T74rvTtdWWGBqf3uH1JJPj3tWP0gA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Muchun Song commit 7c25a0b89a487878b0691e6524fb5a8827322194 upstream. userfaultfd calls mcopy_atomic_pte() and __mcopy_atomic() which do not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which might have an alias issue with the kernel address used to copy the data from the user to. Fix this by insert flush_dcache_page() after copy_from_user() succeeds. Link: https://lkml.kernel.org/r/20220210123058.79206-7-songmuchun@bytedance.com Fixes: b6ebaedb4cb1 ("userfaultfd: avoid mmap_sem read recursion in mcopy_atomic") Fixes: c1a4de99fada ("userfaultfd: mcopy_atomic|mfill_zeropage: UFFDIO_COPY|UFFDIO_ZEROPAGE preparation") Signed-off-by: Muchun Song Cc: Axel Rasmussen Cc: David Rientjes Cc: Fam Zheng Cc: Kirill A. Shutemov Cc: Lars Persson Cc: Mike Kravetz Cc: Peter Xu Cc: Xiongchun Duan Cc: Zi Yan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/userfaultfd.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 885e5adb0168..7259f96faaa0 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -153,6 +153,8 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm, /* don't free the page */ goto out; } + + flush_dcache_page(page); } else { page = *pagep; *pagep = NULL; @@ -628,6 +630,7 @@ static __always_inline ssize_t __mcopy_atomic(struct mm_struct *dst_mm, err = -EFAULT; goto out; } + flush_dcache_page(page); goto retry; } else BUG_ON(page); From patchwork Thu Apr 28 15:42:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76CE8C433EF for ; Thu, 28 Apr 2022 15:43:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349516AbiD1Pq1 (ORCPT ); Thu, 28 Apr 2022 11:46:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349520AbiD1PqU (ORCPT ); Thu, 28 Apr 2022 11:46:20 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9372B821A; Thu, 28 Apr 2022 08:43:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 19A5262010; Thu, 28 Apr 2022 15:43:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EAF52C385AE; Thu, 28 Apr 2022 15:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160581; bh=uBhpMoiEbz/kzy6AX3rpsSw0CWwU7xwTlMeypxaO5Po=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GxkkCEAAWJoTphppzHibkcxUM81IURKMlKh3LYBQkIr+Mn7JeWIL+fLC08TLib+HZ LpQt9ptXPtjFwKApX6HukTOWXquSUqV0D0eeIeZjbfamW2wITFjq7c7HMGdyVYYdNh VzvPyu8kGK2CowpYFCOXtfvUY5cI32P2pEYfLQhY= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Mel Gorman , Vlastimil Babka , Aaron Lu , Dave Hansen , Michal Hocko , Jesper Dangaard Brouer , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 07/14] mm/page_alloc: fetch the correct pcp buddy during bulk free Date: Thu, 28 Apr 2022 17:42:15 +0200 Message-Id: <20220428154222.1230793-7-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3124; i=gregkh@linuxfoundation.org; h=from:subject; bh=tCOXBoUo5NxQepLUpCqvv6otafb9oNj1S1MmG8sm4mw=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8efZNz48WD91FnTyqYi+YEL07d4rHs3UYJaV+966H5 Bxde6YhlYRBkYpAVU2T5so3n6P6KQ4pehranYeawMoEMYeDiFICJpK5gmB/04G/zEu+3uUs9PzIJPS 1qfW3X/othnpriz49rzNUUXN+GcV88sNXiuaHaLwA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Mel Gorman commit ca7b59b1de72450b3e696bada3506a519ac5455c upstream. Patch series "Follow-up on high-order PCP caching", v2. Commit 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") was primarily aimed at reducing the cost of SLUB cache refills of high-order pages in two ways. Firstly, zone lock acquisitions was reduced and secondly, there were fewer buddy list modifications. This is a follow-up series fixing some issues that became apparant after merging. Patch 1 is a functional fix. It's harmless but inefficient. Patches 2-5 reduce the overhead of bulk freeing of PCP pages. While the overhead is small, it's cumulative and noticable when truncating large files. The changelog for patch 4 includes results of a microbench that deletes large sparse files with data in page cache. Sparse files were used to eliminate filesystem overhead. Patch 6 addresses issues with high-order PCP pages being stored on PCP lists for too long. Pages freed on a CPU potentially may not be quickly reused and in some cases this can increase cache miss rates. Details are included in the changelog. This patch (of 6): free_pcppages_bulk() prefetches buddies about to be freed but the order must also be passed in as PCP lists store multiple orders. Link: https://lkml.kernel.org/r/20220217002227.5739-1-mgorman@techsingularity.net Link: https://lkml.kernel.org/r/20220217002227.5739-2-mgorman@techsingularity.net Fixes: 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") Signed-off-by: Mel Gorman Reviewed-by: Vlastimil Babka Reviewed-by: Aaron Lu Tested-by: Aaron Lu Cc: Dave Hansen Cc: Michal Hocko Cc: Jesper Dangaard Brouer Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/page_alloc.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e6f211dcf82e..b2ef0e75fd29 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1432,10 +1432,10 @@ static bool bulkfree_pcp_prepare(struct page *page) } #endif /* CONFIG_DEBUG_VM */ -static inline void prefetch_buddy(struct page *page) +static inline void prefetch_buddy(struct page *page, unsigned int order) { unsigned long pfn = page_to_pfn(page); - unsigned long buddy_pfn = __find_buddy_pfn(pfn, 0); + unsigned long buddy_pfn = __find_buddy_pfn(pfn, order); struct page *buddy = page + (buddy_pfn - pfn); prefetch(buddy); @@ -1512,7 +1512,7 @@ static void free_pcppages_bulk(struct zone *zone, int count, * prefetch buddy for the first pcp->batch nr of pages. */ if (prefetch_nr) { - prefetch_buddy(page); + prefetch_buddy(page, order); prefetch_nr--; } } while (count > 0 && --batch_free && !list_empty(list)); From patchwork Thu Apr 28 15:42:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA392C433F5 for ; Thu, 28 Apr 2022 15:43:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349499AbiD1PqZ (ORCPT ); Thu, 28 Apr 2022 11:46:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349525AbiD1PqV (ORCPT ); Thu, 28 Apr 2022 11:46:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44650B822D; Thu, 28 Apr 2022 08:43:05 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C4A6B61F94; Thu, 28 Apr 2022 15:43:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABE76C385A9; Thu, 28 Apr 2022 15:43:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160584; bh=Fq4knAc0N9RbSq3BiSeCJKGfs+zsd8UULSsA9idnt9I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DhFQEvAacfymIR2uipsYpNI/NEVaKi8nVS6P/l+bb4PqbYXvAdJUr9PY1JhRBcBFE Y/PqMCMGkskzKia+1sInvGmkTYEl2Gu+M6fkEdiyAsqFXZ+6JQb0OM3RSBYvic3msl 3aSFfsGCmxJG5iLjgOk6ifzmhYFOYbgy7fMFrpjw= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Mel Gorman , Eric Dumazet , Shakeel Butt , Vlastimil Babka , David Rientjes , Michal Hocko , Wei Xu , Greg Thelen , Hugh Dickins , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 08/14] mm/page_alloc: check high-order pages for corruption during PCP operations Date: Thu, 28 Apr 2022 17:42:16 +0200 Message-Id: <20220428154222.1230793-8-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4241; i=gregkh@linuxfoundation.org; h=from:subject; bh=RipHcCeR+SVHuWApuTUzNGeBcEpnHJuOJblMoYeKjOw=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8q9Hv/Ovfr6sfZApFxL36u2yT0d/P0So5UNZuaY+5m FT0SHbEsDIJMDLJiiixftvEc3V9xSNHL0PY0zBxWJpAhDFycAjCRuVMY5kcdXiNlVL3pw5F/xu/sp1 cl/Qm6pM0wTyXn0wkuVpGzbSfEPRo+HDPMjNRtBAA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Mel Gorman commit 77fe7f136a7312954b1b8b7eeb4bc91fc3c14a3f upstream. Eric Dumazet pointed out that commit 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") only checks the head page during PCP refill and allocation operations. This was an oversight and all pages should be checked. This will incur a small performance penalty but it's necessary for correctness. Link: https://lkml.kernel.org/r/20220310092456.GJ15701@techsingularity.net Fixes: 44042b449872 ("mm/page_alloc: allow high-order pages to be stored on the per-cpu lists") Signed-off-by: Mel Gorman Reported-by: Eric Dumazet Acked-by: Eric Dumazet Reviewed-by: Shakeel Butt Acked-by: Vlastimil Babka Acked-by: David Rientjes Cc: Michal Hocko Cc: Wei Xu Cc: Greg Thelen Cc: Hugh Dickins Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/page_alloc.c | 46 +++++++++++++++++++++++----------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b2ef0e75fd29..adceee44adf6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2342,23 +2342,36 @@ static inline int check_new_page(struct page *page) return 1; } +static bool check_new_pages(struct page *page, unsigned int order) +{ + int i; + for (i = 0; i < (1 << order); i++) { + struct page *p = page + i; + + if (unlikely(check_new_page(p))) + return true; + } + + return false; +} + #ifdef CONFIG_DEBUG_VM /* * With DEBUG_VM enabled, order-0 pages are checked for expected state when * being allocated from pcp lists. With debug_pagealloc also enabled, they are * also checked when pcp lists are refilled from the free lists. */ -static inline bool check_pcp_refill(struct page *page) +static inline bool check_pcp_refill(struct page *page, unsigned int order) { if (debug_pagealloc_enabled_static()) - return check_new_page(page); + return check_new_pages(page, order); else return false; } -static inline bool check_new_pcp(struct page *page) +static inline bool check_new_pcp(struct page *page, unsigned int order) { - return check_new_page(page); + return check_new_pages(page, order); } #else /* @@ -2366,32 +2379,19 @@ static inline bool check_new_pcp(struct page *page) * when pcp lists are being refilled from the free lists. With debug_pagealloc * enabled, they are also checked when being allocated from the pcp lists. */ -static inline bool check_pcp_refill(struct page *page) +static inline bool check_pcp_refill(struct page *page, unsigned int order) { - return check_new_page(page); + return check_new_pages(page, order); } -static inline bool check_new_pcp(struct page *page) +static inline bool check_new_pcp(struct page *page, unsigned int order) { if (debug_pagealloc_enabled_static()) - return check_new_page(page); + return check_new_pages(page, order); else return false; } #endif /* CONFIG_DEBUG_VM */ -static bool check_new_pages(struct page *page, unsigned int order) -{ - int i; - for (i = 0; i < (1 << order); i++) { - struct page *p = page + i; - - if (unlikely(check_new_page(p))) - return true; - } - - return false; -} - inline void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags) { @@ -3037,7 +3037,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, if (unlikely(page == NULL)) break; - if (unlikely(check_pcp_refill(page))) + if (unlikely(check_pcp_refill(page, order))) continue; /* @@ -3641,7 +3641,7 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, page = list_first_entry(list, struct page, lru); list_del(&page->lru); pcp->count -= 1 << order; - } while (check_new_pcp(page)); + } while (check_new_pcp(page, order)); return page; } From patchwork Thu Apr 28 15:42:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4667C433EF for ; Thu, 28 Apr 2022 15:43:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349502AbiD1Pqa (ORCPT ); Thu, 28 Apr 2022 11:46:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349527AbiD1PqZ (ORCPT ); Thu, 28 Apr 2022 11:46:25 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 027B7B3DEE; Thu, 28 Apr 2022 08:43:10 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6D537B82DE9; Thu, 28 Apr 2022 15:43:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFE14C385A0; Thu, 28 Apr 2022 15:43:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160587; bh=oOTTQiTXbJJNutl+GtVstgkF+NXnhIxj+Hxg+8AQ8Dw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OB2EQAs7PyVqpxi2Lfaj07HXTncbkkVCdkHQG/FCb6oyCw3J+usQafXGEpe/99sjo MAmMZSIGmZHNWnris5ZiucTCq0I9vyevcAcc2wAfuXoSR0mlJKQelJwcSG5dXX8vxv P5xcrr6X1UhcXclL0imLtWu3a4M8OfrnNXGitVxU= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Naoya Horiguchi , Youquan Song , Tony Luck , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 09/14] mm/hwpoison: fix error page recovered but reported "not recovered" Date: Thu, 28 Apr 2022 17:42:17 +0200 Message-Id: <20220428154222.1230793-9-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2501; i=gregkh@linuxfoundation.org; h=from:subject; bh=fS6eim227H0Y5MlY1k72kHj4DPyJTiatgbOo3atAeao=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8Wz+d9/WHfovv3719Yo7xhQeOqzbzx5xPCv+2+NWnd QbHUWR2xLAyCTAyyYoosX7bxHN1fcUjRy9D2NMwcViaQIQxcnAIwkSlNDPMsNZcaznmwME66YNryrJ xJy7J4dRUY5ln/sgpvOsDQVqO6o7P2c76epnZ9OAA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Naoya Horiguchi commit 046545a661af2beec21de7b90ca0e35f05088a81 upstream. When an uncorrected memory error is consumed there is a race between the CMCI from the memory controller reporting an uncorrected error with a UCNA signature, and the core reporting and SRAR signature machine check when the data is about to be consumed. If the CMCI wins that race, the page is marked poisoned when uc_decode_notifier() calls memory_failure() and the machine check processing code finds the page already poisoned. It calls kill_accessing_process() to make sure a SIGBUS is sent. But returns the wrong error code. Console log looks like this: mce: Uncorrected hardware memory error in user-access at 3710b3400 Memory failure: 0x3710b3: recovery action for dirty LRU page: Recovered Memory failure: 0x3710b3: already hardware poisoned Memory failure: 0x3710b3: Sending SIGBUS to einj_mem_uc:361438 due to hardware memory corruption mce: Memory error not recovered kill_accessing_process() is supposed to return -EHWPOISON to notify that SIGBUS is already set to the process and kill_me_maybe() doesn't have to send it again. But current code simply fails to do this, so fix it to make sure to work as intended. This change avoids the noise message "Memory error not recovered" and skips duplicate SIGBUSs. [tony.luck@intel.com: reword some parts of commit message] Link: https://lkml.kernel.org/r/20220113231117.1021405-1-naoya.horiguchi@linux.dev Fixes: a3f5d80ea401 ("mm,hwpoison: send SIGBUS with error virutal address") Signed-off-by: Naoya Horiguchi Reported-by: Youquan Song Cc: Tony Luck Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/memory-failure.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 15dcedbc1730..682eedb5ea75 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -707,8 +707,10 @@ static int kill_accessing_process(struct task_struct *p, unsigned long pfn, (void *)&priv); if (ret == 1 && priv.tk.addr) kill_proc(&priv.tk, pfn, flags); + else + ret = 0; mmap_read_unlock(p->mm); - return ret ? -EFAULT : -EHWPOISON; + return ret > 0 ? -EHWPOISON : -EFAULT; } static const char *action_name[] = { From patchwork Thu Apr 28 15:42:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 453FDC43217 for ; Thu, 28 Apr 2022 15:43:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349532AbiD1Pqa (ORCPT ); Thu, 28 Apr 2022 11:46:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349530AbiD1Pq2 (ORCPT ); Thu, 28 Apr 2022 11:46:28 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D12B9B8215; Thu, 28 Apr 2022 08:43:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 62723B82E60; Thu, 28 Apr 2022 15:43:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BB9B0C385AA; Thu, 28 Apr 2022 15:43:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160590; bh=HCykCgYNoQIXUrLzZeUe7vnBxuFoYC6hSwkFvjaczlU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lLOqRVppL/kuLu5r2KOwp7D8Eah07t3rLWrHpGgLlULYNwb0wLIs9dOT8RoVRsNIY PBCUuOP4vzvzhWC4j9hSjHX8/OvOcohmOH2Dk2bRQvjY0hBLd6e3wZspn3oVYMHCaP uCaJ/vpMxo5YMQ7bkBAzrWUn/XDF/nTWuvBaznk8= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Miaohe Lin , Andrew Morton , Hugh Dickins , Herbert van den Bergh , Chris Mason , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 10/14] mm/mlock: fix potential imbalanced rlimit ucounts adjustment Date: Thu, 28 Apr 2022 17:42:18 +0200 Message-Id: <20220428154222.1230793-10-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1240; i=gregkh@linuxfoundation.org; h=from:subject; bh=c7rlKq+E+z1QCnNVW3vftSDp4l2FfRz0dPmO1I7Qwvs=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8K7XfeuajJsv7sQp0AR/Zcz5/Z70+qrAvX6LLrOzsr tW5bRywLgyATg6yYIsuXbTxH91ccUvQytD0NM4eVCWQIAxenAEwkx4phwUXFVZ5cbnmrTbwr7z/MtL +w1MZQimF+oZf8u5NH3z/iex3NmnQk4KWmencrAA== X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Miaohe Lin commit 5c2a956c3eea173b2bc89f632507c0eeaebf6c4a upstream. user_shm_lock forgets to set allowed to 0 when get_ucounts fails. So the later user_shm_unlock might do the extra dec_rlimit_ucounts. Fix this by resetting allowed to 0. Link: https://lkml.kernel.org/r/20220310132417.41189-1-linmiaohe@huawei.com Fixes: d7c9e99aee48 ("Reimplement RLIMIT_MEMLOCK on top of ucounts") Signed-off-by: Miaohe Lin Reviewed-by: Andrew Morton Acked-by: Hugh Dickins Cc: Herbert van den Bergh Cc: Chris Mason Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/mlock.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/mlock.c b/mm/mlock.c index 37f969ec68fa..b565b1aac8d4 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -838,6 +838,7 @@ int user_shm_lock(size_t size, struct ucounts *ucounts) } if (!get_ucounts(ucounts)) { dec_rlimit_ucounts(ucounts, UCOUNT_RLIMIT_MEMLOCK, locked); + allowed = 0; goto out; } allowed = 1; From patchwork Thu Apr 28 15:42:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2CCFC433F5 for ; Thu, 28 Apr 2022 15:42:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349411AbiD1Pp7 (ORCPT ); Thu, 28 Apr 2022 11:45:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349433AbiD1Ppx (ORCPT ); Thu, 28 Apr 2022 11:45:53 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70B17B820D; Thu, 28 Apr 2022 08:42:37 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E661B61FCF; Thu, 28 Apr 2022 15:42:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C22CAC385A0; Thu, 28 Apr 2022 15:42:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160556; bh=HsbSG3CAnlJ/cEIDeoisNp5iwOSsom+ptW9Us2eqOLs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lvtjL7OoWKYGYZu5yPfrNUT5XoOWuWi6TJGPs3ijLwYyRolb0/fDrLLIGvAM/rLXB ui9VdXpOwMZblhcvzIMnj+CGmotU8NPH9kdWuhhgawLTtg7jnGrmADUKm1FH501Yqr k0FfXv2zf6IllzWN60/WKjB13/GCM746mcDf5DxY= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Huang Ying , Baolin Wang , Dave Hansen , Zi Yan , Oscar Salvador , Yang Shi , zhongjiang-ali , Xunlei Pang , Mel Gorman , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 11/14] mm,migrate: fix establishing demotion target Date: Thu, 28 Apr 2022 17:42:19 +0200 Message-Id: <20220428154222.1230793-11-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3098; i=gregkh@linuxfoundation.org; h=from:subject; bh=2ZV8t8XlvoZw7kzU/r+KeXZFYrnKitmm6D3IvWd8z4w=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+8tsKhKk3/AviXlrpfnjq+9T4uDun7d5lnN8OBB1qEX RtV+HbEsDIJMDLJiiixftvEc3V9xSNHL0PY0zBxWJpAhDFycAjCRLxEMc7ijBSZXrGCwKPq7yH/C6m UlOj+mKTAsWOJ+MmKixPEmmTbxWSVKYccNnq4+CQA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Huang Ying commit fc89213a636c3735eb3386f10a34c082271b4192 upstream. In commit ac16ec835314 ("mm: migrate: support multiple target nodes demotion"), after the first demotion target node is found, we will continue to check the next candidate obtained via find_next_best_node(). This is to find all demotion target nodes with same NUMA distance. But one side effect of find_next_best_node() is that the candidate node returned will be set in "used" parameter, even if the candidate node isn't passed in the following NUMA distance checking, the candidate node will not be used as demotion target node for the following nodes. For example, for system as follows, node distances: node 0 1 2 3 0: 10 21 17 28 1: 21 10 28 17 2: 17 28 10 28 3: 28 17 28 10 when we establish demotion target node for node 0, in the first round node 2 is added to the demotion target node set. Then in the second round, node 3 is checked and failed because distance(0, 3) > distance(0, 2). But node 3 is set in "used" nodemask too. When we establish demotion target node for node 1, there is no available node. This is wrong, node 3 should be set as the demotion target of node 1. To fix this, if the candidate node is failed to pass the distance checking, it will be cleared in "used" nodemask. So that it can be used for the following node. The bug can be reproduced and fixed with this patch on a 2 socket server machine with DRAM and PMEM. Link: https://lkml.kernel.org/r/20220128055940.1792614-1-ying.huang@intel.com Fixes: ac16ec835314 ("mm: migrate: support multiple target nodes demotion") Signed-off-by: "Huang, Ying" Reviewed-by: Baolin Wang Cc: Baolin Wang Cc: Dave Hansen Cc: Zi Yan Cc: Oscar Salvador Cc: Yang Shi Cc: zhongjiang-ali Cc: Xunlei Pang Cc: Mel Gorman Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/migrate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index fc0e14ecd42a..ac7673e43dda 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -3085,18 +3085,21 @@ static int establish_migrate_target(int node, nodemask_t *used, if (best_distance != -1) { val = node_distance(node, migration_target); if (val > best_distance) - return NUMA_NO_NODE; + goto out_clear; } index = nd->nr; if (WARN_ONCE(index >= DEMOTION_TARGET_NODES, "Exceeds maximum demotion target nodes\n")) - return NUMA_NO_NODE; + goto out_clear; nd->nodes[index] = migration_target; nd->nr++; return migration_target; +out_clear: + node_clear(migration_target, *used); + return NUMA_NO_NODE; } /* From patchwork Thu Apr 28 15:42:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB04FC4332F for ; Thu, 28 Apr 2022 15:42:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349446AbiD1PqG (ORCPT ); Thu, 28 Apr 2022 11:46:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349384AbiD1Pp6 (ORCPT ); Thu, 28 Apr 2022 11:45:58 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA861B820F; Thu, 28 Apr 2022 08:42:41 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9885961FCF; Thu, 28 Apr 2022 15:42:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C296C385A0; Thu, 28 Apr 2022 15:42:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160560; bh=ilVGieZZEUUvQGWgxV/ekNNEKBv8j99uQo8L2J5iPnc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TphhwJI1zDAFX85nU7OlzRbFcWxeMhtHQXNO03ghzEVhdh+AnuBmpXvO05ukLPq7w 0r2qwhfPcteuyDNlPvIfVvk2dGevzA8z0PqgEwEZOJ7LY9H2r/vjq8H/8D6b5ab4Kg gF0vGZbGbRGXCWzjyld3aE0UXSXnS0vq58UdCfLM= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Hugh Dickins , Yang Shi , Ralph Campbell , Zi Yan , "Kirill A. Shutemov" , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 12/14] mm/thp: refix __split_huge_pmd_locked() for migration PMD Date: Thu, 28 Apr 2022 17:42:20 +0200 Message-Id: <20220428154222.1230793-12-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2125; i=gregkh@linuxfoundation.org; h=from:subject; bh=2n4FFedWfpMpKCUPl3SXm4kSvYrTBoaoARY2+jFW1hk=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW++pcwp8fCAvs2yu/Yys01Pe9v9Y9m9xb8FF0fsRe97w nOXU64hlYRBkYpAVU2T5so3n6P6KQ4pehranYeawMoEMYeDiFICJfLrEsKCha/qJglfTnob7dEYvW/ HF7Xnr230M80vSHtmarV24JTdQ/Mb+UmHJ9D13+AA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Hugh Dickins commit 9d84604b845c3888d1bede43d16ab3ebedb13e24 upstream. Migration entries do not contribute to a page's reference count: move __split_huge_pmd_locked()'s page_ref_add() into pmd_migration's else block (along with the page_count() check - a page is quite likely to have reference count frozen to 0 when a migration entry is found). This will fix a very rare anonymous memory leak, after a split_huge_pmd() raced with an anon split_huge_page() or an anon THP migrate_pages(): since the wrongly raised refcount stopped the page (perhaps small, perhaps huge, depending on when the race hit) from ever being freed. At first I thought there were worse risks, from prematurely unfreezing a frozen page: but now think that would only affect page cache pages, which do not come this way (except for anonymous pages in swap cache, perhaps). Link: https://lkml.kernel.org/r/84792468-f512-e48f-378c-e34c3641e97@google.com Fixes: ec0abae6dcdf ("mm/thp: fix __split_huge_pmd_locked() for migration PMD") Signed-off-by: Hugh Dickins Reviewed-by: Yang Shi Cc: Ralph Campbell Cc: Zi Yan Cc: "Kirill A. Shutemov" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/huge_memory.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 406a3c28c026..468fca576bc2 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2055,9 +2055,9 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, young = pmd_young(old_pmd); soft_dirty = pmd_soft_dirty(old_pmd); uffd_wp = pmd_uffd_wp(old_pmd); + VM_BUG_ON_PAGE(!page_count(page), page); + page_ref_add(page, HPAGE_PMD_NR - 1); } - VM_BUG_ON_PAGE(!page_count(page), page); - page_ref_add(page, HPAGE_PMD_NR - 1); /* * Withdraw the table only after we mark the pmd entry invalid. From patchwork Thu Apr 28 15:42:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4195C433F5 for ; Thu, 28 Apr 2022 15:42:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347707AbiD1PqG (ORCPT ); Thu, 28 Apr 2022 11:46:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348013AbiD1PqB (ORCPT ); Thu, 28 Apr 2022 11:46:01 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54C7FAC061; Thu, 28 Apr 2022 08:42:46 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 902A6B82E5F; Thu, 28 Apr 2022 15:42:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E84EDC385A9; Thu, 28 Apr 2022 15:42:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160563; bh=L2Rcj50kHvipfO9WA+wNHp0G73eGU8osDpLt0jRoNjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YonJc2ne8OrwzooS6kMzRv6f+Sm0KBqC7ar0P8yzF7Ga18ceL7Rd4xBJKxJ9xNb2X yTXiPVXzQX6Blk9TRuslvvgJVPtUFKPQHZjEnHZqz+SAsPPFvMNmNRkyLRyNHrg+N1 z4CuZUCqrR+YqJP4dcLNPJC4rEgz2Qh9FWNano7E= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Hugh Dickins , Yang Shi , "Kirill A. Shutemov" , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 13/14] mm/thp: ClearPageDoubleMap in first page_add_file_rmap() Date: Thu, 28 Apr 2022 17:42:21 +0200 Message-Id: <20220428154222.1230793-13-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1929; i=gregkh@linuxfoundation.org; h=from:subject; bh=t020Ye3UZged8UWdaHnaf2vQNl0s0QD+/We8io6+w0s=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW+9lFBZvbBGY9XOlT7oth9Uhx3V687STop7c3xy5/jLX hDWSHbEsDIJMDLJiiixftvEc3V9xSNHL0PY0zBxWJpAhDFycAjCRMzYMc2Wnic4UOGbrV8cwT/F3xN wk0QdHghgWrPSOWTLpWXxzRcjLhAernpwMPXrYEgA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Hugh Dickins commit bd55b0c2d64e84a75575f548a33a3dfecc135b65 upstream. PageDoubleMap is maintained differently for anon and for shmem+file: the shmem+file one was never cleared, because a safe place to do so could not be found; so it would blight future use of the cached hugepage until evicted. See https://lore.kernel.org/lkml/1571938066-29031-1-git-send-email-yang.shi@linux.alibaba.com/ But page_add_file_rmap() does provide a safe place to do so (though later than one might wish): allowing testing to return to an initial state without a damaging drop_caches. Link: https://lkml.kernel.org/r/61c5cf99-a962-9a25-597a-53ab1bd8fbc0@google.com Fixes: 9a73f61bdb8a ("thp, mlock: do not mlock PTE-mapped file huge pages") Signed-off-by: Hugh Dickins Reviewed-by: Yang Shi Cc: "Kirill A. Shutemov" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/rmap.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/rmap.c b/mm/rmap.c index 9e27f9f038d3..444d0d958aff 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1252,6 +1252,17 @@ void page_add_file_rmap(struct page *page, bool compound) } if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; + + /* + * It is racy to ClearPageDoubleMap in page_remove_file_rmap(); + * but page lock is held by all page_add_file_rmap() compound + * callers, and SetPageDoubleMap below warns if !PageLocked: + * so here is a place that DoubleMap can be safely cleared. + */ + VM_WARN_ON_ONCE(!PageLocked(page)); + if (nr == nr_pages && PageDoubleMap(page)) + ClearPageDoubleMap(page); + if (PageSwapBacked(page)) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, nr_pages); From patchwork Thu Apr 28 15:42:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 567477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 364DBC433FE for ; Thu, 28 Apr 2022 15:42:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349433AbiD1PqG (ORCPT ); Thu, 28 Apr 2022 11:46:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349448AbiD1PqC (ORCPT ); Thu, 28 Apr 2022 11:46:02 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51A58B715C; Thu, 28 Apr 2022 08:42:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CF1FC61F95; Thu, 28 Apr 2022 15:42:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7F3DC385AA; Thu, 28 Apr 2022 15:42:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1651160566; bh=4dKNV+ceiLtd6M4VTt7w+2x33yEvnPjHequM8WdzUY8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ndhhVA0zXP0WKVFtbkSbktkcOBNoTftoBdq4wF6SHbEtuWfWziLbWN05RUXoEPHMH M3tiFUjiIFRjJ+c9hC+k8+BXuliPMVMvD9WfQMUep2NNI2htsWKKElWyw83RvnXrsU gI6I8IePYVp65dcSW2vnN+8fk5VvW+sGOYm9VVg0= From: Greg Kroah-Hartman To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Hugh Dickins , Yang Shi , "Kirill A. Shutemov" , Andrew Morton , Linus Torvalds , Greg Kroah-Hartman Subject: [PATCH AUTOSEL 14/14] mm/thp: fix NR_FILE_MAPPED accounting in page_*_file_rmap() Date: Thu, 28 Apr 2022 17:42:22 +0200 Message-Id: <20220428154222.1230793-14-gregkh@linuxfoundation.org> X-Mailer: git-send-email 2.36.0 In-Reply-To: <20220428154222.1230793-1-gregkh@linuxfoundation.org> References: <20220428154222.1230793-1-gregkh@linuxfoundation.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4320; i=gregkh@linuxfoundation.org; h=from:subject; bh=DY9PobJ+iVCOW4dJoQet7IIvSJy1rG7kl8G4mbSfMDg=; b=owGbwMvMwCRo6H6F97bub03G02pJDElZW++tOOTj5W0tkVux6LONokf0tOCAnWxPfbl+8a1Ov5XE NO1tRywLgyATg6yYIsuXbTxH91ccUvQytD0NM4eVCWQIAxenAExk8iGGBY0ME0XPNm/Ul1RreRU4ma ecd3JQJsOCS/NeH5qxYWqa+NofpXpXd8+MDfSfCAA= X-Developer-Key: i=gregkh@linuxfoundation.org; a=openpgp; fpr=F4B60CC5BF78C2214A313DCB3147D40DDB2DFB29 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Hugh Dickins commit 5d543f13e2f5580828de885c751d68a35b6a493d upstream. NR_FILE_MAPPED accounting in mm/rmap.c (for /proc/meminfo "Mapped" and /proc/vmstat "nr_mapped" and the memcg's memory.stat "mapped_file") is slightly flawed for file or shmem huge pages. It is well thought out, and looks convincing, but there's a racy case when the careful counting in page_remove_file_rmap() (without page lock) gets discarded. So that in a workload like two "make -j20" kernel builds under memory pressure, with cc1 on hugepage text, "Mapped" can easily grow by a spurious 5MB or more on each iteration, ending up implausibly bigger than most other numbers in /proc/meminfo. And, hypothetically, might grow to the point of seriously interfering in mm/vmscan.c's heuristics, which do take NR_FILE_MAPPED into some consideration. Fixed by moving the __mod_lruvec_page_state() down to where it will not be missed before return (and I've grown a bit tired of that oft-repeated but-not-everywhere comment on the __ness: it gets lost in the move here). Does page_add_file_rmap() need the same change? I suspect not, because page lock is held in all relevant cases, and its skipping case looks safe; but it's much easier to be sure, if we do make the same change. Link: https://lkml.kernel.org/r/e02e52a1-8550-a57c-ed29-f51191ea2375@google.com Fixes: dd78fedde4b9 ("rmap: support file thp") Signed-off-by: Hugh Dickins Reviewed-by: Yang Shi Cc: "Kirill A. Shutemov" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/rmap.c | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 444d0d958aff..fa09b5eaff34 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1239,14 +1239,14 @@ void page_add_new_anon_rmap(struct page *page, */ void page_add_file_rmap(struct page *page, bool compound) { - int i, nr = 1; + int i, nr = 0; VM_BUG_ON_PAGE(compound && !PageTransHuge(page), page); lock_page_memcg(page); if (compound && PageTransHuge(page)) { int nr_pages = thp_nr_pages(page); - for (i = 0, nr = 0; i < nr_pages; i++) { + for (i = 0; i < nr_pages; i++) { if (atomic_inc_and_test(&page[i]._mapcount)) nr++; } @@ -1279,17 +1279,18 @@ void page_add_file_rmap(struct page *page, bool compound) if (PageMlocked(page)) clear_page_mlock(head); } - if (!atomic_inc_and_test(&page->_mapcount)) - goto out; + if (atomic_inc_and_test(&page->_mapcount)) + nr++; } - __mod_lruvec_page_state(page, NR_FILE_MAPPED, nr); out: + if (nr) + __mod_lruvec_page_state(page, NR_FILE_MAPPED, nr); unlock_page_memcg(page); } static void page_remove_file_rmap(struct page *page, bool compound) { - int i, nr = 1; + int i, nr = 0; VM_BUG_ON_PAGE(compound && !PageHead(page), page); @@ -1304,12 +1305,12 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (compound && PageTransHuge(page)) { int nr_pages = thp_nr_pages(page); - for (i = 0, nr = 0; i < nr_pages; i++) { + for (i = 0; i < nr_pages; i++) { if (atomic_add_negative(-1, &page[i]._mapcount)) nr++; } if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) - return; + goto out; if (PageSwapBacked(page)) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -nr_pages); @@ -1317,16 +1318,13 @@ static void page_remove_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, -nr_pages); } else { - if (!atomic_add_negative(-1, &page->_mapcount)) - return; + if (atomic_add_negative(-1, &page->_mapcount)) + nr++; } - /* - * We use the irq-unsafe __{inc|mod}_lruvec_page_state because - * these counters are not modified in interrupt context, and - * pte lock(a spinlock) is held, which implies preemption disabled. - */ - __mod_lruvec_page_state(page, NR_FILE_MAPPED, -nr); +out: + if (nr) + __mod_lruvec_page_state(page, NR_FILE_MAPPED, -nr); if (unlikely(PageMlocked(page))) clear_page_mlock(page);