From patchwork Sun Jan 10 12:40:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 361129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8502C43381 for ; Sun, 10 Jan 2021 12:42:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7FFF922E02 for ; Sun, 10 Jan 2021 12:42:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726495AbhAJMmO (ORCPT ); Sun, 10 Jan 2021 07:42:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726494AbhAJMmN (ORCPT ); Sun, 10 Jan 2021 07:42:13 -0500 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAAB6C0617A3 for ; Sun, 10 Jan 2021 04:41:32 -0800 (PST) Received: by mail-pl1-x62a.google.com with SMTP id e2so8038565plt.12 for ; Sun, 10 Jan 2021 04:41:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ffJDakXCDHXSBit9htKSivl1vC+lnq1/j37kP5/245Q=; b=cUxJC0PDgtphFqteTncK1MMWPA0tvaGir94in2c2J8OS1SoLVX445ms53I6WXgvgZt omcPb99wACh3cWgOjXnBmpbdfO6TGUoKvW5FkIAjWnK2WaGBlkaGuWe3fil1eko5nnNz 0fEHHF+vbssaLNNJZQ4OERgj1TxSq9doOTw9wn6N/E1c9kvgMXSW8khp42cxCiEKOYZc YIT82Ffe2Dm5QVfgBXtzGRcUq66thL3sVDK8CB/y+caCXc3MflE+iN6IsmWSE3r9Wr0Q dmMPD1A8lozfusvXZFMUOYroviV/u8043Bi2YD3mEM6yJnOAy79Fj1HAx7kIXVvsPX6M uFLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ffJDakXCDHXSBit9htKSivl1vC+lnq1/j37kP5/245Q=; b=Fvf2xg7V87n9+aRZNQnSO5zjH7M3MadzbI4C14Pl3DgO3KN8QaQU4Xv2kroHgXbf2R uD6Ox3XEswAtrkPKU32XPuJndqBCJA6ICslmErCXcpoYJ3GlxBaUu50RmQpuvjHUkdIv FNcG5ET9Di+k5DQ9ellFwWvOCb4ENVrs3lfwYVnwVhmxsdk6HmjCsLGfEgc0W6dy1j+J LyiuWZ2xuaDtn5MYwnxdFQqnd1dgJwSKwHkzOL35izqtum2H0GSIRbh4zkrB3KmA9NdP IKMNK+06XfwETFlLKzXImiOT4hDKEc6g8Jjknj2gzHYF67oRJAbG0CmzYVEE+83MRk7D hgrQ== X-Gm-Message-State: AOAM532yMe3GaBoDTe4404y4eX0FM9XpHm/bjdIn0F9iOCH0JogVRLTj d6BjHjjj3IbuPR/d4SCSF1rMXg== X-Google-Smtp-Source: ABdhPJyZ047vfaPcWkl/qH5UFf8rFpsGc+XXjNkX+M8hjJ33pepyB/KnD8+n6in94OwlD32paX1+4g== X-Received: by 2002:a17:902:b782:b029:dc:251:423f with SMTP id e2-20020a170902b782b02900dc0251423fmr12395450pls.30.1610282492291; Sun, 10 Jan 2021 04:41:32 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id p9sm16176960pfq.136.2021.01.10.04.41.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Jan 2021 04:41:31 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , stable@vger.kernel.org Subject: [PATCH v3 2/6] mm: hugetlbfs: fix cannot migrate the fallocated HugeTLB page Date: Sun, 10 Jan 2021 20:40:13 +0800 Message-Id: <20210110124017.86750-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210110124017.86750-1-songmuchun@bytedance.com> References: <20210110124017.86750-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org If a new hugetlb page is allocated during fallocate it will not be marked as active (set_page_huge_active) which will result in a later isolate_huge_page failure when the page migration code would like to move that page. Such a failure would be unexpected and wrong. Only export set_page_huge_active, just leave clear_page_huge_active as static. Because there are no external users. Fixes: 70c3547e36f5 (hugetlbfs: add hugetlbfs_fallocate()) Signed-off-by: Muchun Song Cc: stable@vger.kernel.org Reviewed-by: Mike Kravetz Acked-by: Michal Hocko --- fs/hugetlbfs/inode.c | 3 ++- include/linux/hugetlb.h | 2 ++ mm/hugetlb.c | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index b5c109703daa..21c20fd5f9ee 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -735,9 +735,10 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, mutex_unlock(&hugetlb_fault_mutex_table[hash]); + set_page_huge_active(page); /* * unlock_page because locked by add_to_page_cache() - * page_put due to reference from alloc_huge_page() + * put_page() due to reference from alloc_huge_page() */ unlock_page(page); put_page(page); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ebca2ef02212..b5807f23caf8 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -770,6 +770,8 @@ static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, } #endif +void set_page_huge_active(struct page *page); + #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1f3bf1710b66..4741d60f8955 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1348,7 +1348,7 @@ bool page_huge_active(struct page *page) } /* never called for tail page */ -static void set_page_huge_active(struct page *page) +void set_page_huge_active(struct page *page) { VM_BUG_ON_PAGE(!PageHeadHuge(page), page); SetPagePrivate(&page[1]); From patchwork Sun Jan 10 12:40:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 360416 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95FD0C433E9 for ; Sun, 10 Jan 2021 12:42:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 608B222ADF for ; Sun, 10 Jan 2021 12:42:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726554AbhAJMmS (ORCPT ); Sun, 10 Jan 2021 07:42:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726069AbhAJMmQ (ORCPT ); Sun, 10 Jan 2021 07:42:16 -0500 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AF78C0617A4 for ; Sun, 10 Jan 2021 04:41:36 -0800 (PST) Received: by mail-pl1-x62d.google.com with SMTP id e2so8038606plt.12 for ; Sun, 10 Jan 2021 04:41:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I3gDuLjEjqpY5hJrgoP0zbm1oa1HSY0dQY30oFlgAYQ=; b=rWeFaHhdx26QIcu4oFq9jt4fdoiwTTEMHiX0GcDFnvXCNSEGXKNUOhSowItykhSP+s MU2lB7VGPKOpR26dagIJbSrZ0z4DQUseP8TI1Hr8218icPt2CE59nrthBPoGc5+2L0py GldzLHOZ/qQ22eoxfirNB/sX7sTFkGnYQS4NQwdHXLZ95tD8fo3K17CN+WvUYYuirXhr TpF+IMj8yyO/hRNwe/nRuTb4oq+CLbVTwC4ZMsGRxI8qPFxdbhi6TNcVuZv0oM+ZLqNg 4jK9h5RHhQ6APoWqIDlwAfx9/2k67z/4jNjM/HkPqSz2S5XOK+/rLaWpDAKaMrYgYVJT m2bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I3gDuLjEjqpY5hJrgoP0zbm1oa1HSY0dQY30oFlgAYQ=; b=DFiTh2E7nkdXmDmlDqLZj4X1S6ImZnn8PkF0I1LB8jp37T3OatLeI9G5j5LX99uvTm pkKfWgNY6N3QQEPK/oIlSmLMvyMqfa6GQ4YcYAPTyHoAlYeqxC1kr2aEXzhmcLYPI3Vc 2YZM6YlCB2QjPopq/lTCn/aBo2K3g+sfGTb8dGfMTpSybxRRqupiMGXpu2yecCmahy3A M/fXrx+6RvH6TvM/b0RhTD/NUGP/Ne93Q0PaFaD+Dqo2wumEyMrmeRel8jOW/1q1rDR0 R6V9w40E2Sk3gpuhqkC0KSYndFfhdlpkreFJ1n4w35yWfWEe1UOFHDrNJ1/JkRGo4R5q u8VQ== X-Gm-Message-State: AOAM5321f0s1FaVarncYnj4ndv+UPnMAxVFytycjKBfgiEXTBQtsiGGx KPpu+uwEztl1z9fINUE8GyHpMA== X-Google-Smtp-Source: ABdhPJzWrZDYsSoIxdfRxNA612sIivazrBh4X8Socv1b7/yltfVgFXyC4/ghwU7KuryYYJvTKH3CpQ== X-Received: by 2002:a17:90a:4402:: with SMTP id s2mr12942473pjg.37.1610282495977; Sun, 10 Jan 2021 04:41:35 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id p9sm16176960pfq.136.2021.01.10.04.41.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Jan 2021 04:41:35 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , stable@vger.kernel.org Subject: [PATCH v3 3/6] mm: hugetlb: fix a race between freeing and dissolving the page Date: Sun, 10 Jan 2021 20:40:14 +0800 Message-Id: <20210110124017.86750-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210110124017.86750-1-songmuchun@bytedance.com> References: <20210110124017.86750-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org There is a race condition between __free_huge_page() and dissolve_free_huge_page(). CPU0: CPU1: // page_count(page) == 1 put_page(page) __free_huge_page(page) dissolve_free_huge_page(page) spin_lock(&hugetlb_lock) // PageHuge(page) && !page_count(page) update_and_free_page(page) // page is freed to the buddy spin_unlock(&hugetlb_lock) spin_lock(&hugetlb_lock) clear_page_huge_active(page) enqueue_huge_page(page) // It is wrong, the page is already freed spin_unlock(&hugetlb_lock) The race windows is between put_page() and dissolve_free_huge_page(). We should make sure that the page is already on the free list when it is dissolved. Fixes: c8721bbbdd36 ("mm: memory-hotplug: enable memory hotplug to handle hugepage") Signed-off-by: Muchun Song Cc: stable@vger.kernel.org Reviewed-by: Mike Kravetz --- mm/hugetlb.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4741d60f8955..4a9011e12175 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -79,6 +79,21 @@ DEFINE_SPINLOCK(hugetlb_lock); static int num_fault_mutexes; struct mutex *hugetlb_fault_mutex_table ____cacheline_aligned_in_smp; +static inline bool PageHugeFreed(struct page *head) +{ + return page_private(head + 4) == -1UL; +} + +static inline void SetPageHugeFreed(struct page *head) +{ + set_page_private(head + 4, -1UL); +} + +static inline void ClearPageHugeFreed(struct page *head) +{ + set_page_private(head + 4, 0); +} + /* Forward declaration */ static int hugetlb_acct_memory(struct hstate *h, long delta); @@ -1028,6 +1043,7 @@ static void enqueue_huge_page(struct hstate *h, struct page *page) list_move(&page->lru, &h->hugepage_freelists[nid]); h->free_huge_pages++; h->free_huge_pages_node[nid]++; + SetPageHugeFreed(page); } static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) @@ -1044,6 +1060,7 @@ static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) list_move(&page->lru, &h->hugepage_activelist); set_page_refcounted(page); + ClearPageHugeFreed(page); h->free_huge_pages--; h->free_huge_pages_node[nid]--; return page; @@ -1504,6 +1521,7 @@ static void prep_new_huge_page(struct hstate *h, struct page *page, int nid) spin_lock(&hugetlb_lock); h->nr_huge_pages++; h->nr_huge_pages_node[nid]++; + ClearPageHugeFreed(page); spin_unlock(&hugetlb_lock); } @@ -1770,6 +1788,14 @@ int dissolve_free_huge_page(struct page *page) int nid = page_to_nid(head); if (h->free_huge_pages - h->resv_huge_pages == 0) goto out; + + /* + * We should make sure that the page is already on the free list + * when it is dissolved. + */ + if (unlikely(!PageHugeFreed(head))) + goto out; + /* * Move PageHWPoison flag from head page to the raw error page, * which makes any subpages rather than the error page reusable. From patchwork Sun Jan 10 12:40:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 360415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11297C4332B for ; Sun, 10 Jan 2021 12:43:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB6A122B45 for ; Sun, 10 Jan 2021 12:43:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726660AbhAJMmv (ORCPT ); Sun, 10 Jan 2021 07:42:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726655AbhAJMmu (ORCPT ); Sun, 10 Jan 2021 07:42:50 -0500 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0236C0617A9 for ; Sun, 10 Jan 2021 04:41:43 -0800 (PST) Received: by mail-pf1-x42e.google.com with SMTP id h186so9321373pfe.0 for ; Sun, 10 Jan 2021 04:41:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9zrsneMtVXeg4b61BTwnRw2mnUrcLzSuxYCbbaoWav8=; b=ECYcoVG48fFW0nt6MIsiwAkaHi2G4EDJ1+2KCQwkSOhBIwXu3PyJGG87/xSzWSefKi GMu2prQfnPbOkZ7nBodB5rtI7PrCFTSTz91iSsDjCafbMs3bC8p9jygCAXY4MntDNnf+ w5mjBkib5igUCLB+g9Qbn6YwzpJ/Y3BBS+snBV5Yg2rKIUa+6Y8dSR6qGkKVUN67djYu U5nByE8TlftiWoYt4e0/KTKQDllHu8OY1XLX48S87Wt5Y07PqlxC0Xkub4H2Dwph6M3E w+3M9JgCl6t+/ClUI4RwKPrtcCU+taxqJgsjAi9e1bwMv328pODcFhXWYltAsR+3G9sB +phQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9zrsneMtVXeg4b61BTwnRw2mnUrcLzSuxYCbbaoWav8=; b=HLqmxDlK/tASwpINc4tGbCyeIL1TZS68rniAykR5YI3MGO8mQfDNmsAZC4l5IYxB0U 9bUl+BUiquEctz0q/Cw+nccHhRGhVtdpfEuXlu68k6DD279uOws/H5VQGHRWU3PQVFiY Lz06lcM4zk7NtJemhoLupJeqGAdL/3pXmFb7zv9wEynPcaViecm4dfCMCbOnZccK0itF x0cbpFi9TmkdfIPqmN3JPBKy/ZLKxLqW2aGjilo7sAioMBmHKcWXx1VKx0Odu5PJUbfl l2on/wfTsrspBBtfcjfV1kHzjtR62Sm64a9iU17/8faISvwA4vfGMwJZVKuEflRyibNs 33fg== X-Gm-Message-State: AOAM533nLOIZ7v2NG2frGfXStBx7NBQomrm069oT8eLtZX72xPDJANrR QrIMw3SB07284yY3CrJJNiICzw== X-Google-Smtp-Source: ABdhPJz8VammpoTwnjyKnNe7opIwIAMtj9XxNyBPNM5DGwpJ+k8YsAYnqUEYhJWs/tGCae7TfB+toA== X-Received: by 2002:a63:ea01:: with SMTP id c1mr15452769pgi.138.1610282503368; Sun, 10 Jan 2021 04:41:43 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id p9sm16176960pfq.136.2021.01.10.04.41.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Jan 2021 04:41:42 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Michal Hocko , stable@vger.kernel.org Subject: [PATCH v3 5/6] mm: hugetlb: fix a race between isolating and freeing page Date: Sun, 10 Jan 2021 20:40:16 +0800 Message-Id: <20210110124017.86750-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210110124017.86750-1-songmuchun@bytedance.com> References: <20210110124017.86750-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org There is a race between isolate_huge_page() and __free_huge_page(). CPU0: CPU1: if (PageHuge(page)) put_page(page) __free_huge_page(page) spin_lock(&hugetlb_lock) update_and_free_page(page) set_compound_page_dtor(page, NULL_COMPOUND_DTOR) spin_unlock(&hugetlb_lock) isolate_huge_page(page) // trigger BUG_ON VM_BUG_ON_PAGE(!PageHead(page), page) spin_lock(&hugetlb_lock) page_huge_active(page) // trigger BUG_ON VM_BUG_ON_PAGE(!PageHuge(page), page) spin_unlock(&hugetlb_lock) When we isolate a HugeTLB page on CPU0. Meanwhile, we free it to the buddy allocator on CPU1. Then, we can trigger a BUG_ON on CPU0. Because it is already freed to the buddy allocator. Fixes: c8721bbbdd36 ("mm: memory-hotplug: enable memory hotplug to handle hugepage") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Acked-by: Michal Hocko Cc: stable@vger.kernel.org --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a176ceed55f1..e7ed30afbb8f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5575,9 +5575,9 @@ bool isolate_huge_page(struct page *page, struct list_head *list) { bool ret = true; - VM_BUG_ON_PAGE(!PageHead(page), page); spin_lock(&hugetlb_lock); - if (!page_huge_active(page) || !get_page_unless_zero(page)) { + if (!PageHeadHuge(page) || !page_huge_active(page) || + !get_page_unless_zero(page)) { ret = false; goto unlock; } From patchwork Sun Jan 10 12:40:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 361128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9632C433E9 for ; Sun, 10 Jan 2021 12:43:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF13922ADF for ; Sun, 10 Jan 2021 12:43:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726691AbhAJMmz (ORCPT ); Sun, 10 Jan 2021 07:42:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726686AbhAJMmx (ORCPT ); Sun, 10 Jan 2021 07:42:53 -0500 Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4A2DC0617AB for ; Sun, 10 Jan 2021 04:41:47 -0800 (PST) Received: by mail-pl1-x635.google.com with SMTP id x12so8050097plr.10 for ; Sun, 10 Jan 2021 04:41:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Q2v5mrrSjESraNqHDwGVpPTSDpsByCOvtLmUTzZt6g0=; b=YVg4gth6O1BgEX5TXwXspQm27uCViGy7IU0/RmDe2i8CR9p4G0fXQW217eoFau69+I bcl/d5PDS+T1DBTddzhv7AVOPPvxtHJuE4NLYnjqVW/0n5js6XOdAT8nIdO3C2dc8emE O9WiwNqcJ6vM1ZfRlDiOocrbOoBMp2DVvN5kIPA6BATPRJ/CcI24qXY+NoBib7jrcsVA ap5aPeNdarr1Mp7cskUk3Lym7v7p0xERycNHqBeTH8TJsXETmlV23rndVerXw3okmV7O wvHn4OVoHXtrtEjhk/2bjHta7Ae9R3C8ZittrFvRrBGa9EvFuTjPnot7FHpfbZgaSWLf Jpng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q2v5mrrSjESraNqHDwGVpPTSDpsByCOvtLmUTzZt6g0=; b=DdS9UzKlsBPywlYYItaMQW1DBauq/FQaeYaQ8Mhp087ASZMbTZ+a4tcisMh88STVhE XgBmG06/nv0sGFOzJNnvCWjxQ8IGdtfEaN0obZ7ImP2ZcWtXT+H3XDO+umEktuR+OFZG VuRlr4nJHuvdFi/Ja7XVZ94zHndswf+ZvB8pchSdPKjHcCqjCTNq8MoQn7dT3cDCV6cJ 0HSB65ZBX+u5FXjkOHra9ZKsG7fgOaAdfhiDidklPrhA4DhwzC0O08Ah3dGHyLjEIRVw 8qeLBgqeDoV7Rmp1AeQHlzk496teAjNWUxYFjyhp5eSWhr/uthLJ/cj0pOy9HrIgoTla 6H0A== X-Gm-Message-State: AOAM531lurXvObrNDzPXyIU1p5hU8egY6T9hLo1iZ0zzr4oIQ+yBsONr pgTd8RAMSyF13MUKcIvbTKlV9A== X-Google-Smtp-Source: ABdhPJzl17pIIRw/wPQcY4D0IaWNRYplQXvDOcoTJCykKdxrbXO1sPO/EXGH5SNXKhzi6llVHPNJ5g== X-Received: by 2002:a17:90a:cb84:: with SMTP id a4mr13213004pju.50.1610282507382; Sun, 10 Jan 2021 04:41:47 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id p9sm16176960pfq.136.2021.01.10.04.41.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Jan 2021 04:41:46 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Michal Hocko , stable@vger.kernel.org Subject: [PATCH v3 6/6] mm: hugetlb: remove VM_BUG_ON_PAGE from page_huge_active Date: Sun, 10 Jan 2021 20:40:17 +0800 Message-Id: <20210110124017.86750-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210110124017.86750-1-songmuchun@bytedance.com> References: <20210110124017.86750-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org The page_huge_active() can be called from scan_movable_pages() which do not hold a reference count to the HugeTLB page. So when we call page_huge_active() from scan_movable_pages(), the HugeTLB page can be freed parallel. Then we will trigger a BUG_ON which is in the page_huge_active() when CONFIG_DEBUG_VM is enabled. Just remove the VM_BUG_ON_PAGE. Fixes: 7e1f049efb86 ("mm: hugetlb: cleanup using paeg_huge_active()") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Acked-by: Michal Hocko Cc: stable@vger.kernel.org --- mm/hugetlb.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index e7ed30afbb8f..5940bf0c49b9 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1360,8 +1360,7 @@ struct hstate *size_to_hstate(unsigned long size) */ bool page_huge_active(struct page *page) { - VM_BUG_ON_PAGE(!PageHuge(page), page); - return PageHead(page) && PagePrivate(&page[1]); + return PageHeadHuge(page) && PagePrivate(&page[1]); } /* never called for tail page */