From patchwork Tue May 26 18:52:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kroah-Hartman X-Patchwork-Id: 225275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19CCEC433E1 for ; Tue, 26 May 2020 19:23:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EED6920849 for ; Tue, 26 May 2020 19:23:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1590521015; bh=8AJtd7XJLAdfUVy8MB+O0mDvZE4M9CTw6DQcKauIO0A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=wKrz+c7G9BAt1lcPL+SwtBS2/O7psofOCwgl9aGClA+LeLonkMViaLK/8iHpAxRZW Hl9aIvcPMYPYMRwvY8Wv9zuTUs14c7eYC5G1OVa5pD/PmZbIwZyutMb+EdsmFuo00k EdU/ov/qQxGwVrf/u6D9tXoLFDz72m5zHMEB8gSw= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391824AbgEZTJA (ORCPT ); Tue, 26 May 2020 15:09:00 -0400 Received: from mail.kernel.org ([198.145.29.99]:38042 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391819AbgEZTI7 (ORCPT ); Tue, 26 May 2020 15:08:59 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A4ED520776; Tue, 26 May 2020 19:08:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1590520139; bh=8AJtd7XJLAdfUVy8MB+O0mDvZE4M9CTw6DQcKauIO0A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pRe+mu3hMiWegLmKi/wDGhKHmJvzqZTc9HHNjJ/m4tEYuwj4DDLFfwz6ngeQr6y/3 VJC+8KBK3kGUnXTOZDXb1Bx5QSbfQHjoI84uu8+gyb9ob7EXycRJ7YEw9P4lkvvAgU 1Pw/lCt/9giUDMq+LbSejq09FOWxkJZjSU52lGUI= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Rick Edgecombe , "Peter Zijlstra (Intel)" , Sasha Levin Subject: [PATCH 5.4 033/111] x86/mm/cpa: Flush direct map alias during cpa Date: Tue, 26 May 2020 20:52:51 +0200 Message-Id: <20200526183935.966457158@linuxfoundation.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200526183932.245016380@linuxfoundation.org> References: <20200526183932.245016380@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Rick Edgecombe [ Upstream commit ab5130186d7476dcee0d4e787d19a521ca552ce9 ] As an optimization, cpa_flush() was changed to optionally only flush the range in @cpa if it was small enough. However, this range does not include any direct map aliases changed in cpa_process_alias(). So small set_memory_() calls that touch that alias don't get the direct map changes flushed. This situation can happen when the virtual address taking variants are passed an address in vmalloc or modules space. In these cases, force a full TLB flush. Note this issue does not extend to cases where the set_memory_() calls are passed a direct map address, or page array, etc, as the primary target. In those cases the direct map would be flushed. Fixes: 935f5839827e ("x86/mm/cpa: Optimize cpa_flush_array() TLB invalidation") Signed-off-by: Rick Edgecombe Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20200424105343.GA20730@hirez.programming.kicks-ass.net Signed-off-by: Sasha Levin --- arch/x86/mm/pageattr.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index a19a71b4d185..281e584cfe39 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -42,7 +42,8 @@ struct cpa_data { unsigned long pfn; unsigned int flags; unsigned int force_split : 1, - force_static_prot : 1; + force_static_prot : 1, + force_flush_all : 1; struct page **pages; }; @@ -352,10 +353,10 @@ static void cpa_flush(struct cpa_data *data, int cache) return; } - if (cpa->numpages <= tlb_single_page_flush_ceiling) - on_each_cpu(__cpa_flush_tlb, cpa, 1); - else + if (cpa->force_flush_all || cpa->numpages > tlb_single_page_flush_ceiling) flush_tlb_all(); + else + on_each_cpu(__cpa_flush_tlb, cpa, 1); if (!cache) return; @@ -1584,6 +1585,8 @@ static int cpa_process_alias(struct cpa_data *cpa) alias_cpa.flags &= ~(CPA_PAGES_ARRAY | CPA_ARRAY); alias_cpa.curpage = 0; + cpa->force_flush_all = 1; + ret = __change_page_attr_set_clr(&alias_cpa, 0); if (ret) return ret; @@ -1604,6 +1607,7 @@ static int cpa_process_alias(struct cpa_data *cpa) alias_cpa.flags &= ~(CPA_PAGES_ARRAY | CPA_ARRAY); alias_cpa.curpage = 0; + cpa->force_flush_all = 1; /* * The high mapping range is imprecise, so ignore the * return value.