From patchwork Thu Aug 22 12:39:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 822379 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD748181B87; Thu, 22 Aug 2024 12:40:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330410; cv=none; b=g6rnf/aNP0iiFAL+wWNSUz4D3X7RZTjiTuMEgLORuyjzWwoxpW69AX7TmYoq3J7aBD38f5hFGpnu/ulpdClRx29e2VnH+kmXNXL79Fbs+UfzkUkVsW/L8WTt+cHOMKpOsgu6/quMyoCULulGSWPgRLOfHkgKFs6A0t0I3bfKa/c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330410; c=relaxed/simple; bh=o8GaH3tn8yQHjvX1TbIRinqknzeRTVHuSafiEciVSrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IOkP4eXZE1dREZVSv6Le4dAyziTmoy977JMcr7CojjgxoKtSjY7j3SNUpGNgVAmta3L4aWPD+x9sZPKX0YFX5XyLuGBMhxsrw+Xi7ds4Ed9S66RUrrwdWX07Cag+vXEIutAdV/PCL+mBMKhzo5orCtnW57m+z+DNnxpzzsSGveg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FdN4PAxV; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FdN4PAxV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724330409; x=1755866409; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=o8GaH3tn8yQHjvX1TbIRinqknzeRTVHuSafiEciVSrc=; b=FdN4PAxVJHTdbxPXevO7D7Yv0eJHOg8Cvej6JLuQtWRWxHihPyG1KewW 3qboVo/nhovJ6l0v2W7VJQXtsJVoIdIc26U+HefLKRpJbmNzly3tLFvJ9 o2IGQocBqCJ7WyBLeXIBK0yGhfvXpuPV0+edDxzVy73nbvF1jLfv9175m 8/hN8rp5HHuCbeEGycPUMDGCZfC+S1L5ZJT+p4NLX37lZtEKXzNna+Oo5 6fogC3HnXwwxtkULhlteXyyrL34wYg+UdwRzghNbrMkUJXTmwCfuw1rKh GnxI4fL0zN3WT9Qy7hLBZ9ib2ZcUThri+Xq09PHqKKmlngW8VqqA1sBxb w==; X-CSE-ConnectionGUID: tx5Nl5WYRqKkyblA150ZrQ== X-CSE-MsgGUID: G09r91iOR1OOPeMvyZY6cQ== X-IronPort-AV: E=McAfee;i="6700,10204,11172"; a="25635669" X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="25635669" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2024 05:40:08 -0700 X-CSE-ConnectionGUID: N+QYkLQWRQy3O0wBzY/BNA== X-CSE-MsgGUID: 9JTr/ASpSguPvM1WMUyHYA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="61443076" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa008.fm.intel.com with ESMTP; 22 Aug 2024 05:40:03 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 17D592B3; Thu, 22 Aug 2024 15:40:02 +0300 (EEST) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , "Rafael J. Wysocki" , Andy Lutomirski , Peter Zijlstra , Baoquan He Cc: Ard Biesheuvel , Tom Lendacky , Andrew Morton , Thomas Zimmermann , Sean Christopherson , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, "Kirill A. Shutemov" , Kai Huang Subject: [PATCHv4 1/4] x86/mm/ident_map: Fix virtual address wrap to zero Date: Thu, 22 Aug 2024 15:39:57 +0300 Message-ID: <20240822124000.1171321-2-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> References: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Calculation of 'next' virtual address doesn't protect against wrapping to zero. It can result in page table corruption and hang. The problematic case is possible if user sets high x86_mapping_info::offset. The wrapping to zero only occurs if the top PGD entry is accessed. There are no such users in the upstream. Only hibernate_64.c uses x86_mapping_info::offset, and it operates on the direct mapping range, which is not the top PGD entry. Replace manual 'next' calculation with p?d_addr_end() which handles wrapping correctly. Signed-off-by: Kirill A. Shutemov Reviewed-by: Kai Huang Reviewed-by: Tom Lendacky --- arch/x86/mm/ident_map.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/arch/x86/mm/ident_map.c b/arch/x86/mm/ident_map.c index 437e96fb4977..5872f3ee863c 100644 --- a/arch/x86/mm/ident_map.c +++ b/arch/x86/mm/ident_map.c @@ -101,9 +101,7 @@ static int ident_pud_init(struct x86_mapping_info *info, pud_t *pud_page, pmd_t *pmd; bool use_gbpage; - next = (addr & PUD_MASK) + PUD_SIZE; - if (next > end) - next = end; + next = pud_addr_end(addr, end); /* if this is already a gbpage, this portion is already mapped */ if (pud_leaf(*pud)) @@ -154,10 +152,7 @@ static int ident_p4d_init(struct x86_mapping_info *info, p4d_t *p4d_page, p4d_t *p4d = p4d_page + p4d_index(addr); pud_t *pud; - next = (addr & P4D_MASK) + P4D_SIZE; - if (next > end) - next = end; - + next = p4d_addr_end(addr, end); if (p4d_present(*p4d)) { pud = pud_offset(p4d, 0); result = ident_pud_init(info, pud, addr, next); @@ -199,10 +194,7 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page, pgd_t *pgd = pgd_page + pgd_index(addr); p4d_t *p4d; - next = (addr & PGDIR_MASK) + PGDIR_SIZE; - if (next > end) - next = end; - + next = pgd_addr_end(addr, end); if (pgd_present(*pgd)) { p4d = p4d_offset(pgd, 0); result = ident_p4d_init(info, p4d, addr, next); From patchwork Thu Aug 22 12:39:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 822378 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F36A1C9DCC; Thu, 22 Aug 2024 12:40:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330411; cv=none; b=UH7K2LkdqZi9Zk6kWr9BHjJC28ZkrauyfCEJkgvFVqHiUMdi3j/4Dz61Rxec+xJ1HSYCTfZhUsxKlZgxR+k8+CsVdJxU1Ip8ROLSZlw6K1mfIxn93hZ5lEHCt/hqPAhYV4dwFaOFGdlnBRoujoqzpjmQp0YsaD1Xmn9hOZRgsaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330411; c=relaxed/simple; bh=vy3STgE3PjnKqGO0pLDMgzAb6NmQRTblv+6S2nmqiFo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j2feco1MYCVlrOmG9kQAjhdI/dIZRnimysAL15EEMjHx7MD+wMxjn+GGLU5UYDBfEh71iX2l3N1knAuJdc1uFw4kIWdNKiaqZugX8Mpw3ka3sHvsnmLnpS9DIgzDEkavhMQX7EOANpmYxsPwivODyPMK8bNAwhdKIcCGrSdHdVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AURBxX6P; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AURBxX6P" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724330410; x=1755866410; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vy3STgE3PjnKqGO0pLDMgzAb6NmQRTblv+6S2nmqiFo=; b=AURBxX6PjtGQpJ0Ld3B48a81VnBHlCAwF5dBgXq5LY80ERlI1uv5kSP+ zUNbl1UMvHSB6/M16e6ml5R3XtC12mXS1iuiMmsKVOrHcHTNIEWDzLU2W 6PfW9/FpwciLCV4tEs0YeRuDY7mUUUdV96C25DAt6e/OVc32nBZ/eIEcl qGKdkWl7CFZiaX7I1k4gt82koKy4YBv+veFnh8EgxDz6BZx48ty+WTwkM 3EQWbyAoSSzlsdkNa4lQYXM2kk2AlFN88hZIdmBpByWAhGbwKIDQIc6AX ARcJYRUP2yqGbl0FMShHJYfh6AB4YUUIdhGwN7s1Qu/W1mL0/2CtVmh6F g==; X-CSE-ConnectionGUID: tdSgkcPPReKMiqP5+aXhXg== X-CSE-MsgGUID: BU1PtdsvSjitFnEz1hRuPQ== X-IronPort-AV: E=McAfee;i="6700,10204,11172"; a="22554218" X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="22554218" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2024 05:40:07 -0700 X-CSE-ConnectionGUID: DGNdl5xGTEmpkwyYoIpGgA== X-CSE-MsgGUID: gmvAVoE3Qp6CdEb7/7tIEg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="66320995" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa005.jf.intel.com with ESMTP; 22 Aug 2024 05:40:04 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 29DF2331; Thu, 22 Aug 2024 15:40:02 +0300 (EEST) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , "Rafael J. Wysocki" , Andy Lutomirski , Peter Zijlstra , Baoquan He Cc: Ard Biesheuvel , Tom Lendacky , Andrew Morton , Thomas Zimmermann , Sean Christopherson , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, "Kirill A. Shutemov" , Kai Huang Subject: [PATCHv4 2/4] x86/acpi: Replace manual page table initialization with kernel_ident_mapping_init() Date: Thu, 22 Aug 2024 15:39:58 +0300 Message-ID: <20240822124000.1171321-3-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> References: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The function init_transition_pgtable() maps the page with asm_acpi_mp_play_dead() into an identity mapping. Replace manual page table initialization with kernel_ident_mapping_init() to avoid code duplication. Use x86_mapping_info::offset to get the page mapped at the correct location. Signed-off-by: Kirill A. Shutemov Reviewed-by: Kai Huang Reviewed-by: Tom Lendacky --- arch/x86/kernel/acpi/madt_wakeup.c | 73 ++++++------------------------ 1 file changed, 15 insertions(+), 58 deletions(-) diff --git a/arch/x86/kernel/acpi/madt_wakeup.c b/arch/x86/kernel/acpi/madt_wakeup.c index d5ef6215583b..f36f28405dcc 100644 --- a/arch/x86/kernel/acpi/madt_wakeup.c +++ b/arch/x86/kernel/acpi/madt_wakeup.c @@ -70,58 +70,6 @@ static void __init free_pgt_page(void *pgt, void *dummy) return memblock_free(pgt, PAGE_SIZE); } -/* - * Make sure asm_acpi_mp_play_dead() is present in the identity mapping at - * the same place as in the kernel page tables. asm_acpi_mp_play_dead() switches - * to the identity mapping and the function has be present at the same spot in - * the virtual address space before and after switching page tables. - */ -static int __init init_transition_pgtable(pgd_t *pgd) -{ - pgprot_t prot = PAGE_KERNEL_EXEC_NOENC; - unsigned long vaddr, paddr; - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - vaddr = (unsigned long)asm_acpi_mp_play_dead; - pgd += pgd_index(vaddr); - if (!pgd_present(*pgd)) { - p4d = (p4d_t *)alloc_pgt_page(NULL); - if (!p4d) - return -ENOMEM; - set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE)); - } - p4d = p4d_offset(pgd, vaddr); - if (!p4d_present(*p4d)) { - pud = (pud_t *)alloc_pgt_page(NULL); - if (!pud) - return -ENOMEM; - set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE)); - } - pud = pud_offset(p4d, vaddr); - if (!pud_present(*pud)) { - pmd = (pmd_t *)alloc_pgt_page(NULL); - if (!pmd) - return -ENOMEM; - set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE)); - } - pmd = pmd_offset(pud, vaddr); - if (!pmd_present(*pmd)) { - pte = (pte_t *)alloc_pgt_page(NULL); - if (!pte) - return -ENOMEM; - set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE)); - } - pte = pte_offset_kernel(pmd, vaddr); - - paddr = __pa(vaddr); - set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, prot)); - - return 0; -} - static int __init acpi_mp_setup_reset(u64 reset_vector) { struct x86_mapping_info info = { @@ -130,6 +78,7 @@ static int __init acpi_mp_setup_reset(u64 reset_vector) .page_flag = __PAGE_KERNEL_LARGE_EXEC, .kernpg_flag = _KERNPG_TABLE_NOENC, }; + unsigned long mstart, mend; pgd_t *pgd; pgd = alloc_pgt_page(NULL); @@ -137,8 +86,6 @@ static int __init acpi_mp_setup_reset(u64 reset_vector) return -ENOMEM; for (int i = 0; i < nr_pfn_mapped; i++) { - unsigned long mstart, mend; - mstart = pfn_mapped[i].start << PAGE_SHIFT; mend = pfn_mapped[i].end << PAGE_SHIFT; if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) { @@ -147,14 +94,24 @@ static int __init acpi_mp_setup_reset(u64 reset_vector) } } - if (kernel_ident_mapping_init(&info, pgd, - PAGE_ALIGN_DOWN(reset_vector), - PAGE_ALIGN(reset_vector + 1))) { + mstart = PAGE_ALIGN_DOWN(reset_vector); + mend = mstart + PAGE_SIZE; + if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) { kernel_ident_mapping_free(&info, pgd); return -ENOMEM; } - if (init_transition_pgtable(pgd)) { + /* + * Make sure asm_acpi_mp_play_dead() is present in the identity mapping + * at the same place as in the kernel page tables. + * asm_acpi_mp_play_dead() switches to the identity mapping and the + * function must be present at the same spot in the virtual address space + * before and after switching page tables. + */ + info.offset = __START_KERNEL_map - phys_base; + mstart = PAGE_ALIGN_DOWN(__pa(asm_acpi_mp_play_dead)); + mend = mstart + PAGE_SIZE; + if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) { kernel_ident_mapping_free(&info, pgd); return -ENOMEM; } From patchwork Thu Aug 22 12:39:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 821365 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A26741C9452; Thu, 22 Aug 2024 12:40:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330411; cv=none; b=noWqGY65ogSAR/NL3SvL1fMQbMCLi3GzruIYvHU91G4Z+RhaWtrE29VrLC2A97crr8+wbldEqDix5nBUo41/xbeOmmjn4GRiGtbuLiP8qaoKDNJko/GTdd/9OJki2x3cHSA1b2I8hobVB/s7jC9qjRSQkWYdDncesQGhR6JPvLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330411; c=relaxed/simple; bh=QQ0UbpyqdIB6+YfJjPDDx6eJHiL0EplzplL8JUqBi1Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P69T4lUkHpl+CtRZhuI3QgEAW+lTObbS8Ju3eDgt3jk4PDEgzXu8zjuXs2cN3E1/g5rqEYl5wL0WSvm9GSvIdY5LibUaQqWVt2tosqJpe+qq5C2XV5Gfgpb2rlFPOLGrmtq/UB5OND/b/BSzFPWD2LGUIXbcGBt9fkkp3t3GFFU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mM6FDNtr; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mM6FDNtr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724330410; x=1755866410; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QQ0UbpyqdIB6+YfJjPDDx6eJHiL0EplzplL8JUqBi1Y=; b=mM6FDNtrX3g2cWoEB2Y6UX3PpTnvIRjNdJay5l2L7X+HDbxPv/Hwt3hE kWKIxpd1g7p6Q2tC5KSqJPwaDcQEWSX9iZxDhhXPJ6bXl2ynoiriwDt1z ug9REs27zSJTTwgq1L6HUKFm77pjxaE/IoS0IyqO2WFTYSrqFCdADOZEM GNnURKOM6wKWz24lXkZ8oo15dFhZHFmsKKtNqevsYKXf750VANmw7UJAh FSSHx2pPt3POME32lOKY7dYXd5VtGLRzB2l2gKBiOV4kb5qs2opf4MKYz fmBonY0EtMyck1TiZFO5A5cRWf19eoLxbolpn5yeQEgJtYqmkrRVG5Nnw g==; X-CSE-ConnectionGUID: aBfRq49DTcGhxyoeuojZeA== X-CSE-MsgGUID: EyWXIhf/R5+0QzxnGDBKnQ== X-IronPort-AV: E=McAfee;i="6700,10204,11172"; a="25635649" X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="25635649" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2024 05:40:07 -0700 X-CSE-ConnectionGUID: /8lyXqRMRoWLnT3KU/uRSg== X-CSE-MsgGUID: T6DzAN3XT52XnPEN5X1CJg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="61443073" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa008.fm.intel.com with ESMTP; 22 Aug 2024 05:40:03 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 3FC324D7; Thu, 22 Aug 2024 15:40:02 +0300 (EEST) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , "Rafael J. Wysocki" , Andy Lutomirski , Peter Zijlstra , Baoquan He Cc: Ard Biesheuvel , Tom Lendacky , Andrew Morton , Thomas Zimmermann , Sean Christopherson , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, "Kirill A. Shutemov" , Kai Huang Subject: [PATCHv4 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable() Date: Thu, 22 Aug 2024 15:39:59 +0300 Message-ID: <20240822124000.1171321-4-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> References: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The init_transition_pgtable() function sets up transitional page tables. It ensures that the relocate_kernel() function is present in the identity mapping at the same location as in the kernel page tables. relocate_kernel() switches to the identity mapping, and the function must be present at the same location in the virtual address space before and after switching page tables. init_transition_pgtable() maps a copy of relocate_kernel() in image->control_code_page at the relocate_kernel() virtual address, but the original physical address of relocate_kernel() would also work. It is safe to use original relocate_kernel() physical address cannot be overwritten until swap_pages() is called, and the relocate_kernel() virtual address will not be used by then. Map the original relocate_kernel() at the relocate_kernel() virtual address in the identity mapping. It is preparation to replace the init_transition_pgtable() implementation with a call to kernel_ident_mapping_init(). Note that while relocate_kernel() switches to the identity mapping, it does not flush global TLB entries (CR4.PGE is not cleared). This means that in most cases, the kernel still runs relocate_kernel() from the original physical address before the change. Signed-off-by: Kirill A. Shutemov Reviewed-by: Kai Huang --- arch/x86/kernel/machine_kexec_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c index 9c9ac606893e..645690e81c2d 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -157,7 +157,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd) pte_t *pte; vaddr = (unsigned long)relocate_kernel; - paddr = __pa(page_address(image->control_code_page)+PAGE_SIZE); + paddr = __pa(relocate_kernel); pgd += pgd_index(vaddr); if (!pgd_present(*pgd)) { p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL); From patchwork Thu Aug 22 12:40:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 821366 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60423146A6E; Thu, 22 Aug 2024 12:40:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330410; cv=none; b=cTn5/rfXVKMXIHmdNcFsLvkVsQG1/6GSFL09BKIOufNJ7Gx3n0W+rPJIXW0OVWkyrVkO0jfkAckoJKZSjawVG55TlckeIOePvMro5CmQiPxfilFO1keFQkxwLS220XrFr4GBJLKUVxS/ibYQJPaw0/zBaUh9rArq+VjC5tpHFyc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724330410; c=relaxed/simple; bh=UJSYe9Co8WtZ1/D2W+TpdPAS7LIZhmBIyvDdCVYV3gI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gwpymHgB/oxkBGrqjMVQvzoIQe6+lY9UhU0P0NageNl1i6G8Y/EVxSB/Vn49Q+lq7RUAY9bbNnfdV5u9BKYoUb3ULH+5Lk7ojGe+7/a27AuFsApdE7+8xn3DCx5IeMar6dxJ+Bs7lLq/64iMazqUaQBzNTRCBprS9i9vmhyhZC8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=fOGmcJ0y; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="fOGmcJ0y" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724330408; x=1755866408; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UJSYe9Co8WtZ1/D2W+TpdPAS7LIZhmBIyvDdCVYV3gI=; b=fOGmcJ0y4eJqAUdG0ub3TAHKi5ZrnlxF0b/KlL06c8b71mJnwqdiKqRD q6EZ5YIDSCMpNYvog3PiLqoRTpmmdyxMlQnqO1Vz8W2v3PDqonXWO4O7c VNhNJMXhVqJvpFzhLptMDRGF7CBlXBr5YXH18DWIro1rE3N/TdmX7yAyw cV26etSBFMjiI0ABbT0IGahKEsVPTpdoxWif7e2i0CpubOOK6KZliFvPc NHf5SRnSaCqdjCc2hqdwem2ctgWQMXdNJjCFsgv6/4hZzQWLHOjIXm6A4 3hzGMdDBMq0FpPZzq/9sHXROBmAotD6RzFlRlmVIKyAFzEwvg/20wwS76 g==; X-CSE-ConnectionGUID: 7TaaYo0CQCKVRgR4XbSIFQ== X-CSE-MsgGUID: BKkteXmSRPqtrpnNC2SsgQ== X-IronPort-AV: E=McAfee;i="6700,10204,11172"; a="22554207" X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="22554207" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2024 05:40:07 -0700 X-CSE-ConnectionGUID: +vB+UJC6SUiseNdT72z9Ew== X-CSE-MsgGUID: PueyqbE5TUmfgVndqRXc+A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,167,1719903600"; d="scan'208";a="66320992" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa005.jf.intel.com with ESMTP; 22 Aug 2024 05:40:04 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 5035B6CC; Thu, 22 Aug 2024 15:40:02 +0300 (EEST) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , "Rafael J. Wysocki" , Andy Lutomirski , Peter Zijlstra , Baoquan He Cc: Ard Biesheuvel , Tom Lendacky , Andrew Morton , Thomas Zimmermann , Sean Christopherson , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, "Kirill A. Shutemov" , Kai Huang Subject: [PATCHv4 4/4] x86/64/kexec: Rewrite init_transition_pgtable() with kernel_ident_mapping_init() Date: Thu, 22 Aug 2024 15:40:00 +0300 Message-ID: <20240822124000.1171321-5-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> References: <20240822124000.1171321-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 init_transition_pgtable() sets up transitional page tables. Rewrite it using kernel_ident_mapping_init() to avoid code duplication. Change struct kimage_arch to track allocated page tables as a list, not linking them to specific page table levels. Signed-off-by: Kirill A. Shutemov Reviewed-by: Tom Lendacky Reviewed-by: Kai Huang --- arch/x86/include/asm/kexec.h | 5 +- arch/x86/kernel/machine_kexec_64.c | 89 +++++++++++------------------- 2 files changed, 32 insertions(+), 62 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index ae5482a2f0ca..7f9287f371e6 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -145,10 +145,7 @@ struct kimage_arch { }; #else struct kimage_arch { - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; + struct list_head pages; }; #endif /* CONFIG_X86_32 */ diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c index 645690e81c2d..fb350372835c 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -134,71 +134,42 @@ map_efi_systab(struct x86_mapping_info *info, pgd_t *level4p) return 0; } +static void *alloc_transition_pgt_page(void *data) +{ + struct kimage *image = (struct kimage *)data; + unsigned long virt; + + virt = get_zeroed_page(GFP_KERNEL); + if (!virt) + return NULL; + + list_add(&virt_to_page(virt)->lru, &image->arch.pages); + return (void *)virt; +} + static void free_transition_pgtable(struct kimage *image) { - free_page((unsigned long)image->arch.p4d); - image->arch.p4d = NULL; - free_page((unsigned long)image->arch.pud); - image->arch.pud = NULL; - free_page((unsigned long)image->arch.pmd); - image->arch.pmd = NULL; - free_page((unsigned long)image->arch.pte); - image->arch.pte = NULL; + struct page *page, *tmp; + + list_for_each_entry_safe(page, tmp, &image->arch.pages, lru) { + list_del(&page->lru); + free_page((unsigned long)page_address(page)); + } } static int init_transition_pgtable(struct kimage *image, pgd_t *pgd) { - pgprot_t prot = PAGE_KERNEL_EXEC_NOENC; - unsigned long vaddr, paddr; - int result = -ENOMEM; - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; + struct x86_mapping_info info = { + .alloc_pgt_page = alloc_transition_pgt_page, + .context = image, + .page_flag = __PAGE_KERNEL_LARGE_EXEC, + .kernpg_flag = _KERNPG_TABLE_NOENC, + .offset = __START_KERNEL_map - phys_base, + }; + unsigned long mstart = PAGE_ALIGN_DOWN(__pa(relocate_kernel)); + unsigned long mend = mstart + PAGE_SIZE; - vaddr = (unsigned long)relocate_kernel; - paddr = __pa(relocate_kernel); - pgd += pgd_index(vaddr); - if (!pgd_present(*pgd)) { - p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL); - if (!p4d) - goto err; - image->arch.p4d = p4d; - set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE)); - } - p4d = p4d_offset(pgd, vaddr); - if (!p4d_present(*p4d)) { - pud = (pud_t *)get_zeroed_page(GFP_KERNEL); - if (!pud) - goto err; - image->arch.pud = pud; - set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE)); - } - pud = pud_offset(p4d, vaddr); - if (!pud_present(*pud)) { - pmd = (pmd_t *)get_zeroed_page(GFP_KERNEL); - if (!pmd) - goto err; - image->arch.pmd = pmd; - set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE)); - } - pmd = pmd_offset(pud, vaddr); - if (!pmd_present(*pmd)) { - pte = (pte_t *)get_zeroed_page(GFP_KERNEL); - if (!pte) - goto err; - image->arch.pte = pte; - set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE)); - } - pte = pte_offset_kernel(pmd, vaddr); - - if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) - prot = PAGE_KERNEL_EXEC; - - set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, prot)); - return 0; -err: - return result; + return kernel_ident_mapping_init(&info, pgd, mstart, mend); } static void *alloc_pgt_page(void *data) @@ -299,6 +270,8 @@ int machine_kexec_prepare(struct kimage *image) unsigned long start_pgtable; int result; + INIT_LIST_HEAD(&image->arch.pages); + /* Calculate the offsets */ start_pgtable = page_to_pfn(image->control_code_page) << PAGE_SHIFT;