From patchwork Wed Sep 25 15:01:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830544 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 542C977117 for ; Wed, 25 Sep 2024 15:01:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276518; cv=none; b=Mv5AzVzgt8tckZ7MkbQiZTdWChVYt2bNmgNQqbCj7A0be5nFS0q4VvvKkyTaloAURiLnCjBnKbo2wh4oaTZXTQZPFOpigT3aF6IXl1vMb9ZC7eWDfWutiAVM6DE/lkSRCx3D5Dj5wLOL1CystyqaKSqoclB7R9zO83XljVMlYKA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276518; c=relaxed/simple; bh=rk1TLausu3D2x4uz05lRhrdTMxiTSLA2fDl1meeIK+I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DDcoyVdkDrzQ/p5oNaRlqiFugSnysijWN1fJHM3a5GY/7wYRwU+2GqhZRw88uUkH5NCLhy6NMOFstDmC49VqnQgBYTGSry5qY7yL51sqv/xt9MDza85M7wWai0SqMQKwAR3YnE8iG4+mvWdYGjiNL98rEBfwkXsOIYr81h3zP1o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VFVz+AlT; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VFVz+AlT" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-42cb940cd67so63317415e9.0 for ; Wed, 25 Sep 2024 08:01:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276515; x=1727881315; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yjSReXj8XxN+J1aHSpg+bgY7XO8zJXJrh2uX1BeFRGQ=; b=VFVz+AlT0fVj8J3pb0uIJQLGxkYCKkAvLDgcAohPDUXmaMuk6UTbw4uq+CHcsEoRZ/ 5x5aSnW20fDUqHmuY3uMAA6I80/9rPDD/5xGciRxrWzTnMT2UtipActHUBezkRFpEjjj sOG059kJtCdOCKA5XBwzrZmIOIa5S/UmblboEdVQKaDLNiR6M6X2DIjGQ4+n7oN0yCPj sb6pNNvyVesjn3JTHhpQAoqB4ylt5fKdSAS0LSqxRycl+oVbZWRIh3MrNoaPcUJI0dRJ +onoD54UsA3PxnhIJe/kNKf8LKnhjzLE+eMh0qFTJDtNpVvQBt0X749C/T9XvKosb9xa ZuuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276515; x=1727881315; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yjSReXj8XxN+J1aHSpg+bgY7XO8zJXJrh2uX1BeFRGQ=; b=IeGBh98RloiaDymJ2Bbbi38ZGR4z/49GPLIo1SIH1IOB4pDylZa62eXpSl8TIXgOpS 6tRZZzCydrfD1IAcWZFdkGWLQoS+uQQi1ZUlSkjla/Q9dqhi3eE2g0iL5cI6PKjtXAQi 57Jm6EfVygHvUaUA+wMx/Y+NWrfRrsVef0zQPNnYU97oo/6bPJj4Sow7Fd9RnUyY3mw6 Y7y85++F4Ij2OF69WEON++qLhxjx5lc8sPFtApK8h62xX2VcRZNy6g9Uk6AS45y3Iu3W qE39oExV0FzLYqcUE9TWSnWEqv7frdrkx76QEGGhhaK8hkUM0ivb23WgYC4a1V6AT0Na NuuQ== X-Forwarded-Encrypted: i=1; AJvYcCWOHtC0B1VKTuXGseslV3b8GqwCUHEnJNOUNulnTrvHChRKz98omIjujukGY83QNDablYBq/LLg+F8=@vger.kernel.org X-Gm-Message-State: AOJu0YxytRGw4FWunmanaNUDot9Rm438KDuyStIvdnSCc3zWXQ8LXXgx EqLIZoaoUjqioSOaK6MZRBerp+48gIC2Np5VYlHz6aQxxndm2ZUenFwRoph5rdvtNrPv3Q== X-Google-Smtp-Source: AGHT+IGzIZ2o2YM9LwRx7GUBkPXP1nnHd93bOM06M/lDtcRNJJFbQb+/UGpm4f4GZO5kVPvAO8R9y3yP X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:600c:214f:b0:42c:b32e:6ba7 with SMTP id 5b1f17b1804b1-42e9624246bmr34035e9.6.1727276514236; Wed, 25 Sep 2024 08:01:54 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:02 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1869; i=ardb@kernel.org; h=from:subject; bh=Etb53DEUMtJxbS45M94XzvcBugumXuKFZ3M2B/KYBNo=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6jquDX165ub5J178W1mVH21S/GxP0fqUgljlqYpRR x60nEvpKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABPRa2RkeBV2XNd/yacJf4IS mc47ex9ud90aJbN1Uf8PDSuxLfrPTjD8rwhuEVogxe0Xov++IjA4keVg55+fz6JEZeM/B04o0/B nBgA= X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-32-ardb+git@google.com> Subject: [RFC PATCH 02/28] Documentation: Bump minimum GCC version to 8.1 From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Bump the minimum GCC version to 8.1 to gain unconditional support for referring to the per-task stack cookie using a symbol rather than relying on the fixed offset of 40 bytes from %GS, which requires elaborate hacks to support. Signed-off-by: Ard Biesheuvel Acked-by: Arnd Bergmann Acked-by: Miguel Ojeda --- Documentation/admin-guide/README.rst | 2 +- Documentation/process/changes.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst index f2bebff6a733..3dda41923ed6 100644 --- a/Documentation/admin-guide/README.rst +++ b/Documentation/admin-guide/README.rst @@ -259,7 +259,7 @@ Configuring the kernel Compiling the kernel -------------------- - - Make sure you have at least gcc 5.1 available. + - Make sure you have at least gcc 8.1 available. For more information, refer to :ref:`Documentation/process/changes.rst `. - Do a ``make`` to create a compressed kernel image. It is also possible to do diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 00f1ed7c59c3..59b7d3d8a577 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -29,7 +29,7 @@ you probably needn't concern yourself with pcmciautils. ====================== =============== ======================================== Program Minimal version Command to check the version ====================== =============== ======================================== -GNU C 5.1 gcc --version +GNU C 8.1 gcc --version Clang/LLVM (optional) 13.0.1 clang --version Rust (optional) 1.78.0 rustc --version bindgen (optional) 0.65.1 bindgen --version From patchwork Wed Sep 25 15:01:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830543 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DE7015B999 for ; Wed, 25 Sep 2024 15:02:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276525; cv=none; b=LluYG87z95zL/AI+joAqrioGaAeKoOnr8jQOOaqPA6o5YeFMmvX1IiL7KbLDA7jJTHcHpQZXcVMf4vxDdBbuPCvhVKrQLhyu7DgPewtMf+n7NYkUHvknE3D1J9Z2ztWGAH/mUKxHBaemrq4w6zmZ2cO6i9hHC/zJT2dm5nA60UA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276525; c=relaxed/simple; bh=9IeQ5D389OP3YsIvppVdXgRHsSH2R2gh2yLL82bpmwU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=A58wOOkt1bPwQ5Lfynwi3wzUd51fU89ktIUly5BCaeO2CyKiHsQpe3jlfIpgPQv7LJEbe7T+kMK/OM4iQi+2MktSvxns/c2oSDXgkmp3fwdY8vnOnosl38QA8WUPGxH14OMHVvhn9xJNHoPlOYoAQw7m6Adk6DUgtHSrNjvkKJE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fD083jGZ; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fD083jGZ" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e21dcc7044so16139857b3.1 for ; Wed, 25 Sep 2024 08:02:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276520; x=1727881320; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DA1xJ7QlyinRe20ykWBw1TGfZPAZZ2L4ow9I41mAh9Y=; b=fD083jGZz5kclsqEGn0PeXYqDSpGLWOYSTCBb8UAAsRPYt9jF3ZHXRYkabS7r5p0PA zvrYXsm1ABv3UhphJZ73GbBTQBMwKVmgv/xaOswaVzaJTp74idKtxPJPUhWKfkdVD1V4 5H3BVHpyBN2A2YOhjdjT5bTCZaHh2WTossr029pVLDkZoqCc5HoWGkkG9/UqdRcwQCAP Q4nvwkc+Yk9pRZO9zVsotNFXOCQsvyRQykfBSqsjR6lMT/WpX0ZyIiiD7H1qshH3klIC h91KxOdk5sAPFbd3CDMEvK76txrNKLWUkhQlfQsgCEfv6NPfOuy+mJHCnxs9POxEdn3c 83OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276520; x=1727881320; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DA1xJ7QlyinRe20ykWBw1TGfZPAZZ2L4ow9I41mAh9Y=; b=mZMu9gSS0pQeDPEH2Pep+VIyJGwtQSEuoABLP7W0cOWD8u6ooy0ln88HqQod27iQGS wIn4XeN8yCqaLUfyOEsufdjUqsOZu8lF53ORrHMiEUrNPxZW/GFwkIT02GuwnSNfaDxO ZmnmiMu+bV69hFTuKb7Jwa6spEbG2oVepeI+tBx5f1ivKI0pANYawxB8Rw8qbtUzGQMy 5XFxqFAu0AfhbK9lwkZDhsSiLMJWE57wgBh1hnarfpWijKSzk1RFqXA03LHth197zu7a qv+ozWyDeFgDRvsRw3XqWdTOes0DM28Q4gCWnlzVGlMNO8CqPM4SaUsy6Ubn6rbVEvEf YB4A== X-Forwarded-Encrypted: i=1; AJvYcCUP8qXotonUUoGd9ziVx7SOyt/WJquaJ4wHWLx155VEzT/4EUHMH9b6aGkgBrFEemW4jroYLxL/F2M=@vger.kernel.org X-Gm-Message-State: AOJu0YzrCotCtw59JxKnaY6K8p6Y90oKswLXUHJoorgkMl9zaVfJNJmJ X5zePqooUFJcFSFlYBivT8+HOkpEg4TfDzLdaat0P1NWY5Vepc6soTeKkVMzavm9uSRJ9w== X-Google-Smtp-Source: AGHT+IEVIf0RcL61TjfNJbSKcMrFRGT1L5gz2hG55UguzadiqCTtHCs2NniTQT5w6cgRQkWXw9cPp9k2 X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a81:77c4:0:b0:61c:89a4:dd5f with SMTP id 00721157ae682-6e21d0e8b05mr227547b3.0.1727276519796; Wed, 25 Sep 2024 08:01:59 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:04 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=6485; i=ardb@kernel.org; h=from:subject; bh=xSyMofrsI8W/QeJfcidSH2VUWxFZ3VEJOcPkUtiBlzY=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6sYKk4m5xZLyVelenm+8zuTMnXPA0faQXMTl3XV33 iYnS2/oKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABMRz2b4p2z7TbK64+yiCfEm LE6n5/75udhiw87M0k896bfk2oo/BTL8U20uvRe5bfeVZc9kGMJfb1kvfTNqJu/W1Zs+sddu+hR pywYA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-34-ardb+git@google.com> Subject: [RFC PATCH 04/28] x86/boot: Permit GOTPCREL relocations for x86_64 builds From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Some of the early x86_64 startup code is written in C, and executes in the early 1:1 mapping of the kernel, which is not the address it was linked at, and this requires special care when accessing global variables. This is currently being dealt with on an ad-hoc basis, primarily in head64.c, using explicit pointer fixups, but it would be better to rely on the compiler for this, by using -fPIE to generate code that can run at any address, and uses RIP-relative accesses to refer to global variables. While it is possible to avoid most GOT based symbol references that the compiler typically emits when running in -fPIE mode, by using 'hidden' visibility, there are cases where the compiler will always rely on the GOT, for instance, for weak external references (which may remain unsatisfied at link time). This means the build may produce a small number of GOT entries nonetheless. So update the reloc processing host tool to add support for this, and place the GOT in the .text section rather than discard it. Note that multiple GOT based references to the same symbol will share a single GOT entry, and so naively emitting a relocation for the GOT entry each time a reference to it is encountered could result in duplicates. Work around this by relying on the fact that the relocation lists are sorted, and deduplicate 64-bit relocations as they are emitted by comparing each entry with the previous one. Signed-off-by: Ard Biesheuvel --- arch/x86/Makefile | 4 +++ arch/x86/kernel/vmlinux.lds.S | 5 +++ arch/x86/tools/relocs.c | 33 ++++++++++++++++++-- include/asm-generic/vmlinux.lds.h | 7 +++++ 4 files changed, 47 insertions(+), 2 deletions(-) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 801fd85c3ef6..6b3fe6e2aadd 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -192,6 +192,10 @@ else KBUILD_CFLAGS += -mcmodel=kernel KBUILD_RUSTFLAGS += -Cno-redzone=y KBUILD_RUSTFLAGS += -Ccode-model=kernel + + # Don't emit relaxable GOTPCREL relocations + KBUILD_AFLAGS_KERNEL += -Wa,-mrelax-relocations=no + KBUILD_CFLAGS_KERNEL += -Wa,-mrelax-relocations=no endif # diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 6e73403e874f..7f060d873f75 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -20,6 +20,9 @@ #define RUNTIME_DISCARD_EXIT #define EMITS_PT_NOTE #define RO_EXCEPTION_TABLE_ALIGN 16 +#ifdef CONFIG_X86_64 +#define GOT_IN_RODATA +#endif #include #include @@ -464,10 +467,12 @@ SECTIONS * Sections that should stay zero sized, which is safer to * explicitly check instead of blindly discarding. */ +#ifdef CONFIG_X86_32 .got : { *(.got) *(.igot.*) } ASSERT(SIZEOF(.got) == 0, "Unexpected GOT entries detected!") +#endif .plt : { *(.plt) *(.plt.*) *(.iplt) diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 35a73e4aa74d..880f0f2e465e 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -223,6 +223,8 @@ static const char *rel_type(unsigned type) REL_TYPE(R_X86_64_JUMP_SLOT), REL_TYPE(R_X86_64_RELATIVE), REL_TYPE(R_X86_64_GOTPCREL), + REL_TYPE(R_X86_64_GOTPCRELX), + REL_TYPE(R_X86_64_REX_GOTPCRELX), REL_TYPE(R_X86_64_32), REL_TYPE(R_X86_64_32S), REL_TYPE(R_X86_64_16), @@ -843,6 +845,7 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, case R_X86_64_32: case R_X86_64_32S: case R_X86_64_64: + case R_X86_64_GOTPCREL: /* * References to the percpu area don't need to be adjusted. */ @@ -861,6 +864,31 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, break; } + if (r_type == R_X86_64_GOTPCREL) { + Elf_Shdr *s = &secs[sec->shdr.sh_info].shdr; + unsigned file_off = offset - s->sh_addr + s->sh_offset; + + /* + * GOTPCREL relocations refer to instructions that load + * a 64-bit address via a 32-bit relative reference to + * the GOT. In this case, it is the GOT entry that + * needs to be fixed up, not the immediate offset in + * the opcode. Note that the linker will have applied an + * addend of -4 to compensate for the delta between the + * relocation offset and the value of RIP when the + * instruction executes, and this needs to be backed out + * again. (Addends other than -4 are permitted in + * principle, but make no sense in practice so they are + * not supported.) + */ + if (rel->r_addend != -4) { + die("invalid addend (%ld) for %s relocation: %s\n", + rel->r_addend, rel_type(r_type), symname); + break; + } + offset += 4 + (int32_t)get_unaligned_le32(elf_image + file_off); + } + /* * Relocation offsets for 64 bit kernels are output * as 32 bits and sign extended back to 64 bits when @@ -870,7 +898,7 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, if ((int32_t)offset != (int64_t)offset) die("Relocation offset doesn't fit in 32 bits\n"); - if (r_type == R_X86_64_64) + if (r_type == R_X86_64_64 || r_type == R_X86_64_GOTPCREL) add_reloc(&relocs64, offset); else add_reloc(&relocs32, offset); @@ -1085,7 +1113,8 @@ static void emit_relocs(int as_text, int use_real_mode) /* Now print each relocation */ for (i = 0; i < relocs64.count; i++) - write_reloc(relocs64.offset[i], stdout); + if (!i || relocs64.offset[i] != relocs64.offset[i - 1]) + write_reloc(relocs64.offset[i], stdout); /* Print a stop */ write_reloc(0, stdout); diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 19ec49a9179b..cc14d780c70d 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -443,6 +443,12 @@ #endif #endif +#ifdef GOT_IN_RODATA +#define GOT_RODATA *(.got .igot*) +#else +#define GOT_RODATA +#endif + /* * Read only Data */ @@ -454,6 +460,7 @@ SCHED_DATA \ RO_AFTER_INIT_DATA /* Read only after init */ \ . = ALIGN(8); \ + GOT_RODATA \ BOUNDED_SECTION_BY(__tracepoints_ptrs, ___tracepoints_ptrs) \ *(__tracepoints_strings)/* Tracepoints: strings */ \ } \ From patchwork Wed Sep 25 15:01:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830542 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FF2018A6A9 for ; Wed, 25 Sep 2024 15:02:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276529; cv=none; b=BXChQdtF+VndjGHh1vKJ5B0XHxoVmmI+CpZ0O266xxWI/sdnZqYUXJCF426txhJd/TlOJczTgMUpK2YVjWjbeNHuY2nB5UtmcAILdl5tiBQqtEn7Zbacdnxh0tw/jZ0nSINpfYh2hRSncTP3kRG0dKmlFOEBQD3H0q2mpNAa3lI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276529; c=relaxed/simple; bh=J79MxP7lqCLqptP8R42gKw+nj8+FT+gfetDjKcKkNS8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WIJIyvwdJ963vSWUdWKLGxqQk72UDzG3KD0if9VTKThPyhokcuifXdFBwk25ILfcAVPFliGP0relMM3n2bDdVUs4R67ZniZgN+UhZ/KzsMB0EwKo1Bt/EanfhiHlQ+LF7B4FPyxhngIznjXViejRVJGjaANQlVkCiXLRwpDHLrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=f6dMyRxK; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="f6dMyRxK" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-69a0536b23aso106287217b3.3 for ; Wed, 25 Sep 2024 08:02:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276525; x=1727881325; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=eDSZAaST0F7o2pedlrN6lklBaOasvLDlFvV5LwnRCP8=; b=f6dMyRxKEh/badlNRLcmlg18jSpjvrDmYWxQ7EYbyWsfM9GAk/9cJfeqtthFBSO2QE tmr4Y8dW+RZ7AI2xlbBYV8GQJ0M5ZCm/1FXm9nQl+xGG38Gk72HRI81G0Vx6QxYBg522 cZt8fvb3jQBfwNVQaPg0Lw9QLYK7r0CUkbgNNCVhv2U5L5o1t8Lto/ATXHiV4KdnTizz 7CFRUdGaQKLn6Q+RoVRdtx4B8nRG1c80c7oVtSc+3pvLZHWdkR+HrdPBusieBrnsltZE WkbzMUMLLLSx0FLzA7ZkhwfzxmUWJuCTHzkzMUXD2IaA1o6jwe5GYsOKg2AIFgU5sD2t FVrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276525; x=1727881325; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eDSZAaST0F7o2pedlrN6lklBaOasvLDlFvV5LwnRCP8=; b=evrZIkkaFZKGTbYPl8pfFJmi+aZFVnTuuurqn5A6mo6ZAh+IMVxxlwpFqI4rkUyVUE qfo+36jTOLUw6YZv8jJvkY6poNdUsF4d9FWuSdB9wYaod0mQummNFo3M03mCqOemo9tL qYTeOpmnFm+zJPt0DQl++MclzkqpxshdLaehz+M5NCNE2CGNTvUW2dL5fIBF66R6WqhF HdD7980YUd1m3y/AYGdgPPd3MygzFKAjpJ+Sq7W3rebZ2zHQrLaLvfTeDiMuJ9OHhWrO nLxAIhO9A7X1MM2Seb6kH5JSlS2NxTR1a5EOyyGBHs/qCkOCCdWA4Hzmo66JObeqvM0p TL4w== X-Forwarded-Encrypted: i=1; AJvYcCW0/wyWAxrgX4qBAcM573EAEIulQSeCr/nOQdSBlROqFVHLRfpfMk52YINz+PIweyhQSjBYat97X/E=@vger.kernel.org X-Gm-Message-State: AOJu0Yz/wKNNwlFIb0ccRnXvvOjWs5nmGHyjN+NOvVHr1CMzA3TEAydi dYXlKBKyPCP/Ve/JRW2vgprJ4lGUWWY3DlpWNEqz+7SR2I2i8LMmhnEV2wgKoR5ciTQUZQ== X-Google-Smtp-Source: AGHT+IFVWEnfWJexNr1qxNhJJA6KdWYKxMoVe8Dw+8wG1wSOr4X5mOwe9MyKjjmZekWIuFcjH9KvW3oI X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:6902:1782:b0:e0b:958a:3344 with SMTP id 3f1490d57ef6-e24da39b0c3mr17932276.10.1727276524782; Wed, 25 Sep 2024 08:02:04 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:06 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=12230; i=ardb@kernel.org; h=from:subject; bh=d8zrHLLRsaJb8rjIFgQ7EHO+u22pN8JK1vGiiuXj4lQ=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6pZHteaXk89IXLoXMatcXipz+8w88ZqbTc8vrDt5e 99Nu9mbO0pZGMQ4GGTFFFkEZv99t/P0RKla51myMHNYmUCGMHBxCsBE3LsZ/oo9TvHP2+umfelZ ulBlzIGvqg9nBs08XVmsmMPN+vzurvOMDNcdmH9HcnIw6q7iylwgUHny+rJVkzYJbJk791Zkl/v XRQwA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-36-ardb+git@google.com> Subject: [RFC PATCH 06/28] x86/percpu: Get rid of absolute per-CPU variable placement From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel For historic reasons, per-CPU symbols on x86_64 are emitted in an address space that is disjoint from the ordinary kernel VA space, starting at address 0x0. This splits a per-CPU symbol reference into a base plus offset, where the base is programmed into the GS segment register. This deviates from the usual approach adopted by other SMP architectures, where the base is a reference to the variable in the kernel image's per-CPU template area, and the offset is a per-CPU value accounting for the displacement of that particular CPU's per-CPU region with respect to the template area. This gives per-CPU variable references a range that is identical to ordinary references, and requires no special handling for the startup code, as the offset will simply be 0x0 up until the point where per-CPU variables are initialized properly. The x86_64 approach was needed to accommodate per-task stack protector cookies, which used to live at a fixed offset of GS+40, requiring GS to be treated as a base register. This is no longer the case, though, and so GS can be repurposed as a true per-CPU offset, adopting the same strategy as other architectures. This also removes the need for linker tricks to emit the per-CPU ELF segment at a different virtual address. It also means RIP-relative per-CPU variables no longer need to be relocated in the opposite direction when KASLR is applied, which was necessary because the 0x0 based per-CPU region remains in place even when the kernel is moved around. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/desc.h | 1 - arch/x86/include/asm/percpu.h | 22 -------------- arch/x86/include/asm/processor.h | 5 ++-- arch/x86/kernel/head64.c | 2 +- arch/x86/kernel/head_64.S | 12 ++------ arch/x86/kernel/irq_64.c | 1 - arch/x86/kernel/setup_percpu.c | 9 +----- arch/x86/kernel/vmlinux.lds.S | 30 -------------------- arch/x86/platform/pvh/head.S | 6 ++-- arch/x86/tools/relocs.c | 8 +----- arch/x86/xen/xen-head.S | 10 ++----- init/Kconfig | 1 - 12 files changed, 13 insertions(+), 94 deletions(-) diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h index 62dc9f59ea76..ec95fe44fa3a 100644 --- a/arch/x86/include/asm/desc.h +++ b/arch/x86/include/asm/desc.h @@ -46,7 +46,6 @@ struct gdt_page { } __attribute__((aligned(PAGE_SIZE))); DECLARE_PER_CPU_PAGE_ALIGNED(struct gdt_page, gdt_page); -DECLARE_INIT_PER_CPU(gdt_page); /* Provide the original GDT */ static inline struct desc_struct *get_cpu_gdt_rw(unsigned int cpu) diff --git a/arch/x86/include/asm/percpu.h b/arch/x86/include/asm/percpu.h index c55a79d5feae..1ded1207528d 100644 --- a/arch/x86/include/asm/percpu.h +++ b/arch/x86/include/asm/percpu.h @@ -20,12 +20,6 @@ #define PER_CPU_VAR(var) __percpu(var)__percpu_rel -#ifdef CONFIG_X86_64_SMP -# define INIT_PER_CPU_VAR(var) init_per_cpu__##var -#else -# define INIT_PER_CPU_VAR(var) var -#endif - #else /* !__ASSEMBLY__: */ #include @@ -97,22 +91,6 @@ #define __percpu_arg(x) __percpu_prefix "%" #x #define __force_percpu_arg(x) __force_percpu_prefix "%" #x -/* - * Initialized pointers to per-CPU variables needed for the boot - * processor need to use these macros to get the proper address - * offset from __per_cpu_load on SMP. - * - * There also must be an entry in vmlinux_64.lds.S - */ -#define DECLARE_INIT_PER_CPU(var) \ - extern typeof(var) init_per_cpu_var(var) - -#ifdef CONFIG_X86_64_SMP -# define init_per_cpu_var(var) init_per_cpu__##var -#else -# define init_per_cpu_var(var) var -#endif - /* * For arch-specific code, we can use direct single-insn ops (they * don't give an lvalue though). diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 56bc36116814..d7219e149f24 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -409,11 +409,12 @@ struct fixed_percpu_data { }; DECLARE_PER_CPU_FIRST(struct fixed_percpu_data, fixed_percpu_data) __visible; -DECLARE_INIT_PER_CPU(fixed_percpu_data); static inline unsigned long cpu_kernelmode_gs_base(int cpu) { - return (unsigned long)&per_cpu(fixed_percpu_data, cpu); + extern unsigned long __per_cpu_offset[]; + + return IS_ENABLED(CONFIG_SMP) ? __per_cpu_offset[cpu] : 0; } extern asmlinkage void entry_SYSCALL32_ignore(void); diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 4b9d4557fc94..d4398261ad81 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -559,7 +559,7 @@ void early_setup_idt(void) */ void __head startup_64_setup_gdt_idt(void) { - struct desc_struct *gdt = (void *)(__force unsigned long)init_per_cpu_var(gdt_page.gdt); + struct desc_struct *gdt = (void *)(__force unsigned long)gdt_page.gdt; void *handler = NULL; struct desc_ptr startup_gdt_descr = { diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 330922b328bf..ab6ccee81493 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -68,11 +68,10 @@ SYM_CODE_START_NOALIGN(startup_64) /* Set up the stack for verify_cpu() */ leaq __top_init_kernel_stack(%rip), %rsp - /* Setup GSBASE to allow stack canary access for C code */ + /* Clear %gs so early per-CPU references target the per-CPU load area */ movl $MSR_GS_BASE, %ecx - leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx - movl %edx, %eax - shrq $32, %rdx + xorl %eax, %eax + cdq wrmsr call startup_64_setup_gdt_idt @@ -361,15 +360,10 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) /* Set up %gs. * - * The base of %gs always points to fixed_percpu_data. If the - * stack protector canary is enabled, it is located at %gs:40. * Note that, on SMP, the boot cpu uses init data section until * the per cpu areas are set up. */ movl $MSR_GS_BASE,%ecx -#ifndef CONFIG_SMP - leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx -#endif movl %edx, %eax shrq $32, %rdx wrmsr diff --git a/arch/x86/kernel/irq_64.c b/arch/x86/kernel/irq_64.c index ade0043ce56e..56bdeecd8ee0 100644 --- a/arch/x86/kernel/irq_64.c +++ b/arch/x86/kernel/irq_64.c @@ -27,7 +27,6 @@ #include DEFINE_PER_CPU_PAGE_ALIGNED(struct irq_stack, irq_stack_backing_store) __visible; -DECLARE_INIT_PER_CPU(irq_stack_backing_store); #ifdef CONFIG_VMAP_STACK /* diff --git a/arch/x86/kernel/setup_percpu.c b/arch/x86/kernel/setup_percpu.c index b30d6e180df7..57482420ff42 100644 --- a/arch/x86/kernel/setup_percpu.c +++ b/arch/x86/kernel/setup_percpu.c @@ -23,17 +23,10 @@ #include #include -#ifdef CONFIG_X86_64 -#define BOOT_PERCPU_OFFSET ((unsigned long)__per_cpu_load) -#else -#define BOOT_PERCPU_OFFSET 0 -#endif - -DEFINE_PER_CPU_READ_MOSTLY(unsigned long, this_cpu_off) = BOOT_PERCPU_OFFSET; +DEFINE_PER_CPU_READ_MOSTLY(unsigned long, this_cpu_off) = 0; EXPORT_PER_CPU_SYMBOL(this_cpu_off); unsigned long __per_cpu_offset[NR_CPUS] __ro_after_init = { - [0 ... NR_CPUS-1] = BOOT_PERCPU_OFFSET, }; EXPORT_SYMBOL(__per_cpu_offset); diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 7f060d873f75..00f82db7b3e1 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -103,9 +103,6 @@ PHDRS { text PT_LOAD FLAGS(5); /* R_E */ data PT_LOAD FLAGS(6); /* RW_ */ #ifdef CONFIG_X86_64 -#ifdef CONFIG_SMP - percpu PT_LOAD FLAGS(6); /* RW_ */ -#endif init PT_LOAD FLAGS(7); /* RWE */ #endif note PT_NOTE FLAGS(0); /* ___ */ @@ -225,17 +222,6 @@ SECTIONS __init_begin = .; /* paired with __init_end */ } -#if defined(CONFIG_X86_64) && defined(CONFIG_SMP) - /* - * percpu offsets are zero-based on SMP. PERCPU_VADDR() changes the - * output PHDR, so the next output section - .init.text - should - * start another segment - init. - */ - PERCPU_VADDR(INTERNODE_CACHE_BYTES, 0, :percpu) - ASSERT(SIZEOF(.data..percpu) < CONFIG_PHYSICAL_START, - "per-CPU data too large - increase CONFIG_PHYSICAL_START") -#endif - INIT_TEXT_SECTION(PAGE_SIZE) #ifdef CONFIG_X86_64 :init @@ -356,9 +342,7 @@ SECTIONS EXIT_DATA } -#if !defined(CONFIG_X86_64) || !defined(CONFIG_SMP) PERCPU_SECTION(INTERNODE_CACHE_BYTES) -#endif RUNTIME_CONST(shift, d_hash_shift) RUNTIME_CONST(ptr, dentry_hashtable) @@ -497,20 +481,6 @@ SECTIONS "kernel image bigger than KERNEL_IMAGE_SIZE"); #ifdef CONFIG_X86_64 -/* - * Per-cpu symbols which need to be offset from __per_cpu_load - * for the boot processor. - */ -#define INIT_PER_CPU(x) init_per_cpu__##x = ABSOLUTE(x) + __per_cpu_load -INIT_PER_CPU(gdt_page); -INIT_PER_CPU(fixed_percpu_data); -INIT_PER_CPU(irq_stack_backing_store); - -#ifdef CONFIG_SMP -. = ASSERT((fixed_percpu_data == 0), - "fixed_percpu_data is not at start of per-cpu area"); -#endif - #ifdef CONFIG_MITIGATION_UNRET_ENTRY . = ASSERT((retbleed_return_thunk & 0x3f) == 0, "retbleed_return_thunk not cacheline-aligned"); #endif diff --git a/arch/x86/platform/pvh/head.S b/arch/x86/platform/pvh/head.S index a308b79a887c..11245ecdc08d 100644 --- a/arch/x86/platform/pvh/head.S +++ b/arch/x86/platform/pvh/head.S @@ -95,9 +95,9 @@ SYM_CODE_START_LOCAL(pvh_start_xen) /* 64-bit entry point. */ .code64 1: - /* Set base address in stack canary descriptor. */ + /* Clear %gs so early per-CPU references target the per-CPU load area */ mov $MSR_GS_BASE,%ecx - mov $_pa(canary), %eax + xor %eax, %eax xor %edx, %edx wrmsr @@ -161,8 +161,6 @@ SYM_DATA_START_LOCAL(gdt_start) SYM_DATA_END_LABEL(gdt_start, SYM_L_LOCAL, gdt_end) .balign 16 -SYM_DATA_LOCAL(canary, .fill 48, 1, 0) - SYM_DATA_START_LOCAL(early_stack) .fill BOOT_STACK_SIZE, 1, 0 SYM_DATA_END_LABEL(early_stack, SYM_L_LOCAL, early_stack_end) diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 880f0f2e465e..10add45b99f1 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -88,7 +88,6 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = { "(jiffies|jiffies_64)|" #if ELF_BITS == 64 "__per_cpu_load|" - "init_per_cpu__.*|" "__end_rodata_hpage_align|" #endif "__vvar_page|" @@ -785,10 +784,6 @@ static void percpu_init(void) * The GNU linker incorrectly associates: * __init_begin * __per_cpu_load - * - * The "gold" linker incorrectly associates: - * init_per_cpu__fixed_percpu_data - * init_per_cpu__gdt_page */ static int is_percpu_sym(ElfW(Sym) *sym, const char *symname) { @@ -796,8 +791,7 @@ static int is_percpu_sym(ElfW(Sym) *sym, const char *symname) return (shndx == per_cpu_shndx) && strcmp(symname, "__init_begin") && - strcmp(symname, "__per_cpu_load") && - strncmp(symname, "init_per_cpu_", 13); + strcmp(symname, "__per_cpu_load"); } diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index 758bcd47b72d..faadac7c29e6 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -51,15 +51,9 @@ SYM_CODE_START(startup_xen) leaq __top_init_kernel_stack(%rip), %rsp - /* Set up %gs. - * - * The base of %gs always points to fixed_percpu_data. If the - * stack protector canary is enabled, it is located at %gs:40. - * Note that, on SMP, the boot cpu uses init data section until - * the per cpu areas are set up. - */ + /* Clear %gs so early per-CPU references target the per-CPU load area */ movl $MSR_GS_BASE,%ecx - movq $INIT_PER_CPU_VAR(fixed_percpu_data),%rax + xorl %eax, %eax cdq wrmsr diff --git a/init/Kconfig b/init/Kconfig index b05467014041..be8a9a786d3c 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1838,7 +1838,6 @@ config KALLSYMS_ALL config KALLSYMS_ABSOLUTE_PERCPU bool depends on KALLSYMS - default X86_64 && SMP # end of the "standard kernel features (expert users)" menu From patchwork Wed Sep 25 15:01:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830541 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C2BA18B47F for ; Wed, 25 Sep 2024 15:02:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276534; cv=none; b=VvPYX5mlcHn29zPuKV19waKl8D2MFHqJ7WVlfoq68ymV/m9ct+U7G2kdh0TrCZa84Yypq615+rNwhJzr84TJ/HnEsxVM+1z/Hq8XvnFKodv2ZBzRKeB2atQsv07ILl+YFV+eBP8G/VYYH7264o4dTM/JF0FxeuFQ1c3HmttFhSU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276534; c=relaxed/simple; bh=uNZCRc4hsOOw5KhJ4tKlrDfm3UOmHFTj64isJZp8vsM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=X09OuDCWFAlCR/D9HyW1R4y3sDDH3kqR3iJbkkOyaTRJ/661jGGhNJ1B3KvH80ycsFRhXXItF52k9eiYtQsSYufcXK+4cP2VwgVOs0jDgHTGyEh327u24jiScFQy7c/qmdj4/kKQuupIG4e2WeUO2d13naXKewAAdJ+nUdgEVFM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MW6uUn+Z; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MW6uUn+Z" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-690404fd230so104535547b3.3 for ; Wed, 25 Sep 2024 08:02:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276530; x=1727881330; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lvBkJLcu5Sh3tnaE09NAZ6RZ9jqQeWc/oMvpiTXt3OQ=; b=MW6uUn+ZkZmLYEiLpaElXr/beisIKf4BBTfjJogoXH37rHomkt2A4B7qCm+Kiwfe8i FAqZas4anHYnFdXfVuHp2PvwmUi7ACiKyFFllm2+Z7yRRdclrUYTvUA4tQUPeMShD8LY ZlqhyR7ImxJxRIV9h0l72S+BLqNBA/NpY5xzPQfn4O4hKt/jCRIr3jBlqfjocnzMcaHJ Bpv1ps4IrEFACOyXPoBaXnoQAOR4VLwaJSCtUt+qWWFLIgC2NW6G0QTsvGprB9LCOqBx gmh8TYa5MGBI5H44V6/wqzoDLuiCSDzoe5P5loSjctWpvVH/Z+rmtLwou5SmGz/KPBHe knIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276530; x=1727881330; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lvBkJLcu5Sh3tnaE09NAZ6RZ9jqQeWc/oMvpiTXt3OQ=; b=RtzhrMr/SDz+vwcZ8jbyiwYkxUkKr+rwNGwqQHPZcOVeN0DQebdtgMX3BQlkID/mHP gcme/hYwOuVNf1FIUYuXw/pHp3HB31iu12sGtmFa4eBwVY9rRwWYz11h3oosSk+GhK7z v/7KWB2VBPaCQWwbXHzKm10STdn2sY5Zwp27ZGWjXL8YpoFy5BOb1ELZoKSzxCrbMwq6 WvrWpnulri2Z1s2fI0BhpCG+EUQFOtiyDIRok371KA48Az7mnlAaMrvESdy89n+SCqj1 HvPmQviqHCNNNgT281L3NY5CCjO1H/y03S8fgnzXdGSur7BIxKRAuBJ4m6y4tYzuC12u 4kuQ== X-Forwarded-Encrypted: i=1; AJvYcCVVkZ8Jhg56gcpFCLNNJmU5gn9+XX7tYk+ZnroM1+Bdpwp3sk2uU7R1mQrT4u2tIhDkL2Ww4j0vj/c=@vger.kernel.org X-Gm-Message-State: AOJu0Yz5jMupR0TzQmqI8ZYfeAozXwZgCJ62PU6Pdrz9dFDThv0dxa60 CiPNW/aRTbYDmpN1tSExx9q7qYPGS4PNOgPx7fGop8xlW+jR2d6r82Qih1Ldy2nOL00PMw== X-Google-Smtp-Source: AGHT+IENPFDDE4RBTdISjbf4G4pk2mGOjzo8FmXYzCFJ7WDt9ZFnIbiw80yVnuCNeltTBlMkQAqf4tvt X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:690c:c9c:b0:6b0:d571:3540 with SMTP id 00721157ae682-6e21da796e6mr255067b3.6.1727276529787; Wed, 25 Sep 2024 08:02:09 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:08 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=6334; i=ardb@kernel.org; h=from:subject; bh=rOkV7b1W8gqAvLsLoQ2i9zRn35wAefNGZsd9tIotAWg=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6vYGW5mupTcMu0+/lmvg/R+5wEt4Wbk5z8ydQtZqX U8bN7/vKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABO5Lc3wP3L7HcnzmyvvZR1M rS85onToQ4R+5Y7wJefSbJaqGNxXYmVkaHXklA3NNHruJrHYtjJMvdzeSUx0wZXnutnTlvEcNXD iBAA= X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-38-ardb+git@google.com> Subject: [RFC PATCH 08/28] scripts/kallsyms: Remove support for absolute per-CPU variables From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel SMP on x86_64 no longer needs absolute per-CPU variables, so this support can be dropped from kallsyms as well, as no other architectures rely on this functionality. Signed-off-by: Ard Biesheuvel --- init/Kconfig | 4 -- kernel/kallsyms.c | 12 +---- scripts/kallsyms.c | 51 +++----------------- scripts/link-vmlinux.sh | 4 -- 4 files changed, 9 insertions(+), 62 deletions(-) diff --git a/init/Kconfig b/init/Kconfig index be8a9a786d3c..f6eeba81282d 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1835,10 +1835,6 @@ config KALLSYMS_ALL Say N unless you really need all symbols, or kernel live patching. -config KALLSYMS_ABSOLUTE_PERCPU - bool - depends on KALLSYMS - # end of the "standard kernel features (expert users)" menu config ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index a9a0ca605d4a..4198f30aac3c 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -148,16 +148,8 @@ static unsigned int get_symbol_offset(unsigned long pos) unsigned long kallsyms_sym_address(int idx) { - /* values are unsigned offsets if --absolute-percpu is not in effect */ - if (!IS_ENABLED(CONFIG_KALLSYMS_ABSOLUTE_PERCPU)) - return kallsyms_relative_base + (u32)kallsyms_offsets[idx]; - - /* ...otherwise, positive offsets are absolute values */ - if (kallsyms_offsets[idx] >= 0) - return kallsyms_offsets[idx]; - - /* ...and negative offsets are relative to kallsyms_relative_base - 1 */ - return kallsyms_relative_base - 1 - kallsyms_offsets[idx]; + /* values are unsigned offsets */ + return kallsyms_relative_base + (u32)kallsyms_offsets[idx]; } static unsigned int get_symbol_seq(int index) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 09757d300a05..9c34b9397872 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -5,7 +5,7 @@ * This software may be used and distributed according to the terms * of the GNU General Public License, incorporated herein by reference. * - * Usage: kallsyms [--all-symbols] [--absolute-percpu] in.map > out.S + * Usage: kallsyms [--all-symbols] in.map > out.S * * Table compression uses all the unused char codes on the symbols and * maps these to the most used substrings (tokens). For instance, it might @@ -37,7 +37,6 @@ struct sym_entry { unsigned long long addr; unsigned int len; unsigned int seq; - bool percpu_absolute; unsigned char sym[]; }; @@ -62,7 +61,6 @@ static struct addr_range percpu_range = { static struct sym_entry **table; static unsigned int table_size, table_cnt; static int all_symbols; -static int absolute_percpu; static int token_profit[0x10000]; @@ -73,7 +71,7 @@ static unsigned char best_table_len[256]; static void usage(void) { - fprintf(stderr, "Usage: kallsyms [--all-symbols] [--absolute-percpu] in.map > out.S\n"); + fprintf(stderr, "Usage: kallsyms [--all-symbols] in.map > out.S\n"); exit(1); } @@ -175,7 +173,6 @@ static struct sym_entry *read_symbol(FILE *in, char **buf, size_t *buf_len) sym->len = len; sym->sym[0] = type; strcpy(sym_name(sym), name); - sym->percpu_absolute = false; return sym; } @@ -319,11 +316,6 @@ static int expand_symbol(const unsigned char *data, int len, char *result) return total; } -static bool symbol_absolute(const struct sym_entry *s) -{ - return s->percpu_absolute; -} - static int compare_names(const void *a, const void *b) { int ret; @@ -457,20 +449,10 @@ static void write_src(void) long long offset; bool overflow; - if (!absolute_percpu) { - offset = table[i]->addr - relative_base; - overflow = offset < 0 || offset > UINT_MAX; - } else if (symbol_absolute(table[i])) { - offset = table[i]->addr; - overflow = offset < 0 || offset > INT_MAX; - } else { - offset = relative_base - table[i]->addr - 1; - overflow = offset < INT_MIN || offset >= 0; - } + offset = table[i]->addr - relative_base; + overflow = (offset < 0 || offset > UINT_MAX); if (overflow) { - fprintf(stderr, "kallsyms failure: " - "%s symbol value %#llx out of range in relative mode\n", - symbol_absolute(table[i]) ? "absolute" : "relative", + fprintf(stderr, "kallsyms failure: symbol value %#llx out of range\n", table[i]->addr); exit(EXIT_FAILURE); } @@ -725,32 +707,16 @@ static void sort_symbols(void) qsort(table, table_cnt, sizeof(table[0]), compare_symbols); } -static void make_percpus_absolute(void) -{ - unsigned int i; - - for (i = 0; i < table_cnt; i++) - if (symbol_in_range(table[i], &percpu_range, 1)) { - /* - * Keep the 'A' override for percpu symbols to - * ensure consistent behavior compared to older - * versions of this tool. - */ - table[i]->sym[0] = 'A'; - table[i]->percpu_absolute = true; - } -} - /* find the minimum non-absolute symbol address */ static void record_relative_base(void) { unsigned int i; for (i = 0; i < table_cnt; i++) - if (table[i]->addr && !symbol_absolute(table[i])) { + if (table[i]->addr) { /* * The table is sorted by address. - * Take the first non-absolute symbol value. + * Take the first non-zero symbol value. */ relative_base = table[i]->addr; return; @@ -762,7 +728,6 @@ int main(int argc, char **argv) while (1) { static const struct option long_options[] = { {"all-symbols", no_argument, &all_symbols, 1}, - {"absolute-percpu", no_argument, &absolute_percpu, 1}, {}, }; @@ -779,8 +744,6 @@ int main(int argc, char **argv) read_map(argv[optind]); shrink_table(); - if (absolute_percpu) - make_percpus_absolute(); sort_symbols(); record_relative_base(); optimize_token_table(); diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index a9b3f34a78d2..df5f3fbb46f3 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -140,10 +140,6 @@ kallsyms() kallsymopt="${kallsymopt} --all-symbols" fi - if is_enabled CONFIG_KALLSYMS_ABSOLUTE_PERCPU; then - kallsymopt="${kallsymopt} --absolute-percpu" - fi - info KSYMS "${2}.S" scripts/kallsyms ${kallsymopt} "${1}" > "${2}.S" From patchwork Wed Sep 25 15:01:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830540 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42B0C18C925 for ; Wed, 25 Sep 2024 15:02:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276539; cv=none; b=To0/y7B9EAvZpLgdo7UKMyAyltp+vmORQgAcnrjeW4d7/2UKyGtSkrW7LXkWYLt/TdyujFjrnsrUslcLnrJZfZdqfeK2YqtjcOTrETYiT97acn10d/OIAGL/+CNr6drGLLaur7XgAWUeHOiJWjcs/kwLhpguzOmrlJi61aT1qvc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276539; c=relaxed/simple; bh=UKT9SQcfQYmFYYWRpiFhP5glZL5JX0TJEX6TTbNQ3aU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qblysZJ3f17D3qu6McxDP6JBP0jpIxH9KI/um1dly8Il5hhzhWpeIbX2EC+GKdbzrIopo7DX3BjpIlSlYO/J0ELPGb7Y8XexrOsMqW/wNlAbtXzjL1a4izT7s+6jaB3gukltUxZftHHeRAzqL5PW3Ohmc4G6MlGUPZjcBd9DA2k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GFu9ndsn; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GFu9ndsn" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6de0b23f4c5so13720167b3.1 for ; Wed, 25 Sep 2024 08:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276535; x=1727881335; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mwy8kTVjZ/08dn4MFhxsJ/CPkFSWb32MIOuFSl+DFsU=; b=GFu9ndsnRjzSFcE0ymWC4FAjAXMOdToQQro2m7qyhCSaggFnIjbbq3FpZ9qojdOI1s 0icht9YZ+QmexTkBBMna+b6TcfJ6SMyLaEYeP7uWtcH7NlzUmxwLuBH/jwabKj+rcKHD 1A3QfpZOmnZ1awDYuofRV/zXjo+F27c32ESQ2zd4+CO8fI+yiUCE2/aUQBHJP6z1F5l6 BWYglyCJd/yWtXSY6GhWHncUbFDipkvw4vOPtO2JYTWqZSgZyWg4X+jdafRAO2hkRySt 9TVjFeBL7Q4UGHa+PJA6ypNh1zBGT1EPxgpv3z3Y4saJIvvU8VraQ0+ltB6KCorCuUt4 5QwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276535; x=1727881335; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mwy8kTVjZ/08dn4MFhxsJ/CPkFSWb32MIOuFSl+DFsU=; b=UUfMxUw/7jihB9eflBmgnnoQYPZffD/rnebbWyDIkzf4hJZnOpCYJ7hL8bzahF7KN8 DwJNrO7QrqGAtHEfvI0O2UQYDWQNUZPRF47JxQnIrshCEP/7eqG/U9lIXGOqjUUGxJ5v 3NyPSHAV81OZeH44btRJ9CuVJHBMznsDQVpFkU2UogXJ1KlYiYIpIcwQOzizbQyfzLxz BinQgAfujJ5USm8HWdn1aeolCER9wTa+J6yPJqi1bkvhNMzAVS8+wyXw4Pc6vZve4hWa 6au06oOUWHIqBFjogIBGpQYQtLems2sQbdPJZQ6kC4uyWLXu/t5Fbwujk+iQfFWmw6xY aIfQ== X-Forwarded-Encrypted: i=1; AJvYcCUnby0ysOOnS4vEOOGECMOLEnhp1YKQTarKjSV+BBjaWvXbd+ABqEEO/iFCWlCGxv++g+nUCA+mRFM=@vger.kernel.org X-Gm-Message-State: AOJu0Yy189gY4b/SlwEOf2dqQ2KzvSCPeornNc1VVTlKGylQZjtsXTcF QzShz7UuAmGwmxzJkD74cp3TS5KOWF1a3rYR3ey+ORl9I0tYOkOuLQuzhi4sW6nQpxR6gw== X-Google-Smtp-Source: AGHT+IEEpXdESu+AVZ5xAH4ocoULrkWfhbmfoqTWPQarKhwafrufRJLTuhlZhXvWoZ56ZcFdzDjPCpOs X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:690c:6a0d:b0:6da:3596:21b8 with SMTP id 00721157ae682-6e2089c8234mr1848507b3.4.1727276535155; Wed, 25 Sep 2024 08:02:15 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:10 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4217; i=ardb@kernel.org; h=from:subject; bh=umOxUEgpm+XSdzq7sqW2wzWT3DKWlT715fLzy00ZrIc=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6q4zMe0fH8zn/6yon+W25vvljx/znOb8fBedfXMvk 4PJy+hrHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAiC5MZGX675CxMX+/71dxB RTLX6OaHWScZikUU3t9k3TfnWIl7YijDP8vjN+r/hng3npecsefHgZqi2WFz5zPUNTIeVXy24tT y6awA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-40-ardb+git@google.com> Subject: [RFC PATCH 10/28] x86/xen: Avoid relocatable quantities in Xen ELF notes From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Xen puts virtual and physical addresses into ELF notes that are treated by the linker as relocatable by default. Doing so is not only pointless, given that the ELF notes are only intended for consumption by Xen before the kernel boots. It is also a KASLR leak, given that the kernel's ELF notes are exposed via the world readable /sys/kernel/notes. So emit these constants in a way that prevents the linker from marking them as relocatable. This involves place-relative relocations (which subtract their own virtual address from the symbol value) and linker provided absolute symbols that add the address of the place to the desired value. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/vmlinux.lds.S | 13 +++++++++++++ arch/x86/platform/pvh/head.S | 6 +++--- arch/x86/tools/relocs.c | 1 + arch/x86/xen/xen-head.S | 6 ++++-- 4 files changed, 21 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 00f82db7b3e1..52b8db931d0f 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -111,6 +111,19 @@ PHDRS { SECTIONS { . = __START_KERNEL; + +#ifdef CONFIG_XEN_PV +xen_elfnote_entry_offset = + ABSOLUTE(xen_elfnote_entry) + ABSOLUTE(startup_xen); +xen_elfnote_hypercall_page_offset = + ABSOLUTE(xen_elfnote_hypercall_page) + ABSOLUTE(hypercall_page); +#endif + +#ifdef CONFIG_PVH +xen_elfnote_phys32_entry_offset = + ABSOLUTE(xen_elfnote_phys32_entry) + ABSOLUTE(pvh_start_xen - LOAD_OFFSET); +#endif + #ifdef CONFIG_X86_32 phys_startup_32 = ABSOLUTE(startup_32 - LOAD_OFFSET); #else diff --git a/arch/x86/platform/pvh/head.S b/arch/x86/platform/pvh/head.S index 11245ecdc08d..adbf57e83e4e 100644 --- a/arch/x86/platform/pvh/head.S +++ b/arch/x86/platform/pvh/head.S @@ -50,7 +50,7 @@ #define PVH_CS_SEL (PVH_GDT_ENTRY_CS * 8) #define PVH_DS_SEL (PVH_GDT_ENTRY_DS * 8) -SYM_CODE_START_LOCAL(pvh_start_xen) +SYM_CODE_START(pvh_start_xen) UNWIND_HINT_END_OF_STACK cld @@ -165,5 +165,5 @@ SYM_DATA_START_LOCAL(early_stack) .fill BOOT_STACK_SIZE, 1, 0 SYM_DATA_END_LABEL(early_stack, SYM_L_LOCAL, early_stack_end) - ELFNOTE(Xen, XEN_ELFNOTE_PHYS32_ENTRY, - _ASM_PTR (pvh_start_xen - __START_KERNEL_map)) + ELFNOTE(Xen, XEN_ELFNOTE_PHYS32_ENTRY, .global xen_elfnote_phys32_entry; + xen_elfnote_phys32_entry: _ASM_PTR xen_elfnote_phys32_entry_offset - .) diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 942c029a5067..22c2d3f07a57 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -57,6 +57,7 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = { [S_ABS] = "^(xen_irq_disable_direct_reloc$|" "xen_save_fl_direct_reloc$|" + "xen_elfnote_.+_offset$|" "VDSO|" "__kcfi_typeid_|" "__crc_)", diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index faadac7c29e6..4d246a48a85f 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -88,7 +88,8 @@ SYM_CODE_END(xen_cpu_bringup_again) ELFNOTE(Xen, XEN_ELFNOTE_VIRT_BASE, _ASM_PTR __START_KERNEL_map) /* Map the p2m table to a 512GB-aligned user address. */ ELFNOTE(Xen, XEN_ELFNOTE_INIT_P2M, .quad (PUD_SIZE * PTRS_PER_PUD)) - ELFNOTE(Xen, XEN_ELFNOTE_ENTRY, _ASM_PTR startup_xen) + ELFNOTE(Xen, XEN_ELFNOTE_ENTRY, .globl xen_elfnote_entry; + xen_elfnote_entry: _ASM_PTR xen_elfnote_entry_offset - .) ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .ascii "!writable_page_tables") ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz "yes") ELFNOTE(Xen, XEN_ELFNOTE_L1_MFN_VALID, @@ -109,7 +110,8 @@ SYM_CODE_END(xen_cpu_bringup_again) #else # define FEATURES_DOM0 0 #endif - ELFNOTE(Xen, XEN_ELFNOTE_HYPERCALL_PAGE, _ASM_PTR hypercall_page) + ELFNOTE(Xen, XEN_ELFNOTE_HYPERCALL_PAGE, .globl xen_elfnote_hypercall_page; + xen_elfnote_hypercall_page: _ASM_PTR xen_elfnote_hypercall_page_offset - .) ELFNOTE(Xen, XEN_ELFNOTE_SUPPORTED_FEATURES, .long FEATURES_PV | FEATURES_PVH | FEATURES_DOM0) ELFNOTE(Xen, XEN_ELFNOTE_LOADER, .asciz "generic") From patchwork Wed Sep 25 15:01:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830539 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 180FF18E778 for ; Wed, 25 Sep 2024 15:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276544; cv=none; b=uteGzuaYNN+WtYPDV+hiFae+VMyQacTOI8S8nsK3qQNrNw9PcLuRoMKjoJFwuWOtSAd2kRKlAfL9GgFSZzsri4+BiQ0Uft8e56E5MQq4QnWqBLecqnpQ1gbdcADfzv3ID9zbh5tK+4ieaBKhVGzvQFnvBipcwXFsjzs1JkiBoFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276544; c=relaxed/simple; bh=5QDZ8e6v38oe4vdIzJiUkWBfZtQ8UHIt+vUykdg9FgY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kG4PR6lqqUz1+bhdrYB/Fgw9Qv8WWrH9DOwIlOccPGXp04hE9BPHdkRRUKMBxK8ay9Of7UYsluwEu0UIGlT87FDdGRS1Pgvgg1aPooGFA2GGBzyefZgU9rhrVmFqgaa6D8U8DT2frBf6lw6rruFaIA5IKLYaT5RPpvwhaPxMb/M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=btIfw3b8; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="btIfw3b8" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-5c250f3b18aso4898139a12.2 for ; Wed, 25 Sep 2024 08:02:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276540; x=1727881340; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sMLI1adUSxUIp8UXfDcqdD38QxXI+e+F/+Ag4yCxL7E=; b=btIfw3b8uzUX3qs7RbZNOIodIRQS3kYxnKbB07MkGpXPYtKc+y45geOrWIRE35o0GJ 5YzfMc8m/wwUmK922QVfZyXI+B9wuA11T+flD5UOxahZBXEtemDkGE3W7/J4g0rup0hY RyswEcm3adw5fvIh3eha8RaOBvemjfbbMcjxK6TO86cFNcWaluqJU/uHawwpriFuKlzC b5Ay07lm9xUJ9+hbAhwj5blziH/yfFSGqOTaAfwczL1nqILOKB83UYvc+v0ZoNMQWtNe zpF1gMq1hKxaI61gGVFhZCZNMi0lZFV8gGArhbSqFgJ/PphdZxS/hEKLUNBePwgIibqI GGTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276540; x=1727881340; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sMLI1adUSxUIp8UXfDcqdD38QxXI+e+F/+Ag4yCxL7E=; b=fQGYoumyo13eG+MCtWtfdSWBK+0dZzFuIfNW2GlWo1QbI1dZ1Tiv0v2QmiBJRB0FG8 8UI1pU876IlYJvCCtHVtZx1GUBFrcXltUVP9VKse2mZBC+etFx/0sGdiWVXHwXmTaPWN TMlNVMu1dtSdphpGBNdAP9FULGxkMEYA2qr7OFb/d1Xt3K3v3GeF09hEAecaeyWjbr32 1pS03XaJg4cqfYL37qR8p94as7OI1lbomszD00AngymkzRyCeF6AHSE6LDXlNI9tKbPU pQ5MImcafy4Pyc1wvgtq9fyIEgyYaFKorVNa21Gx5Q71YsDcloRTIQhvSIvTxnxwI8/H kOVw== X-Forwarded-Encrypted: i=1; AJvYcCVCcvBLMiTyHmUdbi888bQvxvlTjUNVhjKEN6CScHUjPoeXhETx0nGOySHjxg1ArRy9gFuJQiB4pY8=@vger.kernel.org X-Gm-Message-State: AOJu0YxSh8RXgmN/TqokzYi9FPy4oGW73VEffk0bI0uchS+uOMrXfIBk gzxxUJHbuMP74wrkzyFkgR3LaK2emhGSasWLiQ53yCAiSHZ8NX4u0L41Ba1OsCmFV36HRQ== X-Google-Smtp-Source: AGHT+IHqdrbnTURYSpf/5afh17e1i3rk5opqYbKAlLurQBauppiTWqSh5+YV6ICpl369+Ha35/G5jy3r X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a17:906:6d8:b0:a8a:76fd:ae67 with SMTP id a640c23a62f3a-a93a061ba72mr129666b.10.1727276539991; Wed, 25 Sep 2024 08:02:19 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:12 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2056; i=ardb@kernel.org; h=from:subject; bh=VKZKKQxE7f774DDxhaNjhf6gM8y+4CHtkhz4ITFKHVU=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6l6HjphM91K+tMbfPlum+q87xXbcaBq/plqB6J0rH Clz7dQ6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwESez2L4w3XU9mNkQveSSXqf bkfpTfktlPbu753X3m4//t4RvX456gUjw9ItwQrWWxY9aPiwtX3xVeeW5YpKNdGO1l4qU7KmnF4 VzwoA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-42-ardb+git@google.com> Subject: [RFC PATCH 12/28] x86/pm-trace: Use RIP-relative accesses for .tracedata From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Use RIP-relative accesses and 32-bit offsets for .tracedata, to avoid the need for relocation fixups at boot time. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/pm-trace.h | 4 ++-- drivers/base/power/trace.c | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/pm-trace.h b/arch/x86/include/asm/pm-trace.h index bfa32aa428e5..123faf978473 100644 --- a/arch/x86/include/asm/pm-trace.h +++ b/arch/x86/include/asm/pm-trace.h @@ -8,10 +8,10 @@ do { \ if (pm_trace_enabled) { \ const void *tracedata; \ - asm volatile(_ASM_MOV " $1f,%0\n" \ + asm volatile("lea " _ASM_RIP(1f) ", %0\n" \ ".section .tracedata,\"a\"\n" \ "1:\t.word %c1\n\t" \ - _ASM_PTR " %c2\n" \ + ".long %c2 - .\n" \ ".previous" \ :"=r" (tracedata) \ : "i" (__LINE__), "i" (__FILE__)); \ diff --git a/drivers/base/power/trace.c b/drivers/base/power/trace.c index cd6e559648b2..686a0276ccfc 100644 --- a/drivers/base/power/trace.c +++ b/drivers/base/power/trace.c @@ -167,7 +167,7 @@ EXPORT_SYMBOL(set_trace_device); void generate_pm_trace(const void *tracedata, unsigned int user) { unsigned short lineno = *(unsigned short *)tracedata; - const char *file = *(const char **)(tracedata + 2); + const char *file = offset_to_ptr((int *)(tracedata + 2)); unsigned int user_hash_value, file_hash_value; if (!x86_platform.legacy.rtc) @@ -187,9 +187,9 @@ static int show_file_hash(unsigned int value) match = 0; for (tracedata = __tracedata_start ; tracedata < __tracedata_end ; - tracedata += 2 + sizeof(unsigned long)) { + tracedata += 2 + sizeof(int)) { unsigned short lineno = *(unsigned short *)tracedata; - const char *file = *(const char **)(tracedata + 2); + const char *file = offset_to_ptr((int *)(tracedata + 2)); unsigned int hash = hash_string(lineno, file, FILEHASH); if (hash != value) continue; From patchwork Wed Sep 25 15:01:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830538 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9F5F18FDC6 for ; Wed, 25 Sep 2024 15:02:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276548; cv=none; b=WPHe1lagZ1czp3FHbPBd5QWtoP62jF9xQAuUnfio0o9rYdvgWuZqBaOHFlAGuUDrLXEh5j0VbIIAdXmCjprJqwkJJzd1tTEFIqLtEhQNGTw4kd7Hl/y7ORs54qGVFUMwrqjdImq6Of+i8X/gsFFXbrKOLv2biAc4tU/TeNR1N3Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276548; c=relaxed/simple; bh=vuy2srbwlGWCNhjJ429H+XQTUepQW5BfwFX52st1yXg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=J+A9kjD+lRv381W8pFV1yUKfho4AZkUY0EQ5+USCdyTCFpA6WtlX2OkMEmEYz867rnTxGNFH3WbiXfaos9RxZNpTclknB0//auGlOIAvFnpdQG+N5tYsTDABUou6MzA4Tca1oF49Vi5WjgVoq7Hpw2fCMkgXVbW2EQkrKL3bNK4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=P1KxEfU7; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="P1KxEfU7" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6db7a8c6910so102659287b3.0 for ; Wed, 25 Sep 2024 08:02:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276546; x=1727881346; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=TEh1/zkt2ebfm60sAnflL74oR8UqJJcIMQFWRvgC+dE=; b=P1KxEfU7NFI1FLa68htOn3FIemF2YoJ6ZngfHUhsKoFBRiXxiwVBOTYecj3iV5fXut Kp5pAy4MyYpTuIwIh1ul53pcejuye0feuUW/N2sSiiV23IoHpesKme5QnUN3QIRK9Scm jOJSpxVOtjp+R4vV2H1BXWolNHvOaltnkFQZO4b8HH+yxzEJYon4nNKIV9abmiJ3Qr6z MB45/dPAX38qoLlqeKxr2UzmTg6b3zQjOj2t0/4aKMhnPp9H0u99nESwyciFmWtxtP7X WlwMG8GRS226I1RCGZJjImxo/4LOik3jTeV5QqHXPPvoYIyhU83s5SKOqD4zhyJaMGyV AqrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276546; x=1727881346; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TEh1/zkt2ebfm60sAnflL74oR8UqJJcIMQFWRvgC+dE=; b=jzAm+iie5P8I6V3+lD2nElvqPUY0dk5NT5tW9u+V/OPioBKF8EfhuGbeeEGrxNRFLV cbVB0recgQZN6yU33rq5LbkASrCH+Wk39iGYBBtRq6JdQud1sIWpjgd40SCgWxm4j1EA J41C6QAdAPdxh4/7gySJk8O5NN2o/DW3Bw/iRfBrWl939iX3dBd6H/we81Op9FLU0MX7 HYxRqStiSDCwEoqEVPmdD1Ux9N1lgUadpG4/O0HTGBJoxmapvovi+Mk9+SORKqQSvaGx wKlVfDRl9TdTcNbl7EVqLU6Hm0exPq7/kkBKJRL4XLgUTprAmuQo+BJlHGB0XrQY9Rmx rg1w== X-Forwarded-Encrypted: i=1; AJvYcCXoOCYsdT7+Bjb6wAjH4Bd91kMMokMy2386vqepQLQ9VkpEX89/nzw0aUpCewJEtu63PYoBPPo9zcs=@vger.kernel.org X-Gm-Message-State: AOJu0YxDZoCy0mR50syNIaW6HWsu55e53SPBxQpz4aaEVCyQFOtaK//7 ULK8W7zbQhT3ssnOGWoWrmjc+2VIMUtZY9e/rlE1cJNy5wRtO/oPNDf3IdONapWu0GiAsQ== X-Google-Smtp-Source: AGHT+IGW5+6XYu5evov+Dvb+HROZj8E6YTyxVVP1a7XWDg5djajQJxqHbA/Whb8+5/ewCu11GBBmm/SO X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:690c:438d:b0:6b1:8b74:978a with SMTP id 00721157ae682-6e21d9d4ae9mr51697b3.4.1727276545723; Wed, 25 Sep 2024 08:02:25 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:14 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=934; i=ardb@kernel.org; h=from:subject; bh=DJZDFnSYvzXqQFYRPN/gizuDfobz3ccWU866DGFdU/U=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6v7XV9cI3q9cJcJR8+Vf2+Fz2Wk/Dh/Y/X1ToOXyt a351cxsHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAiXdsZGZaErou6cPCEcv22 Prcpp+bqHWznepHA8X6Z9Zn/83etFNBlZOg9+zlgLqMlq/AX9n2rM647K2+Mmvyn9GzNS4f81qf fJ/EAAA== X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-44-ardb+git@google.com> Subject: [RFC PATCH 14/28] x86/rethook: Use RIP-relative reference for return address From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Instead of pushing an immediate absolute address, which is incompatible with PIE codegen or linking, use a LEA instruction to take the address into a register. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/rethook.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/rethook.c b/arch/x86/kernel/rethook.c index 8a1c0111ae79..3b3c17ba3cd5 100644 --- a/arch/x86/kernel/rethook.c +++ b/arch/x86/kernel/rethook.c @@ -27,7 +27,8 @@ asm( #ifdef CONFIG_X86_64 ANNOTATE_NOENDBR /* This is only jumped from ret instruction */ /* Push a fake return address to tell the unwinder it's a rethook. */ - " pushq $arch_rethook_trampoline\n" + " leaq arch_rethook_trampoline(%rip), %rdi\n" + " pushq %rdi\n" UNWIND_HINT_FUNC " pushq $" __stringify(__KERNEL_DS) "\n" /* Save the 'sp - 16', this will be fixed later. */ From patchwork Wed Sep 25 15:01:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830537 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 110DB1917EB for ; Wed, 25 Sep 2024 15:02:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276554; cv=none; b=rZ93EI341QSXcnua283Z2XY4p8/xkfQyVAVPaee21fWKL49y+9ZrYdMjSLK95YB+Ih2SUMqJbqWQMK/6P54fQwjiXmw0s4P5+m9A0+g+1l5ocshZnvn8soQUX9O//mRlQ0/w9mpJuIwdjyxPeK9mdE6kjTSp7dHhFieUyUbb7nE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276554; c=relaxed/simple; bh=w/P+2/7/i0/j2dzTpIfGv0L554wt2D6+DkL221lAYlA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hlde/89UvJ2hiV3qalgr7ILnWEzejlIFVrT0wgjx0vfpG3mjM43YGJS/lVE0PqOMfbpOFw6gszuPNGD1wmEj5gRfJheWCci0xIEEJX5jl17JNvLMseIHx4saDYBjUlo27FlxzPQnrm2qOqjuOKQbuuRMqdobkUu7ys/rzc4ALCc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=A6rF0mzg; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="A6rF0mzg" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-42cb374f0cdso5460505e9.0 for ; Wed, 25 Sep 2024 08:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276550; x=1727881350; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=vAC8l3Y8WvFUyajZb+pirsZCUpFvZ3r8+BwrpQuj/SE=; b=A6rF0mzgNO5k5xH80dgQj186dPnQGeeVqRHb+orNsMAT0mKVCvVjECgFsIle1N6xP2 ypjnsxssWW1e1kjkRgMWJak2mtDJty2lviQC2542ZJvNFLmZuTN+xmpbQhsO9uWHv1Ta +sZDu12TEmKoQRiRvIDyUWMuIGN0jpUGTprTdMQIO35g6lOi6uf4r34B3eiyp6Z8uZpz 0+mPvBXp+mo5wvs+AUIkM7xqZEmT7UfsE+ahovCiwiFefAM4l+bvzewa8OpTczEzKWTJ hQrVFsMw/dqKBz74Ih7+kqUMo30V7mIQzQOjeobfgXu+xeUrqiQbh2vm8j4jdZbHykud dGJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276550; x=1727881350; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vAC8l3Y8WvFUyajZb+pirsZCUpFvZ3r8+BwrpQuj/SE=; b=rktgIG2cXxmA6eMpAHMArj1qC+9VgdH12bEwWbYhjV2pk4PgJ7DUEKZLDzUmHE/lE+ Ytaaas+GmTw1yRd4GVTn05jhdTlsxkmILLbO6oHfFJl+8VhWUdtdbfB45UwQWc3ARu0u 6FKHMaKzQFq6NnzvaGKovUJHCtB6U1bIXpCcOlPg2sNhE4rw3j8JxI1s5ar9Dw27z+V3 vitlBYXOQiihiPFcQcNgP7Pl9XvIzoaRDsTcYJ162H18Zy57xX4xsNOcXPuJDnku5FD9 sDgshhnJNWp9sbva9/CaW1g6y+Pqvu2c5NxbEfGVUHzxJVF4+ZazIgUrJcbSRxHXp4Q5 wcfQ== X-Forwarded-Encrypted: i=1; AJvYcCWS2ZENuXfESDE+B3EryMN7IQpMMAA+/sDzlrbI/l9tJZTGO8qgWY/SziFYLPCzDgyV0BZizFNmpmI=@vger.kernel.org X-Gm-Message-State: AOJu0YwmFvqX3XbRbcfLxi/xNA3pNCi+XRChJ33cwZibPYm8lExlbXyr G0QQMgQPsgwRehJLh8icx/l8Sg8DFQaGhua7rvKr1hW3vAP2QHKf8vyY2I4MebF+jNLx1A== X-Google-Smtp-Source: AGHT+IGsHQZbBz7v7rjB2VogUNbzT3V2zpNGtAWZsXzYUPDihu4mWF0GF+yn2VIMs1ZNTkRqyJuf4PXN X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:600c:4ca2:b0:42c:acd5:c641 with SMTP id 5b1f17b1804b1-42e96037975mr279665e9.2.1727276550412; Wed, 25 Sep 2024 08:02:30 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:16 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2723; i=ardb@kernel.org; h=from:subject; bh=tMElkct6gfSM6gp7MMWL/mRF1vGpc7RrYvn7ljZWXyE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6iGZPaJMr8LDnd//b/RW8SszXOfdmXXq3NuftZV75 oQv+lfaUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACZy4w/DHw7GS21Or9/PeKL6 7ZX8zCMs6WX7p7QFXu6zKz6oZDhn3S9GhnkFPcEfOSfJKv9IYrKOXfBu+/X/Xl4zqlnSnqz7v02 4kwsA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-46-ardb+git@google.com> Subject: [RFC PATCH 16/28] x86/entry_64: Use RIP-relative addressing From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Fix up a couple of occurrences in the x86_64 entry code where we take the absolute address of a symbol while we could use RIP-relative addressing just the same. This avoids relocation fixups at boot for these quantities. Signed-off-by: Ard Biesheuvel --- arch/x86/entry/calling.h | 9 +++++---- arch/x86/entry/entry_64.S | 12 +++++++----- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index ea81770629ee..099da5aaf929 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -375,8 +375,8 @@ For 32-bit we have the following conventions - kernel is built with .endm .macro SAVE_AND_SET_GSBASE scratch_reg:req save_reg:req + GET_PERCPU_BASE \scratch_reg \save_reg rdgsbase \save_reg - GET_PERCPU_BASE \scratch_reg wrgsbase \scratch_reg .endm @@ -412,15 +412,16 @@ For 32-bit we have the following conventions - kernel is built with * Thus the kernel would consume a guest's TSC_AUX if an NMI arrives * while running KVM's run loop. */ -.macro GET_PERCPU_BASE reg:req +.macro GET_PERCPU_BASE reg:req scratch:req LOAD_CPU_AND_NODE_SEG_LIMIT \reg andq $VDSO_CPUNODE_MASK, \reg - movq __per_cpu_offset(, \reg, 8), \reg + leaq __per_cpu_offset(%rip), \scratch + movq (\scratch, \reg, 8), \reg .endm #else -.macro GET_PERCPU_BASE reg:req +.macro GET_PERCPU_BASE reg:req scratch:req movq pcpu_unit_offsets(%rip), \reg .endm diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 1b5be07f8669..6509e12b6329 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1038,7 +1038,8 @@ SYM_CODE_START(error_entry) movl %ecx, %eax /* zero extend */ cmpq %rax, RIP+8(%rsp) je .Lbstep_iret - cmpq $.Lgs_change, RIP+8(%rsp) + leaq .Lgs_change(%rip), %rcx + cmpq %rcx, RIP+8(%rsp) jne .Lerror_entry_done_lfence /* @@ -1250,10 +1251,10 @@ SYM_CODE_START(asm_exc_nmi) * the outer NMI. */ - movq $repeat_nmi, %rdx + leaq repeat_nmi(%rip), %rdx cmpq 8(%rsp), %rdx ja 1f - movq $end_repeat_nmi, %rdx + leaq end_repeat_nmi(%rip), %rdx cmpq 8(%rsp), %rdx ja nested_nmi_out 1: @@ -1307,7 +1308,8 @@ nested_nmi: pushq %rdx pushfq pushq $__KERNEL_CS - pushq $repeat_nmi + leaq repeat_nmi(%rip), %rdx + pushq %rdx /* Put stack back */ addq $(6*8), %rsp @@ -1346,7 +1348,7 @@ first_nmi: addq $8, (%rsp) /* Fix up RSP */ pushfq /* RFLAGS */ pushq $__KERNEL_CS /* CS */ - pushq $1f /* RIP */ + pushq 1f@GOTPCREL(%rip) /* RIP */ iretq /* continues at repeat_nmi below */ UNWIND_HINT_IRET_REGS 1: From patchwork Wed Sep 25 15:01:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830536 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D05D61922F4 for ; Wed, 25 Sep 2024 15:02:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276560; cv=none; b=bur3Om5I340QM4W2bFOUceFu+UFPtGxd3hn9kN3MPPxqnv+zYM4Zg63axrpRjpsxazh8kVYy2FS7zNUW1vBHj44TnSAuCZo8i/8natmFcwnSS04AZujIQrWHEkZLlNdyOXnUtRU1G/BMnrncFqaqR/C70Pqtjf/4zH4+DTILa58= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276560; c=relaxed/simple; bh=i/QYg4imAjIe1bwTsD55fhiHAflGFWUqPqRf587cmXw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IgzoE777BNWqdW7LPPolRJtmsZ0uanPUnrWUOEGXp4or/E1HjhwjcdE7fWon5gVgnMH4SR+qDKNSaqeWzn77ds6U3fybSmtbNz4xgEjZfCslSF5r3xl2eKuYJ7G9Nbyqtnmbcu+2Qc4XWEI2Xady1IGDbYLrOCKg85RVxBWz10o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=t4Rppe0o; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="t4Rppe0o" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-69a0536b23aso106303237b3.3 for ; Wed, 25 Sep 2024 08:02:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276556; x=1727881356; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yDR79FvO/p7pnSynPl14YMv6NBkHhQBDE9t/OHYGGd4=; b=t4Rppe0oOZrM8rrVvSNlggls1ZRJPpECq26qFnMncuOtFgzJr46mGusEymy3qWdvjG OkmwarU5hNsJqz/Wh5t61KCFoWUz+AuEgMqrOJ1TazD5+6aIFjevSDXJbKGmDSqTX7Ew CCDANtXnw+qfm76PgKJlBQWSB6BVvF38zmAyKZeZifaZ3dAQZc/naSXMoWSrWh54iiZu 19+JCAkmeYxbqwZHm1oMglO4xPvB/zNYVQddPlc5M9ZsGsOYzlPE3GC8tDYY4nhgyeBS TJ5ONJAIqyf9vEuBbIouLLhXZfht7FqE/E141vxkUC/oXnxa2811TyADIndgQmoWmG7P kKUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276556; x=1727881356; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yDR79FvO/p7pnSynPl14YMv6NBkHhQBDE9t/OHYGGd4=; b=Ir66YibovZdhUgcLZS4NKAPA4AGesH1rsldoiAytWXIyNhwwlRxFp+TYKIcFqCvCbR 35jPXh2N+Xl2WzhPxSLY74gprW+d5JSZG1J1skmyZBFgpYAce3Xt0/uHEiEegO6SeThD xe8DDzSN2TTyZFAHuMq8B6am8eFUU+k0Dy8D0AG6HELjbjUPfPPMA15Zae8ABDqDUzXF jyGnwe01Cwfjao8YssJR5j/XFg+N9Ag2sWpKYHDawNXTORNrrbg2RpVAFvvrHjWn54E/ /L3gS1gMhad+yLi/96ddJ7do8oXbrgIndNENidDpHlG/GAWXkv6mtwUjSLwtnJO0fbnt CnQQ== X-Forwarded-Encrypted: i=1; AJvYcCX8td9nhtRZwrWKK2Yha+a/6+YydqbUMZtd38II2uEzaqaiual3AyzvNq7RSHy0N0+FqvtnAYncUus=@vger.kernel.org X-Gm-Message-State: AOJu0YzUlJmW5dDy7fl0OX/1/ZwnYgsk5GyG5ULr2lgZzgdykx3M3Uwg CiSSXfU/668ig0thrV5K0AAvJcDoq4pPsQNSuZ5lygLlTn7/PdBSQQrGPVtlDcvCbHZucQ== X-Google-Smtp-Source: AGHT+IHZ4KYfHMEhV3OEcung24V1cc8XfcFAbHD7O3r+Hgg2lidAI5AomtBdq038iHk4HR3+YtV8+eXL X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a5b:ec8:0:b0:e0b:f93:fe8c with SMTP id 3f1490d57ef6-e24d47abf13mr23895276.0.1727276555668; Wed, 25 Sep 2024 08:02:35 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:18 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4435; i=ardb@kernel.org; h=from:subject; bh=GHJcalcWc5Qb+zlhfpCjetJZ0xMQ4h6Eadch5mSu/kI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6tGDolHZO8I/hR3x3e/2aucUz1XW9pJNyVMcgt6LK d6perupo5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAExkjgkjw6k/n413NQiwSlie DnVk14nq3/xeMf2gfMyRHcx1Ja3nuRj+h2Vr7vj9abJ2U31nzqv2/TMymqLWPwla++d7/1o/hhv mDAA= X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-48-ardb+git@google.com> Subject: [RFC PATCH 18/28] x86/boot/64: Determine VA/PA offset before entering C code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Implicit absolute symbol references (e.g., taking the address of a global variable) must be avoided in the C code that runs from the early 1:1 mapping of the kernel, given that this is a practice that violates assumptions on the part of the toolchain. I.e., RIP-relative and absolute references are expected to produce the same values, and so the compiler is free to choose either. However, the code currently assumes that RIP-relative references are never emitted here. So an explicit virtual-to-physical offset needs to be used instead to derive the kernel virtual addresses of _text and _end, instead of simply taking the addresses and assuming that the compiler will not choose to use a RIP-relative references in this particular case. Currently, phys_base is already used to perform such calculations, but it is derived from the kernel virtual address of _text, which is taken using an implicit absolute symbol reference. So instead, derive this VA-to-PA offset in asm code, using the kernel VA of common_startup_64 (which we already keep in a global variable for other reasons), and pass it to the C startup code. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/setup.h | 2 +- arch/x86/kernel/head64.c | 8 +++++--- arch/x86/kernel/head_64.S | 9 ++++++++- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h index 0667b2a88614..85f4fde3515c 100644 --- a/arch/x86/include/asm/setup.h +++ b/arch/x86/include/asm/setup.h @@ -49,7 +49,7 @@ extern unsigned long saved_video_mode; extern void reserve_standard_io_resources(void); extern void i386_reserve_resources(void); -extern unsigned long __startup_64(unsigned long physaddr, struct boot_params *bp); +extern unsigned long __startup_64(unsigned long p2v_offset, struct boot_params *bp); extern void startup_64_setup_gdt_idt(void); extern void early_setup_idt(void); extern void __init do_early_exception(struct pt_regs *regs, int trapnr); diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index d4398261ad81..de33ac34773c 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -138,12 +138,14 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv * doesn't have to generate PC-relative relocations when accessing globals from * that function. Clang actually does not generate them, which leads to * boot-time crashes. To work around this problem, every global pointer must - * be accessed using RIP_REL_REF(). + * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determined + * by subtracting p2v_offset from the RIP-relative address. */ -unsigned long __head __startup_64(unsigned long physaddr, +unsigned long __head __startup_64(unsigned long p2v_offset, struct boot_params *bp) { pmd_t (*early_pgts)[PTRS_PER_PMD] = RIP_REL_REF(early_dynamic_pgts); + unsigned long physaddr = (unsigned long)&RIP_REL_REF(_text); unsigned long pgtable_flags; unsigned long load_delta; pgdval_t *pgd; @@ -163,7 +165,7 @@ unsigned long __head __startup_64(unsigned long physaddr, * Compute the delta between the address I am compiled to run at * and the address I am actually running at. */ - load_delta = physaddr - (unsigned long)(_text - __START_KERNEL_map); + load_delta = __START_KERNEL_map + p2v_offset; RIP_REL_REF(phys_base) = load_delta; /* Is the address not 2M aligned? */ diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index ab6ccee81493..db71cf64204b 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -99,13 +99,20 @@ SYM_CODE_START_NOALIGN(startup_64) /* Sanitize CPU configuration */ call verify_cpu + /* + * Use the 1:1 physical and kernel virtual addresses of + * common_startup_64 to determine the physical-to-virtual offset, and + * pass it as the first argument to __startup_64(). + */ + leaq common_startup_64(%rip), %rdi + subq 0f(%rip), %rdi + /* * Perform pagetable fixups. Additionally, if SME is active, encrypt * the kernel and retrieve the modifier (SME encryption mask if SME * is active) to be added to the initial pgdir entry that will be * programmed into CR3. */ - leaq _text(%rip), %rdi movq %r15, %rsi call __startup_64 From patchwork Wed Sep 25 15:01:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830535 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AC93192D6A for ; Wed, 25 Sep 2024 15:02:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276564; cv=none; b=SoY5PC4nEyF0r5hQADWbG2k/IsG24vhv2fISRfGxdmXLwvpZPuBu9i96GKPeY7ksaCDSjlUVhzWWbt4vFNjQEdJyIk/qslKMBEW6PKZZcbcfXtB/U7hX3rpAqKpBmgOkuv0wHKEKQNwbSBTVv1mYP2zK8n3UDwawZ3h0Pd983/k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276564; c=relaxed/simple; bh=jP/irMOJlRaYkkcw0c6zTUJt0fVtAuXqgia734+CBv4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hD62Z56J60e47z1wJlAVhXM0HnqydFENNujdIz58xR8im2P2PZaZ8uYrdDgxUaIuKR8RrNFAHFxaIuYp/60JmFfQGZWRGYrtFQsItHfB7Q5GkFs/F6MmWci1X5avAk02mgyzEJyEkh8IMPoPqrbH/4AIl37h9IybQGiPiY8OLxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Wb3LCK7b; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Wb3LCK7b" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-37ccc21ceb1so214421f8f.2 for ; Wed, 25 Sep 2024 08:02:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276558; x=1727881358; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=If2fzGYCGwBnqYyJZ9PtA7MpTD5EDRXIER0Mmi+qoTw=; b=Wb3LCK7ba3HMiyiqoZoWtKhuoZSMqXPKUYnmqyXbXA8sc9bQ7g1Ayt1Hi3pfkhMzDY eXWJNLau0OwPd2pT27ntqDg1AIpkhE4ir33pSpotvLQ6C2zQbBT8gzCzBJ0Jyd9cKdov 9efhTkRNel+3uw17iMxB4v/jEbepMyDxWS22XCw71TJCVYSOvZBAlbU0ggK8hwbrKHz6 UE9d6To4UXIs9nQVln1jcy7+FEhGH1trXfiP/m/GlyiPVRZ+XLlHynakSskC5XJGA/lM D1Af4Ib0poxC1uf/z+7EVimyDRdO+1jymWnke+Qi/CC+dy1RnzBaoi8BjuJ3gzejYLNH mvEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276558; x=1727881358; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=If2fzGYCGwBnqYyJZ9PtA7MpTD5EDRXIER0Mmi+qoTw=; b=UZsUvTZs4UmVZXuUcMMo+u9z40H3y6vgXTTEAv5gcsJ+lM2sB60bg5uLis5d9k4nYw TgdSmZhyT9rdrUStTJbz5RIi74/IBWzYQKuk6txKirYmo+FRfq0Dv72l2libNp0YGNnO nOMJm4LZRtwe/JkPkbQfG6dnAP32hwqeMUuLFs6faLuLFmxWk/wTtUXz5BKPPmCa0OiO 10hWCQ6jsUDmD8pnZwKvGalJaWybkrnm9vu/PjdbzNw1p1iSVhAdn3+tuRdZSZCbRLSq OD9z5rDfKQ3+C0A7Sh+T13T4P922sotcWJGF/SX64REOghgy/b4+BD1F1xwLR85eDL0T swuQ== X-Forwarded-Encrypted: i=1; AJvYcCUxlZZ628/eaEtuBap/EHkD9+BWkKWLsvl+y+gImfR4lcGmg4Zyzqa7piqOBrsGPI81icGUmQI5/rY=@vger.kernel.org X-Gm-Message-State: AOJu0YxSDWzMFU9jhb2F141OQ4tnHcjuuH2cAv+KjM8mZ1R+aMUoOUFQ oTdQpSUxLEaMe5ainEog/C58bzYb7K+/cTgOBU1yzvygcNdyXrvWYfHnihKcv4RfNdaT0g== X-Google-Smtp-Source: AGHT+IGkF5zmfUWW4O6+9JFmmh7xBD2vdGsjZEUVC3p8TPU+fzBQ5QlHTxnNWj2zBWn7gVrNU3EsN2Rx X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a5d:5547:0:b0:378:89be:1826 with SMTP id ffacd0b85a97d-37cc2473cc2mr1714f8f.4.1727276557999; Wed, 25 Sep 2024 08:02:37 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:19 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4681; i=ardb@kernel.org; h=from:subject; bh=1WKyFOMOSweUVoFxWBVIeZQGadqaqXWeT1GGGyj8jAA=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6jGPaQaG+1hu3Z0QvW9R+99QHb2J68ydL88WzUpdH Hf1+4blHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAiex8y/GZ5P9H2XQc705Yd pyrK5txZNU1B4+0LjklBfTe9r2ed6mlhZNiZU/rz52b9K88lGacbreAU2ufVkfLX5bhxnpO8ivb mR0wA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-49-ardb+git@google.com> Subject: [RFC PATCH 19/28] x86/boot/64: Avoid intentional absolute symbol references in .head.text From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel The code in .head.text executes from a 1:1 mapping and cannot generally refer to global variables using their kernel virtual addresses. However, there are some occurrences of such references that are valid: the kernel virtual addresses of _text and _end are needed to populate the page tables correctly, and some other section markers are used in a similar way. To avoid the need for making exceptions to the rule that .head.text must not contain any absolute symbol references, derive these addresses from the RIP-relative 1:1 mapped physical addresses, which can be safely determined using RIP_REL_REF(). Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head64.c | 30 ++++++++++++-------- 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index de33ac34773c..49e8ba1c0d34 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -91,9 +91,11 @@ static inline bool check_la57_support(void) return true; } -static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) +static unsigned long __head sme_postprocess_startup(struct boot_params *bp, + pmdval_t *pmd, + unsigned long p2v_offset) { - unsigned long vaddr, vaddr_end; + unsigned long paddr, paddr_end; int i; /* Encrypt the kernel and related (if SME is active) */ @@ -106,10 +108,10 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv * attribute. */ if (sme_get_me_mask()) { - vaddr = (unsigned long)__start_bss_decrypted; - vaddr_end = (unsigned long)__end_bss_decrypted; + paddr = (unsigned long)&RIP_REL_REF(__start_bss_decrypted); + paddr_end = (unsigned long)&RIP_REL_REF(__end_bss_decrypted); - for (; vaddr < vaddr_end; vaddr += PMD_SIZE) { + for (; paddr < paddr_end; paddr += PMD_SIZE) { /* * On SNP, transition the page to shared in the RMP table so that * it is consistent with the page table attribute change. @@ -118,11 +120,11 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv * mapping (kernel .text). PVALIDATE, by way of * early_snp_set_memory_shared(), requires a valid virtual * address but the kernel is currently running off of the identity - * mapping so use __pa() to get a *currently* valid virtual address. + * mapping so use the PA to get a *currently* valid virtual address. */ - early_snp_set_memory_shared(__pa(vaddr), __pa(vaddr), PTRS_PER_PMD); + early_snp_set_memory_shared(paddr, paddr, PTRS_PER_PMD); - i = pmd_index(vaddr); + i = pmd_index(paddr - p2v_offset); pmd[i] -= sme_get_me_mask(); } } @@ -146,6 +148,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset, { pmd_t (*early_pgts)[PTRS_PER_PMD] = RIP_REL_REF(early_dynamic_pgts); unsigned long physaddr = (unsigned long)&RIP_REL_REF(_text); + unsigned long va_text, va_end; unsigned long pgtable_flags; unsigned long load_delta; pgdval_t *pgd; @@ -172,6 +175,9 @@ unsigned long __head __startup_64(unsigned long p2v_offset, if (load_delta & ~PMD_MASK) for (;;); + va_text = physaddr - p2v_offset; + va_end = (unsigned long)&RIP_REL_REF(_end) - p2v_offset; + /* Include the SME encryption mask in the fixup value */ load_delta += sme_get_me_mask(); @@ -232,7 +238,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset, pmd_entry += sme_get_me_mask(); pmd_entry += physaddr; - for (i = 0; i < DIV_ROUND_UP(_end - _text, PMD_SIZE); i++) { + for (i = 0; i < DIV_ROUND_UP(va_end - va_text, PMD_SIZE); i++) { int idx = i + (physaddr >> PMD_SHIFT); pmd[idx % PTRS_PER_PMD] = pmd_entry + i * PMD_SIZE; @@ -257,11 +263,11 @@ unsigned long __head __startup_64(unsigned long p2v_offset, pmd = &RIP_REL_REF(level2_kernel_pgt)->pmd; /* invalidate pages before the kernel image */ - for (i = 0; i < pmd_index((unsigned long)_text); i++) + for (i = 0; i < pmd_index(va_text); i++) pmd[i] &= ~_PAGE_PRESENT; /* fixup pages that are part of the kernel image */ - for (; i <= pmd_index((unsigned long)_end); i++) + for (; i <= pmd_index(va_end); i++) if (pmd[i] & _PAGE_PRESENT) pmd[i] += load_delta; @@ -269,7 +275,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset, for (; i < PTRS_PER_PMD; i++) pmd[i] &= ~_PAGE_PRESENT; - return sme_postprocess_startup(bp, pmd); + return sme_postprocess_startup(bp, pmd, p2v_offset); } /* Wipe all early page tables except for the kernel symbol map */ From patchwork Wed Sep 25 15:01:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830534 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60208198A0D for ; Wed, 25 Sep 2024 15:02:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276571; cv=none; b=jta8gMTfyHQQ0MT0OiFGXCe9F5JVP0ewE+nvaG510jSOvw+TemzXsOp/8eyzfuZuB/o/WojKdMQ0TtcfASKDodIZnx/6sfilm8PB9CuXairI2x4qWElWbCXRi1f6HaBXTDvxKLkEEAZvO5ifHARO+jWpSiSQ7HjVnnMFQ+cbpCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276571; c=relaxed/simple; bh=4t4Zd7d88HUB4IQR7hyWqg3RF4QpMaNzLYFQ0q/SbWM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Sof7DKK/oSANXLxclLMPhjSAVI0+w+CYaDkZgQ1pMsW5/auZ8wLGObtMKpGShtPeq+ccueGA+b/DXqpq6mIDrqxnQ/RRWIj0lqaR4ySdAwXQ+oofElrYOqnpnfi90Jyg9EONI6mY/a+4762VA5ZYEagWLr0gimbaqWRMNbFpLgk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xpdc7b0r; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xpdc7b0r" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-3780c8d689bso3486774f8f.2 for ; Wed, 25 Sep 2024 08:02:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276567; x=1727881367; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pNx8CgL2JjB1FACn8gZOCDgH3XlG/cZ7BMhQN82yM0M=; b=xpdc7b0reBbwnnjkdvySuBhqxCUup4ROen4arCBet5tBUn/jAPZrevZ0XydaQWVEdR NptfvC2h5L3r3+buanOYePlkab/2vDQbYR/I2i9wuWm4TvkCnKkf9vNjvdgt80uJ8+SW iLr4Wbpv+HxRref707wqaz2SoYCW0MfU9uJo8GCadxjy+7NiRAnESsNrx2pnflAkCsms o7NuB+N+ESfhIaXEtIqiTN+f95AS/6k7Spad2NqYoV//cdCFL0JwjorjV7Fhr1bPSoQ6 6GKDsf/GGwmOOxvTmd0CsCY8/y/kVK0718CGugnH/HehdzmKjcWznuNpm26kPgWMkSJ9 d2Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276567; x=1727881367; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pNx8CgL2JjB1FACn8gZOCDgH3XlG/cZ7BMhQN82yM0M=; b=bX8kwYynrDKRA4wkJgjdc5htE21K25Vu+F6XZoRJZXSsJEGeP3VrqJCljsyTUkdPu7 qysGUj4apJPsQeWDqJpUcs2OJsDfPb+l7LghY1fsVBZ7D2+43+0yrL0t1DU3v4/zMc5f L73iviLp+6v00DjyL11dcI/vCqs6LkfnA8JurB3zgbkw1e1uyNz4hmKJqd0jRbbIXSs4 w9/s5htLiD9B5tKkZ0O1RmqmaxLnsbzSX28rentaneCqtIsqnjA7rOrD8NXkgu63DifC pkkyCa5uLwrjO77R25K1EOiADdHxRp5IxFUfMxCQe5EaZ27SwCBcco+iY9ycQwKkLnur SoAQ== X-Forwarded-Encrypted: i=1; AJvYcCWAYp0c/lI+lp9cve7A7vgaC6A9QhCukD/n58ZAe3dCo/2fUHgGjoli3b2AdstFrPrBic0REaBsq4U=@vger.kernel.org X-Gm-Message-State: AOJu0Yz/Cvh2XvPMt4THtn/f0Dcud6FiXTfeb8rBcDyb3xYG3/kHoz0r 4n0e+W8voBUq1LI4p+wD5eHafCAq7PmPaKf4jhJDMmCzJiJbzZpWdYgsSJDXz6nLDSQCMQ== X-Google-Smtp-Source: AGHT+IHtZA65C9eNefhgBWqmYh0kGm6u7UMHidCQ+DZrqbEOemMqvMt90MEtVjIphkQcIFg1qUNObHvF X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a5d:5e04:0:b0:37c:c51b:d07f with SMTP id ffacd0b85a97d-37cc51bd43bmr945f8f.9.1727276566478; Wed, 25 Sep 2024 08:02:46 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:22 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4300; i=ardb@kernel.org; h=from:subject; bh=LZda20wnQiwQiTXt6tiHm30A1wODb0KBnxm/teE+Lu4=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6unSO+7/NsayfzpQ7uW+6/sf035bm8SjMXouItt6v vsqNLN3lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIlUFzIyLClfembn6Wlfb79Z nLiyrmmade3VvmNH2xq0FwTP8FnOwsHwmyXhPONLDfu9MQfFQ+MyCsJWuy/VFOpa8t9x77FzLsv /8AAA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-52-ardb+git@google.com> Subject: [RFC PATCH 22/28] asm-generic: Treat PIC .data.rel.ro sections as .rodata From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel When running the compiler in PIC/PIE mode, it will emit data objects that are 'const' in the context of the program into the .data.rel.ro section if they contain absolute addresses of statically allocated global objects. This helps the dynamic loader distinguish between objects that are truly const from objects that will need to be fixed up by the loader before starting the program. This is not a concern for the kernel, but it does mean those .data.rel.ro input sections need to be handled. So treat them as .rodata. It also means some explicit uses of .rodata for global structures containing absolute addresses need to be changed to .data.rel.ro to prevent the linker from warning about incompatible section flags. Signed-off-by: Ard Biesheuvel --- include/asm-generic/vmlinux.lds.h | 2 +- include/linux/compiler.h | 2 +- scripts/kallsyms.c | 2 +- tools/objtool/check.c | 11 ++++++----- tools/objtool/include/objtool/special.h | 2 +- 5 files changed, 10 insertions(+), 9 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index cc14d780c70d..2b079f73820f 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -456,7 +456,7 @@ . = ALIGN((align)); \ .rodata : AT(ADDR(.rodata) - LOAD_OFFSET) { \ __start_rodata = .; \ - *(.rodata) *(.rodata.*) \ + *(.rodata .rodata.* .data.rel.ro*) \ SCHED_DATA \ RO_AFTER_INIT_DATA /* Read only after init */ \ . = ALIGN(8); \ diff --git a/include/linux/compiler.h b/include/linux/compiler.h index ec55bcce4146..f7c48b7c0a6b 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -133,7 +133,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, #define annotate_unreachable() __annotate_unreachable(__COUNTER__) /* Annotate a C jump table to allow objtool to follow the code flow */ -#define __annotate_jump_table __section(".rodata..c_jump_table") +#define __annotate_jump_table __section(".data.rel.ro.c_jump_table") #else /* !CONFIG_OBJTOOL */ #define annotate_reachable() diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 9c34b9397872..1700e97400aa 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -357,7 +357,7 @@ static void write_src(void) printf("#define ALGN .balign 4\n"); printf("#endif\n"); - printf("\t.section .rodata, \"a\"\n"); + printf("\t.section .data.rel.ro, \"a\"\n"); output_label("kallsyms_num_syms"); printf("\t.long\t%u\n", table_cnt); diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 01237d167223..04725bd83232 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -2575,15 +2575,16 @@ static void mark_rodata(struct objtool_file *file) * Search for the following rodata sections, each of which can * potentially contain jump tables: * - * - .rodata: can contain GCC switch tables - * - .rodata.: same, if -fdata-sections is being used - * - .rodata..c_jump_table: contains C annotated jump tables + * - .rodata .data.rel.ro : can contain GCC switch tables + * - .rodata. .data.rel.ro. : same, if -fdata-sections is being used + * - .data.rel.ro.c_jump_table : contains C annotated jump tables * * .rodata.str1.* sections are ignored; they don't contain jump tables. */ for_each_sec(file, sec) { - if (!strncmp(sec->name, ".rodata", 7) && - !strstr(sec->name, ".str1.")) { + if ((!strncmp(sec->name, ".rodata", 7) && + !strstr(sec->name, ".str1.")) || + !strncmp(sec->name, ".data.rel.ro", 12)) { sec->rodata = true; found = true; } diff --git a/tools/objtool/include/objtool/special.h b/tools/objtool/include/objtool/special.h index 86d4af9c5aa9..89ee12b1a138 100644 --- a/tools/objtool/include/objtool/special.h +++ b/tools/objtool/include/objtool/special.h @@ -10,7 +10,7 @@ #include #include -#define C_JUMP_TABLE_SECTION ".rodata..c_jump_table" +#define C_JUMP_TABLE_SECTION ".data.rel.ro.c_jump_table" struct special_alt { struct list_head list; From patchwork Wed Sep 25 15:01:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830533 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FC45199FBA for ; Wed, 25 Sep 2024 15:02:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276575; cv=none; b=UrgTTJDzuTOBlbTRJn2YYdnzT6g1IcHVIhOQUJfMFGMAaehMKzo3kLcYrqNyctO3D2DnAnFHSeztayanYoAi6cLYx2EDwRN30BJ1Jh6zNTUCiJaKIb272HlzLWn2pErU0XwCl9KowGVeDFDxj1+fCUiBmq1W8CsUYyf11APuX88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276575; c=relaxed/simple; bh=31w9JHpb/FOSX9EXfChPlaKIxGcZxzD1C7F5EPjfIx4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QwnKqkgjGWreNuFsC9IWZSkxhz4NJhMNNY4xsxHUpR9nxslO4jrHtxUWUYg5spbg+T35SlsiGq1qUoVQLOacczLF2BAbRfY2D77Sfmuv1kH8++MfMQGtm7l+KjwY2Ysa+kL+KzdLPI893EYZNZkACRqy8VcGNLCNyHqEYQq1z0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rXWec9Qx; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rXWec9Qx" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2261adfdeso8799427b3.2 for ; Wed, 25 Sep 2024 08:02:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276571; x=1727881371; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=q7Siui3EQaHZbOP2Zf5u6bj1Z6F9xVc5htz8ViId7Yw=; b=rXWec9QxGp9iLyg3M9A0z666zG+oP1PcSihhjizChPy05vnNcsyx9xLL/z4hX6lSef qFy4BSKuzWSFawXI68+XsubjKO+eet/40TIgqZ7RsfGZS986AuWvF8nKN5qpiFKslsmO 2bArQxQb3mhxvvbCgKFbAX7uhsCXllG5X1zigVUxzuu6x5bX2y0QgHhJ8KJxprmV7B07 1T1VVmdSq3ytxkjpX8wNbaWviGn7gBt+HOuYUzOBw3QVk5wbVkr97BL/tcUWZ1Mjf+Hh KBv/YdE7rb1fZQmYY8Dofb7w+CogKyoRXNS8KyaxBsHDBW0ay0LD+K2ehinLZWsJxlgN zhew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276571; x=1727881371; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=q7Siui3EQaHZbOP2Zf5u6bj1Z6F9xVc5htz8ViId7Yw=; b=nWauf/XUL1Brqyu27iFhfminCwOp19gT4y8g9iA/OyPqTCnYW55yLHpRS6YIOrNjpQ Oee+p7U1MZDNyoKsOk/z3oZDKAFT5nCjezqEPgFrFt8FbpdRV/U2/eksqXqTTHptrDmU TuSQRV/VCrF/7tEJ3xGvmvy7LgMPexhz3RInKfFXW7ElsXLSAfXgm9Tfu1WirePwjRsv X7FhkAxxNHzU6Mkp88ti27UvqLIeCxmnyKe9J6XFi8iMWjlzYRH8oMFbUTXk/pKUUmiO vyaW0+/j0ZgTJMnCqegnrpa4EznA+0fjnn5nMNpKee5u098un/6WN4hHghli6tqMcTUo 2aAQ== X-Forwarded-Encrypted: i=1; AJvYcCUjDufRxAxz/ynj+mnfM79aNsguryQkVcKX7wy+Ier0QlDtGZWvFpwmxJNTzEfXn57U2Td0vI8aohc=@vger.kernel.org X-Gm-Message-State: AOJu0YxF9R3jDhFNOKFYFzAZflimn1e39MRKZF7/Jhc6VAvBVoQp07hS BIlmIlboVQfi9R1rVzLR4TrgHWJT3gwiS6YvVHQFM1+mxu/ECYamFlbh8thc7VfRYmdjbw== X-Google-Smtp-Source: AGHT+IGUGK/IbClqwN6y6QvmI8hCfQn6s2hVSiVHB1d+dhN6jVJxxhzTiRxXUiYwgEqTcxKFspPDWKFo X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:690c:f90:b0:6c1:298e:5a7 with SMTP id 00721157ae682-6e21d9f2676mr100587b3.5.1727276571410; Wed, 25 Sep 2024 08:02:51 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:24 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2378; i=ardb@kernel.org; h=from:subject; bh=s5eg4POBUMz0+xutAa57cnb7wChxapKHTEsRfaj5xkM=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6rmrbx/K9pd0n9poZ7rKarM71/pb64Mu3OE7WFCew +trFZHXUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACYiIMfIsHzp8ptxntWap0qz /54LMvVhTo1RvbxcftLGA6VFgfY+Jxn+B/cyx9dHTT/wO+rQQaXszxPMFk1geHr9aqrVTv/NPxI dmAE= X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-54-ardb+git@google.com> Subject: [RFC PATCH 24/28] tools/objtool: Treat indirect ftrace calls as direct calls From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel In some cases, the compiler may rely on indirect calls using GOT slots as memory operands to emit function calls. This leaves it up to the linker to relax the call to a direct call if possible, i.e., if the destination address is known at link time and in range, which may not be the case when building shared libraries for user space. On x86, this may happen when building in PIC mode with ftrace enabled, and given that vmlinux is a fully linked binary, this relaxation is always possible, and therefore mandatory per the x86_64 psABI. This means that the indirect calls to __fentry__ that are observeable in vmlinux.o will have been converted to direct calls in vmlinux, and can be treated as such by objtool. Signed-off-by: Ard Biesheuvel --- tools/objtool/check.c | 32 ++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 04725bd83232..94a56099e22d 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -1696,11 +1696,39 @@ static int add_call_destinations(struct objtool_file *file) struct reloc *reloc; for_each_insn(file, insn) { - if (insn->type != INSN_CALL) + if (insn->type != INSN_CALL && + insn->type != INSN_CALL_DYNAMIC) continue; reloc = insn_reloc(file, insn); - if (!reloc) { + if (insn->type == INSN_CALL_DYNAMIC) { + if (!reloc) + continue; + + /* + * GCC 13 and older on x86 will always emit the call to + * __fentry__ using a relaxable GOT-based symbol + * reference when operating in PIC mode, i.e., + * + * call *0x0(%rip) + * R_X86_64_GOTPCRELX __fentry__-0x4 + * + * where it is left up to the linker to relax this into + * + * call __fentry__ + * nop + * + * if __fentry__ turns out to be DSO local, which is + * always the case for vmlinux. Given that this + * relaxation is mandatory per the x86_64 psABI, these + * calls can simply be treated as direct calls. + */ + if (arch_ftrace_match(reloc->sym->name)) { + insn->type = INSN_CALL; + add_call_dest(file, insn, reloc->sym, false); + } + + } else if (!reloc) { dest_off = arch_jump_destination(insn); dest = find_call_destination(insn->sec, dest_off); From patchwork Wed Sep 25 15:01:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830532 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DE5A19CC13 for ; Wed, 25 Sep 2024 15:02:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276579; cv=none; b=WXc+OjAdHNlQYWZTRZrAWgqsIRJyarYZpzD6IdziOLVoMj+3Q2bqjlLWRlDT/NDM/qLhQbKfI1KZzErMDT+htVBIJb+S3bz5E4C7+cnZpJuOSavSGa76rXthMSWAAhpmb/39xLx8xcmV5wmz9Wan6YNXSvkB6EvG6qtFb98N7rA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276579; c=relaxed/simple; bh=nfca0ivNA/FjKhvJ9VN6/KPVpcFIyJuoOSLlvvE4PM8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F1wiQxmmdFyP7tBhUjSgY+Wz4UTmD3se0GBr7Nkt/yscv/Hm/cl1QJxWgHp/eSAFuhYRITHI5uUGvC0xJj9gInSiSOoopRTx8WDJ/2oLIIXAl6FkhTQO0ioAu6JV29MJ9WuSbeLxxqQEzK+FM913JxEnr4YQ9PtGLT4+bRMt/lk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Ok3a9T2p; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Ok3a9T2p" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6886cd07673so105562977b3.3 for ; Wed, 25 Sep 2024 08:02:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276576; x=1727881376; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NyUlP50R7ff/vN5G1h18De2z2GbJesVTam/AShMXJgw=; b=Ok3a9T2p3ibVbaGa+iDQfUanzp87DDbsL94ZqKhNs+9c/Ud49sfDXpQt7Gfb2EsaF/ gadpvilcZAGklooS3T6lStXbODnIoBKO//bCZZ+X05Ob6tVi0ppABAQAGaZ+8D794/4t 3TTaMkvaXSIa48ycsJ6pcun5rvO9tRyLpAtg0JQ9JaRR2BcGnuSlGcmraLwytMSpJRsH MgmKJw+S99Y2SAhHH0sa/Tu1sdGcbXjPM7tXeYLo48J66lzOuXkH0aFHeeYwApvuN70s FLGcT96FtMrmdt6ioLGnl7xmAQxCDpnpdLprlxFpPpPQj6UhZUoVM7Xq9uPkoC6TdYvs rIVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276576; x=1727881376; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NyUlP50R7ff/vN5G1h18De2z2GbJesVTam/AShMXJgw=; b=T3ICov/61dsgjxzrvZwqKIZAj6+qTdxOQxYekQ7gUcZu6fJ7GctjjdJU09KbWaFjbL UGvWSs2s24Iw+5sljaJgCTUnBcytpVqjki00WyZ9tXS+actzCj48MHQxWi22VgWLqKm4 6uN9WWmW2vJBDN7mE20qcnjcsullNjXYI6QxqHP6o/dGn/F20cV90rl4mI1GQUgC8F1g b/vJEDg19JvW67yEnpZwaD/4qJWWSCN/8HWB6N4Fv22kv0uHIez2wLvNGHq6sV2bIT1/ 1mCwKSjwzYqI/DWqGKaaGg3DckcTX9/Ux+syeXQBtqCVcSwpBXIQzJBTtenLbwSmj0T8 aMMw== X-Forwarded-Encrypted: i=1; AJvYcCUpgJszmy3vcrWThOsBA88jAJfsLKpVt/l0xvQMZ0QNj4CWdGY0fdEedEgxZXD9X8dMykKzCOJiuLE=@vger.kernel.org X-Gm-Message-State: AOJu0YzF/XqNaGINGDCM7K63ZPKPmppDKHLF3XGVus7N2dqsiP3BiuuL NRnFaXi65g4SuDcn5MvCQkJB3aIXQxxtenzbVhJoS8/Byh13KIkg/W+Cvp9etqsG6kdYqQ== X-Google-Smtp-Source: AGHT+IF9cT+1HnKGMG9rCL4XBpFAiRqL2rxQrsa7AeT9AU3GHDLsQi/G2t9Fg7OjYNkaCpx/AYMRBr5P X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:690c:3149:b0:6de:19f:34d7 with SMTP id 00721157ae682-6e21d81e811mr94787b3.2.1727276576540; Wed, 25 Sep 2024 08:02:56 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:26 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=7472; i=ardb@kernel.org; h=from:subject; bh=ZjnajWIKcY5DrOh/WpJS8LufZBk9grHctEdumwSdFZ4=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6oXaCAm2n4qhvN+cWPhMgzpaNBxMv/te55wX/7Z/7 fXGH00dpSwMYhwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCJp0owMs7Y8P9MQNOWv4+0U 10OXF7hWP92jueKnm9Tq9WavZtsdSmX4Z/Uk0GTGvv61f15k371aLH96espP9/0Kyz6kZF4+IJv 9hR8A X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-56-ardb+git@google.com> Subject: [RFC PATCH 26/28] x86/boot: Implement support for ELF RELA/RELR relocations From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel Add support for standard dynamic ELF relocations to perform the virtual relocation of the core kernel at boot. The RELR format results in a 10x reduction in memory footprint of the relocation data, and can be generated by the linker directly. This removes the need for a) a host tool 'relocs' and a bespoke, clunky relocation table format where the table is simply concatenated to the vmlinux payload when building the decompressor; b) dependence on the --emit-relocs linker switch, which dumps static, intermediate build time relocations into the ELF binary, to be subsequently used as runtime relocations. The latter is especially problematic, as linkers may apply relaxations that result in the code going out of sync with the static relocation that annotated it in the input. This requires additional work on the part of the linker to update the static relocation, which is not even possible in all cases. Therefore, it is much better to consume a runtime, dynamic relocation format in the way it was intended. This will require switching to linking vmlinux in PIE mode - this is implemented in a subsequent patch. Signed-off-by: Ard Biesheuvel --- Documentation/arch/x86/zero-page.rst | 3 +- arch/x86/Kconfig | 1 + arch/x86/include/asm/setup.h | 1 + arch/x86/include/uapi/asm/bootparam.h | 2 +- arch/x86/kernel/head64.c | 36 ++++++++++++++++++++ arch/x86/kernel/head_64.S | 5 +++ arch/x86/kernel/vmlinux.lds.S | 24 +++++++++---- 7 files changed, 64 insertions(+), 8 deletions(-) diff --git a/Documentation/arch/x86/zero-page.rst b/Documentation/arch/x86/zero-page.rst index 45aa9cceb4f1..fd18b77113e2 100644 --- a/Documentation/arch/x86/zero-page.rst +++ b/Documentation/arch/x86/zero-page.rst @@ -3,7 +3,7 @@ ========= Zero Page ========= -The additional fields in struct boot_params as a part of 32-bit boot +The additional fields in struct boot_params as a part of 32/64-bit boot protocol of kernel. These should be filled by bootloader or 16-bit real-mode setup code of the kernel. References/settings to it mainly are in:: @@ -20,6 +20,7 @@ Offset/Size Proto Name Meaning 060/010 ALL ist_info Intel SpeedStep (IST) BIOS support information (struct ist_info) 070/008 ALL acpi_rsdp_addr Physical address of ACPI RSDP table +078/008 64-bit kaslr_va_shift Virtual kASLR displacement of the core kernel 080/010 ALL hd0_info hd0 disk parameter, OBSOLETE!! 090/010 ALL hd1_info hd1 disk parameter, OBSOLETE!! 0A0/010 ALL sys_desc_table System description table (struct sys_desc_table), diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 2852fcd82cbd..54cb1f14218b 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -26,6 +26,7 @@ config X86_64 depends on 64BIT # Options that are inherently 64-bit kernel only: select ARCH_HAS_GIGANTIC_PAGE + select ARCH_HAS_RELR select ARCH_SUPPORTS_INT128 if CC_HAS_INT128 select ARCH_SUPPORTS_PER_VMA_LOCK select ARCH_SUPPORTS_HUGE_PFNMAP if TRANSPARENT_HUGEPAGE diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h index 85f4fde3515c..a4d7dd81f773 100644 --- a/arch/x86/include/asm/setup.h +++ b/arch/x86/include/asm/setup.h @@ -51,6 +51,7 @@ extern void reserve_standard_io_resources(void); extern void i386_reserve_resources(void); extern unsigned long __startup_64(unsigned long p2v_offset, struct boot_params *bp); extern void startup_64_setup_gdt_idt(void); +extern void startup_64_apply_relocations(struct boot_params *bp); extern void early_setup_idt(void); extern void __init do_early_exception(struct pt_regs *regs, int trapnr); diff --git a/arch/x86/include/uapi/asm/bootparam.h b/arch/x86/include/uapi/asm/bootparam.h index 9b82eebd7add..3389b1be234c 100644 --- a/arch/x86/include/uapi/asm/bootparam.h +++ b/arch/x86/include/uapi/asm/bootparam.h @@ -120,7 +120,7 @@ struct boot_params { __u64 tboot_addr; /* 0x058 */ struct ist_info ist_info; /* 0x060 */ __u64 acpi_rsdp_addr; /* 0x070 */ - __u8 _pad3[8]; /* 0x078 */ + __u64 kaslr_va_shift; /* 0x078 */ __u8 hd0_info[16]; /* obsolete! */ /* 0x080 */ __u8 hd1_info[16]; /* obsolete! */ /* 0x090 */ struct sys_desc_table sys_desc_table; /* obsolete! */ /* 0x0a0 */ diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 49e8ba1c0d34..6609e1012f2f 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include @@ -588,3 +589,38 @@ void __head startup_64_setup_gdt_idt(void) startup_64_load_idt(handler); } + +#ifdef CONFIG_RELOCATABLE +void __head startup_64_apply_relocations(struct boot_params *bp) +{ + extern const Elf64_Rela __rela_start[], __rela_end[]; + extern const u64 __relr_start[], __relr_end[]; + u64 va_offset = (u64)RIP_REL_REF(_text) - __START_KERNEL; + u64 va_shift = bp->kaslr_va_shift; + u64 *place = NULL; + + if (!va_shift) + return; + + for (const Elf64_Rela *r = __rela_start; r < __rela_end; r++) { + if (ELF64_R_TYPE(r->r_info) != R_X86_64_RELATIVE) + continue; + + place = (u64 *)(r->r_offset + va_offset); + *place += va_shift; + } + + for (const u64 *rel = __relr_start; rel < __relr_end; rel++) { + if ((*rel & 1) == 0) { + place = (u64 *)(*rel + va_offset); + *place++ += va_shift; + continue; + } + + for (u64 *p = place, r = *rel >> 1; r; p++, r >>= 1) + if (r & 1) + *p += va_shift; + place += 63; + } +} +#endif diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index cc2fec3de4b7..88cdc5a0c7a3 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -74,6 +74,11 @@ SYM_CODE_START_NOALIGN(startup_64) cdq wrmsr +#ifdef CONFIG_RELOCATABLE + movq %r15, %rdi + call startup_64_apply_relocations +#endif + call startup_64_setup_gdt_idt /* Now switch to __KERNEL_CS so IRET works reliably */ diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 52b8db931d0f..f7e832c2ac61 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -240,6 +240,18 @@ xen_elfnote_phys32_entry_offset = :init #endif + .init.rela : { + __rela_start = .; + *(.rela.*) *(.rela_*) + __rela_end = .; + } + + .init.relr : { + __relr_start = .; + *(.relr.*) + __relr_end = .; + } + /* * Section for code used exclusively before alternatives are run. All * references to such code must be patched out by alternatives, normally @@ -469,12 +481,6 @@ xen_elfnote_phys32_entry_offset = *(.got) *(.igot.*) } ASSERT(SIZEOF(.got) == 0, "Unexpected GOT entries detected!") -#endif - - .plt : { - *(.plt) *(.plt.*) *(.iplt) - } - ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!") .rel.dyn : { *(.rel.*) *(.rel_*) @@ -485,6 +491,12 @@ xen_elfnote_phys32_entry_offset = *(.rela.*) *(.rela_*) } ASSERT(SIZEOF(.rela.dyn) == 0, "Unexpected run-time relocations (.rela) detected!") +#endif + + .plt : { + *(.plt) *(.plt.*) *(.iplt) + } + ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!") } /* From patchwork Wed Sep 25 15:01:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 830531 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0459619FA66 for ; Wed, 25 Sep 2024 15:03:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276585; cv=none; b=G+h7IIpBuPjc1FwJY07pkzvyOwKRwEbcIBM+/QBRFJpP001RDoI9XXRVr5kKVaY1CutSQRN1ZYcDnIfRHPFd2ACbSTeo/rzMCjHOaA/JHkVQlvR3OA6Dxk25imN9tYmnTqcCcmQWXM7m2iy3lgs4XUXO+7zEEDTpBmaWgy4Gq3Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727276585; c=relaxed/simple; bh=YvM9TRdamHMMVDaQnr/PY1VL/3cpzPBrLHBshWo7LWw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gW2Ocd9oJIZCDzI/NFCA9O2feBc9PrJSsgHCdVWJJLlCvpliriupEym7bNSpvoJ8hDnqI7ENhiSIIE7xiKrHIpvTNMOo59u589LaR0hH6yVCBYNaDURcdeagJxqGgle7kbAcZLaEIv9Rzi+d2JDnWfV1Q1X4pqu2F8ASKFlofeM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ongWkgeN; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ongWkgeN" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-42ac185e26cso52460375e9.3 for ; Wed, 25 Sep 2024 08:03:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727276581; x=1727881381; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=H/fxT1DdHMwMyeW8np4BBTWW8phQCD6XQ4SETP482c8=; b=ongWkgeNEUsXO/yFix7MO/NjZ2xvYjoDtIUAZporkb9MuWeyH9GspZlyxNuptzMS9z 27Wcx+k6toKj75Lqn7eTnYkTBw2H2B/ooJKtEmb4PCjaA7tabdO0KaZ0MASZ2VwCe6tF dCZWRxhB0Df0qIrum8EqqV+z13+lw9hhfUWtEGwpL0BsUWnrImwdQkmrlVuWFiGfgQHl eZS6XOrp4ju9m/p48G2u5gOy3ga2/DqLQRrgaaVl4yJsa7a2bT7kHX5MYA4LyuWo4zSD gRzCcLC+XGFxdW9bdOm28v1dXeZRvGrlwwA3ODwA2n6vokwpU4qtgvNJ+5kk0Dpe76FB qzQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727276581; x=1727881381; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=H/fxT1DdHMwMyeW8np4BBTWW8phQCD6XQ4SETP482c8=; b=larFswLfsbCVf7jX4e0ItGYQ2tQM0kG9DY0PiPtY3yRxBcdzmJlJicZ7sLdv48TA2J YClOtvhrGNp7w3WBkIJCNhSbI1jisy26dHmVXVLC8aY/Gib63GuzAT2tw5eDY2z1/12T y/mkG0aewo+jzqTem+3ZMDkdNuSh/K5IXrDjgR9Ufx/R06SlWgOlp811mte0rPjYcO4T gUnjVeGMOYaCYBqFIdD2VYxJFA15TCtwqju8Z647lg3AcsLPvCRGInL2R6gG9TBtdUx1 RPKKI2y6a69LEvRz2NgN0jtyP/HvnR+YE6mPa+kvcatvsV4EnVtEveF8113cSn/qN82O wEzQ== X-Forwarded-Encrypted: i=1; AJvYcCVRauL0dsuw5dK08Xnr2sGeiNqXh67rPh6s8uGgd9oJGNgwO4IHF+v8dHlbzsE0sQtxm04FxZIc7fU=@vger.kernel.org X-Gm-Message-State: AOJu0YyF1dzC6/cAByeAJ4JxY6p13xNuvT5KrpZmLJtoyPb+5FXLSvOU oMhsct0dM+SVqsSn8WukIkrqMtBKwVUvAL+tiL8sEfWeUq9ujNonm0QZRSTbHaPKkUC0ug== X-Google-Smtp-Source: AGHT+IH1WYrW/UKZ6ngnkIpIsR2dgkaJ0zFY0ft3XgXQ0QWdCfqsR4NIFZKbE/zMO6K3p2uoWgwYBDA5 X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:7b:198d:ac11:8138]) (user=ardb job=sendgmr) by 2002:a05:600c:5709:b0:42c:a879:3d0f with SMTP id 5b1f17b1804b1-42e960af3c0mr226185e9.0.1727276581262; Wed, 25 Sep 2024 08:03:01 -0700 (PDT) Date: Wed, 25 Sep 2024 17:01:28 +0200 In-Reply-To: <20240925150059.3955569-30-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-efi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240925150059.3955569-30-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=11373; i=ardb@kernel.org; h=from:subject; bh=vOsRO/wusTOgSIMrTbPQkuvbTEvedgqm5gFKQyyQ6LE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIe2L6uU7KcXh52pbo2UXaeZq6U587fAt6o/qk6KOindLu xmSJDk6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwES8vjP8D6r8/Whqj0Ue89/g jLlev1knyS9cOWlNtI5Y6rLiT9GiiYwMF5pPLt85df5ck61Kskef/1wi37sr+6Xjrf+r3xWdTEj U5wUA X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240925150059.3955569-58-ardb+git@google.com> Subject: [RFC PATCH 28/28] x86/tools: Drop x86_64 support from 'relocs' tool From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , x86@kernel.org, "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Uros Bizjak , Dennis Zhou , Tejun Heo , Christoph Lameter , Mathieu Desnoyers , Paolo Bonzini , Vitaly Kuznetsov , Juergen Gross , Boris Ostrovsky , Greg Kroah-Hartman , Arnd Bergmann , Masahiro Yamada , Kees Cook , Nathan Chancellor , Keith Packard , Justin Stitt , Josh Poimboeuf , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , Kan Liang , linux-doc@vger.kernel.org, linux-pm@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-efi@vger.kernel.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-perf-users@vger.kernel.org, rust-for-linux@vger.kernel.org, llvm@lists.linux.dev From: Ard Biesheuvel The relocs tool is no longer used on vmlinux, which is the only 64-bit ELF executable that it used to operate on in the 64-bit build. (It is still used for parts of the decompressor) So drop the 64-bit handling - it is dead code now. Signed-off-by: Ard Biesheuvel --- arch/x86/tools/Makefile | 2 +- arch/x86/tools/relocs.c | 178 +------------------- arch/x86/tools/relocs.h | 9 +- arch/x86/tools/relocs_64.c | 18 -- arch/x86/tools/relocs_common.c | 11 +- 5 files changed, 9 insertions(+), 209 deletions(-) diff --git a/arch/x86/tools/Makefile b/arch/x86/tools/Makefile index 7278e2545c35..f7d12a9dccfc 100644 --- a/arch/x86/tools/Makefile +++ b/arch/x86/tools/Makefile @@ -40,7 +40,7 @@ $(obj)/insn_sanity.o: $(srctree)/tools/arch/x86/lib/insn.c $(srctree)/tools/arch HOST_EXTRACFLAGS += -I$(srctree)/tools/include hostprogs += relocs -relocs-objs := relocs_32.o relocs_64.o relocs_common.o +relocs-objs := relocs_32.o relocs_common.o PHONY += relocs relocs: $(obj)/relocs @: diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 22c2d3f07a57..ff5578e63ff8 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -28,12 +28,7 @@ struct relocs { static struct relocs relocs16; static struct relocs relocs32; -#if ELF_BITS == 64 -static struct relocs relocs64; -# define FMT PRIu64 -#else # define FMT PRIu32 -#endif struct section { Elf_Shdr shdr; @@ -86,10 +81,6 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = { "__end_rodata_aligned|" "__initramfs_start|" "(jiffies|jiffies_64)|" -#if ELF_BITS == 64 - "__per_cpu_load|" - "__end_rodata_hpage_align|" -#endif "__vvar_page|" "_end)$" }; @@ -210,27 +201,6 @@ static const char *rel_type(unsigned type) { static const char *type_name[] = { #define REL_TYPE(X) [X] = #X -#if ELF_BITS == 64 - REL_TYPE(R_X86_64_NONE), - REL_TYPE(R_X86_64_64), - REL_TYPE(R_X86_64_PC64), - REL_TYPE(R_X86_64_PC32), - REL_TYPE(R_X86_64_GOT32), - REL_TYPE(R_X86_64_PLT32), - REL_TYPE(R_X86_64_COPY), - REL_TYPE(R_X86_64_GLOB_DAT), - REL_TYPE(R_X86_64_JUMP_SLOT), - REL_TYPE(R_X86_64_RELATIVE), - REL_TYPE(R_X86_64_GOTPCREL), - REL_TYPE(R_X86_64_GOTPCRELX), - REL_TYPE(R_X86_64_REX_GOTPCRELX), - REL_TYPE(R_X86_64_32), - REL_TYPE(R_X86_64_32S), - REL_TYPE(R_X86_64_16), - REL_TYPE(R_X86_64_PC16), - REL_TYPE(R_X86_64_8), - REL_TYPE(R_X86_64_PC8), -#else REL_TYPE(R_386_NONE), REL_TYPE(R_386_32), REL_TYPE(R_386_PC32), @@ -246,7 +216,6 @@ static const char *rel_type(unsigned type) REL_TYPE(R_386_PC8), REL_TYPE(R_386_16), REL_TYPE(R_386_PC16), -#endif #undef REL_TYPE }; const char *name = "unknown type rel type name"; @@ -312,19 +281,9 @@ static uint32_t elf32_to_cpu(uint32_t val) #define elf_half_to_cpu(x) elf16_to_cpu(x) #define elf_word_to_cpu(x) elf32_to_cpu(x) -#if ELF_BITS == 64 -static uint64_t elf64_to_cpu(uint64_t val) -{ - return le64_to_cpu(val); -} -# define elf_addr_to_cpu(x) elf64_to_cpu(x) -# define elf_off_to_cpu(x) elf64_to_cpu(x) -# define elf_xword_to_cpu(x) elf64_to_cpu(x) -#else # define elf_addr_to_cpu(x) elf32_to_cpu(x) # define elf_off_to_cpu(x) elf32_to_cpu(x) # define elf_xword_to_cpu(x) elf32_to_cpu(x) -#endif static int sym_index(Elf_Sym *sym) { @@ -515,10 +474,7 @@ static void print_absolute_symbols(void) int i; const char *format; - if (ELF_BITS == 64) - format = "%5d %016"PRIx64" %5"PRId64" %10s %10s %12s %s\n"; - else - format = "%5d %08"PRIx32" %5"PRId32" %10s %10s %12s %s\n"; + format = "%5d %08"PRIx32" %5"PRId32" %10s %10s %12s %s\n"; printf("Absolute symbols\n"); printf(" Num: Value Size Type Bind Visibility Name\n"); @@ -559,10 +515,7 @@ static void print_absolute_relocs(void) int i, printed = 0; const char *format; - if (ELF_BITS == 64) - format = "%016"PRIx64" %016"PRIx64" %10s %016"PRIx64" %s\n"; - else - format = "%08"PRIx32" %08"PRIx32" %10s %08"PRIx32" %s\n"; + format = "%08"PRIx32" %08"PRIx32" %10s %08"PRIx32" %s\n"; for (i = 0; i < shnum; i++) { struct section *sec = &secs[i]; @@ -694,104 +647,6 @@ static void walk_relocs(int (*process)(struct section *sec, Elf_Rel *rel, } } -#if ELF_BITS == 64 - -static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, - const char *symname) -{ - unsigned r_type = ELF64_R_TYPE(rel->r_info); - ElfW(Addr) offset = rel->r_offset; - int shn_abs = (sym->st_shndx == SHN_ABS) && !is_reloc(S_REL, symname); - - if (sym->st_shndx == SHN_UNDEF) - return 0; - - switch (r_type) { - case R_X86_64_NONE: - /* NONE can be ignored. */ - break; - - case R_X86_64_PC32: - case R_X86_64_PLT32: - /* - * PC relative relocations don't need to be adjusted. - * - * NB: R_X86_64_PLT32 can be treated as R_X86_64_PC32. - */ - break; - - case R_X86_64_PC64: - /* - * Only used by jump labels - */ - break; - - case R_X86_64_32: - case R_X86_64_32S: - case R_X86_64_64: - case R_X86_64_GOTPCREL: - if (shn_abs) { - /* - * Whitelisted absolute symbols do not require - * relocation. - */ - if (is_reloc(S_ABS, symname)) - break; - - die("Invalid absolute %s relocation: %s\n", rel_type(r_type), symname); - break; - } - - if (r_type == R_X86_64_GOTPCREL) { - Elf_Shdr *s = &secs[sec->shdr.sh_info].shdr; - unsigned file_off = offset - s->sh_addr + s->sh_offset; - - /* - * GOTPCREL relocations refer to instructions that load - * a 64-bit address via a 32-bit relative reference to - * the GOT. In this case, it is the GOT entry that - * needs to be fixed up, not the immediate offset in - * the opcode. Note that the linker will have applied an - * addend of -4 to compensate for the delta between the - * relocation offset and the value of RIP when the - * instruction executes, and this needs to be backed out - * again. (Addends other than -4 are permitted in - * principle, but make no sense in practice so they are - * not supported.) - */ - if (rel->r_addend != -4) { - die("invalid addend (%ld) for %s relocation: %s\n", - rel->r_addend, rel_type(r_type), symname); - break; - } - offset += 4 + (int32_t)get_unaligned_le32(elf_image + file_off); - } - - /* - * Relocation offsets for 64 bit kernels are output - * as 32 bits and sign extended back to 64 bits when - * the relocations are processed. - * Make sure that the offset will fit. - */ - if ((int32_t)offset != (int64_t)offset) - die("Relocation offset doesn't fit in 32 bits\n"); - - if (r_type == R_X86_64_64 || r_type == R_X86_64_GOTPCREL) - add_reloc(&relocs64, offset); - else - add_reloc(&relocs32, offset); - break; - - default: - die("Unsupported relocation type: %s (%d)\n", rel_type(r_type), r_type); - break; - } - - return 0; -} - -#else - static int do_reloc32(struct section *sec, Elf_Rel *rel, Elf_Sym *sym, const char *symname) { @@ -902,8 +757,6 @@ static int do_reloc_real(struct section *sec, Elf_Rel *rel, Elf_Sym *sym, const return 0; } -#endif - static int cmp_relocs(const void *va, const void *vb) { const uint32_t *a, *b; @@ -939,17 +792,10 @@ static void emit_relocs(int as_text, int use_real_mode) int (*write_reloc)(uint32_t, FILE *) = write32; int (*do_reloc)(struct section *sec, Elf_Rel *rel, Elf_Sym *sym, const char *symname); -#if ELF_BITS == 64 - if (!use_real_mode) - do_reloc = do_reloc64; - else - die("--realmode not valid for a 64-bit ELF file"); -#else if (!use_real_mode) do_reloc = do_reloc32; else do_reloc = do_reloc_real; -#endif /* Collect up the relocations */ walk_relocs(do_reloc); @@ -959,11 +805,7 @@ static void emit_relocs(int as_text, int use_real_mode) /* Order the relocations for more efficient processing */ sort_relocs(&relocs32); -#if ELF_BITS == 64 - sort_relocs(&relocs64); -#else sort_relocs(&relocs16); -#endif /* Print the relocations */ if (as_text) { @@ -984,16 +826,6 @@ static void emit_relocs(int as_text, int use_real_mode) for (i = 0; i < relocs32.count; i++) write_reloc(relocs32.offset[i], stdout); } else { -#if ELF_BITS == 64 - /* Print a stop */ - write_reloc(0, stdout); - - /* Now print each relocation */ - for (i = 0; i < relocs64.count; i++) - if (!i || relocs64.offset[i] != relocs64.offset[i - 1]) - write_reloc(relocs64.offset[i], stdout); -#endif - /* Print a stop */ write_reloc(0, stdout); @@ -1027,12 +859,6 @@ static void print_reloc_info(void) walk_relocs(do_reloc_info); } -#if ELF_BITS == 64 -# define process process_64 -#else -# define process process_32 -#endif - void process(FILE *fp, int use_real_mode, int as_text, int show_absolute_syms, int show_absolute_relocs, int show_reloc_info) diff --git a/arch/x86/tools/relocs.h b/arch/x86/tools/relocs.h index 7a509604ff92..ef9eec96bd62 100644 --- a/arch/x86/tools/relocs.h +++ b/arch/x86/tools/relocs.h @@ -32,10 +32,7 @@ enum symtype { S_NSYMTYPES }; -void process_32(FILE *fp, int use_real_mode, int as_text, - int show_absolute_syms, int show_absolute_relocs, - int show_reloc_info); -void process_64(FILE *fp, int use_real_mode, int as_text, - int show_absolute_syms, int show_absolute_relocs, - int show_reloc_info); +void process(FILE *fp, int use_real_mode, int as_text, + int show_absolute_syms, int show_absolute_relocs, + int show_reloc_info); #endif /* RELOCS_H */ diff --git a/arch/x86/tools/relocs_64.c b/arch/x86/tools/relocs_64.c deleted file mode 100644 index 9029cb619cb1..000000000000 --- a/arch/x86/tools/relocs_64.c +++ /dev/null @@ -1,18 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -#include "relocs.h" - -#define ELF_BITS 64 - -#define ELF_MACHINE EM_X86_64 -#define ELF_MACHINE_NAME "x86_64" -#define SHT_REL_TYPE SHT_RELA -#define Elf_Rel Elf64_Rela - -#define ELF_CLASS ELFCLASS64 -#define ELF_R_SYM(val) ELF64_R_SYM(val) -#define ELF_R_TYPE(val) ELF64_R_TYPE(val) -#define ELF_ST_TYPE(o) ELF64_ST_TYPE(o) -#define ELF_ST_BIND(o) ELF64_ST_BIND(o) -#define ELF_ST_VISIBILITY(o) ELF64_ST_VISIBILITY(o) - -#include "relocs.c" diff --git a/arch/x86/tools/relocs_common.c b/arch/x86/tools/relocs_common.c index 6634352a20bc..167985ecd544 100644 --- a/arch/x86/tools/relocs_common.c +++ b/arch/x86/tools/relocs_common.c @@ -72,14 +72,9 @@ int main(int argc, char **argv) die("Cannot read %s: %s", fname, strerror(errno)); } rewind(fp); - if (e_ident[EI_CLASS] == ELFCLASS64) - process_64(fp, use_real_mode, as_text, - show_absolute_syms, show_absolute_relocs, - show_reloc_info); - else - process_32(fp, use_real_mode, as_text, - show_absolute_syms, show_absolute_relocs, - show_reloc_info); + process(fp, use_real_mode, as_text, + show_absolute_syms, show_absolute_relocs, + show_reloc_info); fclose(fp); return 0; }