From patchwork Sat Feb 3 12:25:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769696 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32B865D72C; Sat, 3 Feb 2024 12:27:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963249; cv=none; b=GWCRMnPUZDwLNDaaaO3oy2L88/yIA8m7uSGFp4qtWVBAhfrxyM251BcvoynD53GznWUQy8b6m2hNramPrMmkDrwzGfLH0lmbqg1cvJOlw9BML5wotCms6JkVMjg/qlKKUyOuJIdC4HSMQRMQ5ndnj9ofA4gDWwRK8WFGoRLBpAM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963249; c=relaxed/simple; bh=3IJ5SeAASdHtIIStcvs359UoBpZ4hMPDZ8zRhgL+KJQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oJ5sBCw1h+R+s6+ZaYvbq1c+ntatnxA4uEhCORpM0fHvx9Q5OYEgRVqiO1Xb9Z28PbXCrLy3rKnODL+SU7zdujug+5DZ6gRjfJ/jUzNIPZWd0uCrJsLB5EQq47hChi4iSZKksMgNVjHdYetoJ+J8HzAVudGKQDM7o/f6ZNIh3aI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DPmH+nfN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DPmH+nfN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35388C43609; Sat, 3 Feb 2024 12:27:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963248; bh=3IJ5SeAASdHtIIStcvs359UoBpZ4hMPDZ8zRhgL+KJQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=DPmH+nfNwutz8I2cy45COQPzbwAEfPjI6ffHVfjCNllrgl6vp33v6POzMgRpiUz1H R6TyvxMdEhSqVGHpIeiEr0p+nf/z4YAOXLc817SR8gY0hxjRWonN4ZYWB1mELfcD+u J7wYEJW9thoiDrQF7mUZhwtuBsqwnml271Qg555AgCbsSA+Nx3F6RaqEpoowp7DVyT JEBlCPvrwO3xFaqY3QrwHh/DKN1ZVngK60F7Ml/DEtqx6WVIyWbFL2sNR7OFKWGnAX rthP6rYhkhBxKkG8iwQoOZdxqrQDiD6u4jT4CVAYdvIIt6mGqI3GifpPZr4J/thkmP IBUzzRC/4Qz0Q== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:28 +0000 Subject: [PATCH v8 02/38] prctl: arch-agnostic prctl for shadow stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-2-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=4768; i=broonie@kernel.org; h=from:subject:message-id; bh=3IJ5SeAASdHtIIStcvs359UoBpZ4hMPDZ8zRhgL+KJQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDS1VpII3RNTbS21ZqAUQaox+Eucg15vDi3Rm0d /5PJGGWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w0gAKCRAk1otyXVSH0HTNB/ 9zYcH0WIf7htiNJ645c2X8FzMF1bRnXmpIa3HQrk1F0SgbCOhNYcpXWoBqfZBZpYKq+oSSh4xxf9L8 HBVGI/kvF8TwcIyTGk4Qv5eb07yqP+ld18BNiEn5N6adaeLXpm0CQfVz9kiSZJeuoM0uTM6ijzD0H/ ibNLhTdz2oJ8tP5z1WmYMA4uTUBS+Ox3HKYsVRRGjlWNGcpyHCWagK7vzKAwsdUOLmfePgoE3OLKAv IIb8m6ZlcL7kGrSIDv+Z64fInWKFktfQjQrcnKkyt+MFXDI+X/4ArV4mww497KBHQuEsk0c7q1cALZ ISEhWbo8N3Sl5Sj2gdItzpIbuFjN2Q X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Three architectures (x86, aarch64, riscv) have announced support for shadow stacks with fairly similar functionality. While x86 is using arch_prctl() to control the functionality neither arm64 nor riscv uses that interface so this patch adds arch-agnostic prctl() support to get and set status of shadow stacks and lock the current configuation to prevent further changes, with support for turning on and off individual subfeatures so applications can limit their exposure to features that they do not need. The features are: - PR_SHADOW_STACK_ENABLE: Tracking and enforcement of shadow stacks, including allocation of a shadow stack if one is not already allocated. - PR_SHADOW_STACK_WRITE: Writes to specific addresses in the shadow stack. - PR_SHADOW_STACK_PUSH: Push additional values onto the shadow stack. These features are expected to be inherited by new threads and cleared on exec(), unknown features should be rejected for enable but accepted for locking (in order to allow for future proofing). This is based on a patch originally written by Deepak Gupta but modified fairly heavily, support for indirect landing pads is removed, additional modes added and the locking interface reworked. The set status prctl() is also reworked to just set flags, if setting/reading the shadow stack pointer is required this could be a separate prctl. Signed-off-by: Mark Brown --- include/linux/mm.h | 4 ++++ include/uapi/linux/prctl.h | 22 ++++++++++++++++++++++ kernel/sys.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index c0a782eda803..0b1139c5df60 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4182,4 +4182,8 @@ static inline bool pfn_is_unaccepted_memory(unsigned long pfn) return range_contains_unaccepted_memory(paddr, paddr + PAGE_SIZE); } +int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status); +int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); +int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); + #endif /* _LINUX_MM_H */ diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 370ed14b1ae0..3c66ed8f46d8 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -306,4 +306,26 @@ struct prctl_mm_map { # define PR_RISCV_V_VSTATE_CTRL_NEXT_MASK 0xc # define PR_RISCV_V_VSTATE_CTRL_MASK 0x1f +/* + * Get the current shadow stack configuration for the current thread, + * this will be the value configured via PR_SET_SHADOW_STACK_STATUS. + */ +#define PR_GET_SHADOW_STACK_STATUS 71 + +/* + * Set the current shadow stack configuration. Enabling the shadow + * stack will cause a shadow stack to be allocated for the thread. + */ +#define PR_SET_SHADOW_STACK_STATUS 72 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) +# define PR_SHADOW_STACK_WRITE (1UL << 1) +# define PR_SHADOW_STACK_PUSH (1UL << 2) + +/* + * Prevent further changes to the specified shadow stack + * configuration. All bits may be locked via this call, including + * undefined bits. + */ +#define PR_LOCK_SHADOW_STACK_STATUS 73 + #endif /* _LINUX_PRCTL_H */ diff --git a/kernel/sys.c b/kernel/sys.c index e219fcfa112d..96e8a6b5993a 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -2301,6 +2301,21 @@ int __weak arch_prctl_spec_ctrl_set(struct task_struct *t, unsigned long which, return -EINVAL; } +int __weak arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status) +{ + return -EINVAL; +} + +int __weak arch_set_shadow_stack_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + +int __weak arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + #define PR_IO_FLUSHER (PF_MEMALLOC_NOIO | PF_LOCAL_THROTTLE) #ifdef CONFIG_ANON_VMA_NAME @@ -2743,6 +2758,21 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, case PR_RISCV_V_GET_CONTROL: error = RISCV_V_GET_CONTROL(); break; + case PR_GET_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_get_shadow_stack_status(me, (unsigned long __user *) arg2); + break; + case PR_SET_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_set_shadow_stack_status(me, arg2); + break; + case PR_LOCK_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_lock_shadow_stack_status(me, arg2); + break; default: error = -EINVAL; break; From patchwork Sat Feb 3 12:25:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769695 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5EC6A5DF0B; Sat, 3 Feb 2024 12:28:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963307; cv=none; b=n1OSAhuzFzMtej2VQaqRptWZtuxOdJFP0+7uKHGO7suYzektaL+xeL1ghimMw+zo1Gul2xSIZFW3DISiCXpKgnVZdkMeww3t7BStx1uxgScrBWSILFYXXgXNSoxAmof50kGeTa5Fj0q/u3L/ydDO924qxHvc8984gonS6CgBHbs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963307; c=relaxed/simple; bh=Z3injfV8td0LqFLeGew+/6AX1Y4kpAJIpTs5MPlZTRs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=uouGEWXOKqb6zr0SeEAQ21KZBMkJlrCe9CrmJKoYAb0WARCH60yhI7O/VnO40DsVUY13pP9DSH22spyry2U62ij5k05R8bRYKvOebXTtR6HqERT90WSCP4yOZE8DM8JxldzmasCMWxqkGnoQ/iNizF9IZ7rvNNqUFdFWHQFcZJo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mqlIJ5g3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mqlIJ5g3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8D8ABC43390; Sat, 3 Feb 2024 12:27:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963306; bh=Z3injfV8td0LqFLeGew+/6AX1Y4kpAJIpTs5MPlZTRs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=mqlIJ5g3tl5q0vYK7ykUXUM8Tc+tv8rON+LlVXaOXLZkci6RbJ9fIy9D+0L3q7RZE LNE4wWCXSxPrW+6yc449KPyyth2xlowITE1wQPdj9VVu14q4p28fH6+Mr/dcOpooOb BbEVSpIijcUScCJRz0nwuXKIusgNuQtMB02ImSKBl+XJwLtQ3oHiZv0JTqosJnP+BW vVz3wocrNiQnKimT9yoE4DaY5NZfLqfpeiSu3Mb9VHmwvHRJOGxxextL0Vny9D7SQt gGZECkE2b9Lri6+3C5cMXoGpJNtp9oXBohTewFa3e/ub1biMSTSfmS2n0YRloz6YyT PQqWESBoDqKPA== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:30 +0000 Subject: [PATCH v8 04/38] arm64: Document boot requirements for Guarded Control Stacks Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-4-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=1640; i=broonie@kernel.org; h=from:subject:message-id; bh=Z3injfV8td0LqFLeGew+/6AX1Y4kpAJIpTs5MPlZTRs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDUsly811JO8MY9VKwAyipGXg/WYYy7DrZJ9PfF WUTZtWCJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w1AAKCRAk1otyXVSH0Cp2B/ 9ZbiDjwSsFZbMG4QBc+H/K8vKVC04W/sEbxEueoW6RvocX423DsgVKt7TW3y4gTB9CNsk3bfhBpSJy 6Ade2Y1WKbQ8UudQ6Q46T26n3UDyrCs+lO5Ity3TveBReAWASeqybFoYtghSJjhoomY3nitWgA805k L7UsEzNP/UsrMpJz13RSBlmn74EryWZcdDt0milhnxX9Vqz/uyKUjQnYUghRBg+gp4joxeRK9ZJH5A V7SJkdsQX6g3OWnLAUYwC5zljMqXzEGptSSf89vn4Fg8NoQPePUIgsU6RQ6Ebg2hlmb8qTSvHBEJ6p pwKlr8cYsYxj5L+KGlC1UFZ6mVJsri X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB FEAT_GCS introduces a number of new system registers, we require that access to these registers is not trapped when we identify that the feature is detected. Signed-off-by: Mark Brown --- Documentation/arch/arm64/booting.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/Documentation/arch/arm64/booting.rst b/Documentation/arch/arm64/booting.rst index b57776a68f15..de3679770c64 100644 --- a/Documentation/arch/arm64/booting.rst +++ b/Documentation/arch/arm64/booting.rst @@ -411,6 +411,28 @@ Before jumping into the kernel, the following conditions must be met: - HFGRWR_EL2.nPIRE0_EL1 (bit 57) must be initialised to 0b1. + - For features with Guarded Control Stacks (FEAT_GCS): + + - If EL3 is present: + + - SCR_EL3.GCSEn (bit 39) must be initialised to 0b1. + + - If the kernel is entered at EL1 and EL2 is present: + + - HFGITR_EL2.nGCSEPP (bit 59) must be initialised to 0b1. + + - HFGITR_EL2.nGCSSTR_EL1 (bit 58) must be initialised to 0b1. + + - HFGITR_EL2.nGCSPUSHM_EL1 (bit 57) must be initialised to 0b1. + + - HFGRTR_EL2.nGCS_EL1 (bit 53) must be initialised to 0b1. + + - HFGRTR_EL2.nGCS_EL0 (bit 52) must be initialised to 0b1. + + - HFGWTR_EL2.nGCS_EL1 (bit 53) must be initialised to 0b1. + + - HFGWTR_EL2.nGCS_EL0 (bit 52) must be initialised to 0b1. + The requirements described above for CPU mode, caches, MMUs, architected timers, coherency and system registers apply to all CPUs. All CPUs must enter the kernel in the same exception level. Where the values documented From patchwork Sat Feb 3 12:25:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769694 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB6075FB87; Sat, 3 Feb 2024 12:29:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963369; cv=none; b=U5dajNLiR0OBbQgN8k/s0G9fm7T6el8DSyGOSp/ileLh1K/Ke4Kr1BGZk+r/6TOv2BBtwI5r/r4pZV4/DlPGkd0/Ati9NI54yvgVMsRl0m7V+bW7AspfrerbhubsAdCSolotwUP7sdg9h6fpyDH1DMMTvb5ptXaaxLwWcSOlz/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963369; c=relaxed/simple; bh=nfp6EMTmsacTE8jIWijq0csUQnWyRS9dbPzwDPbc9Oo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HV3AlT4YECIm8I1SlWDZrxo21QHSghNaHq28yfYVtHc1/gHOCzpmgCAK+Gz2XHJn7l72G7U03e4fqVBYyBYQ1luiQI+/wTRvEwXAo0YmWGM4t7Hu5p63cdkBDt5RnIvBjsI8aq7Z/q4HfraPeO2n1cMK5+7O/7HOvKcQINala0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=osiScCWg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="osiScCWg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2C2D4C433C7; Sat, 3 Feb 2024 12:29:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963368; bh=nfp6EMTmsacTE8jIWijq0csUQnWyRS9dbPzwDPbc9Oo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=osiScCWg4d3KIQZcrJeOEESWG83yHbzhiGcQeX7MowDI3BpjPG0b59J5w0TMyUGhx AiZFxIDmeGUFCmE3Gx5qt5Ekc13a+DGzvGFihiMuVXGFOpCy0HP16OhiVdFRNmINdk B+++NJvgEKQkXUHftLefN+RkWlfmRMCdidNYbJscH4DWcTBUpNux0R9eWYq6BgUyJf ZIpFreOOPDY1dmytZEu+vqVlYuYTsEYckA3a3Ae8EXP8Uz0zdzKlWfbpI0+Y+oqN/+ 8BARDvmTtKiPJDePxkqMCbgacVKxEXJM9wA9NjmKcvMUJHFKUtFzBD3+BpE2U3mXGq 6FWOb2QFd08tw== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:32 +0000 Subject: [PATCH v8 06/38] arm64/sysreg: Add definitions for architected GCS caps Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-6-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=1355; i=broonie@kernel.org; h=from:subject:message-id; bh=nfp6EMTmsacTE8jIWijq0csUQnWyRS9dbPzwDPbc9Oo=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDV9XvzQG5mbwdmnxaEA2beaNmzTLR4QnZfsb5k dABRgtOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w1QAKCRAk1otyXVSH0DSDB/ 4ndgRTySkwuTALaejt2vvGqwqgfiRnfYkefv1ma8rX3ZFKkMhDmmWTaqgOWLCRoex5Fe5UhcFwYrHT a2aI58d9V1IAUodeTLqo1GD5DaDA4u1/tDmYlgYmy9u9PJrnPonUUaYzkKVfkOHjIVRvFL75b9VcNH ax5DpBY3FxYokPpg6PWwG8R0LApWNrwrEMY/jxDYInXcBnW8nnqGaG5PQ7XlpwM04YBo00koRuPJ/3 Sc3T/ew/2wbt3217BNKGVIUpFtzCn83x7VLwr/Ht3vfMrHO0IkX0V/RAxhBMYq9JvGYMm5rL/cCkbA OLa29/wK/D7MMMDE1wda7wLwRtVOwc X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The architecture defines a format for guarded control stack caps, used to mark the top of an unused GCS in order to limit the potential for exploitation via stack switching. Add definitions associated with these. Signed-off-by: Mark Brown --- arch/arm64/include/asm/sysreg.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index c3b19b376c86..6ed813e856c1 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -1064,6 +1064,26 @@ #define POE_RXW UL(0x7) #define POE_MASK UL(0xf) +/* + * Definitions for Guarded Control Stack + */ + +#define GCS_CAP_ADDR_MASK GENMASK(63, 12) +#define GCS_CAP_ADDR_SHIFT 12 +#define GCS_CAP_ADDR_WIDTH 52 +#define GCS_CAP_ADDR(x) FIELD_GET(GCS_CAP_ADDR_MASK, x) + +#define GCS_CAP_TOKEN_MASK GENMASK(11, 0) +#define GCS_CAP_TOKEN_SHIFT 0 +#define GCS_CAP_TOKEN_WIDTH 12 +#define GCS_CAP_TOKEN(x) FIELD_GET(GCS_CAP_TOKEN_MASK, x) + +#define GCS_CAP_VALID_TOKEN 0x1 +#define GCS_CAP_IN_PROGRESS_TOKEN 0x5 + +#define GCS_CAP(x) ((((unsigned long)x) & GCS_CAP_ADDR_MASK) | \ + GCS_CAP_VALID_TOKEN) + #define ARM64_FEATURE_FIELD_BITS 4 /* Defined for compatibility only, do not add new users. */ From patchwork Sat Feb 3 12:25:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769693 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1485A60873; Sat, 3 Feb 2024 12:30:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963406; cv=none; b=mdshrv186sBy8CC4Mu+Xn0No7+mdlE7fVYSD8QyprqKUw/mgkRGhmtMEMkn4XrzJuP6+g4xpm4yWvzRQua4arf2xfALE+jSO6JkF6piPlu+mBwkFQ8yu/e57gc50zHJs4YhYaSDY3dHXKujQxwdygxFApKPH3MVn2k5Q7XVkUb8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963406; c=relaxed/simple; bh=39WiY3pJD/byvb21132HZ1FFf2AaFNt9bxr2VNbG8Fw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=g8CQRPbRxyjgwHVvJFFbnb3Y+DFHklLi5cSvAMq9Lx+qBwrCiEsseKMVm2gG3YfpJE4uNHCZqzBHApcq1gWDypQv1/Eb2mnMCLYGONU2s1j4yyb5JqKv0H870KfoXa1CiVA9/Ng6W68Aw7OtnzCsXr+alddUhiGu+ba9Bdj4m4E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=shZj4zmb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="shZj4zmb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 74CBBC43394; Sat, 3 Feb 2024 12:29:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963405; bh=39WiY3pJD/byvb21132HZ1FFf2AaFNt9bxr2VNbG8Fw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=shZj4zmbEG51YJXT2HR/hbkSqqeTtDXlNZZf7YSTfM8hx53j8LXpmJZLw3Iv82n8T EmsnjZlBS3Whdw84UibP2OywFky27fJU1kV1ZQZrvMHmo5oo6DDxG7Ve+KACATX2av DyY97pWgxe8gwFmEDtgEyjX1bk3f/dH3s4xHFsuiFE52tVb78u03HcN/O2BbuJHAi7 KNMQRWaioIbZCcU/WSDqiLZ81LH+jBUTthLazxEBHx9RWs8ibKA/VQQFMjwhktb/Sh 61x79dnbtbMxzrMbLJs9q2PhBw/ScBw5eG6rQTZlugcKruZnKQ6rD0cO+jKGPxrzE1 jHMBTdnV+2cdA== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:34 +0000 Subject: [PATCH v8 08/38] arm64/gcs: Provide put_user_gcs() Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-8-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=1037; i=broonie@kernel.org; h=from:subject:message-id; bh=39WiY3pJD/byvb21132HZ1FFf2AaFNt9bxr2VNbG8Fw=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDXlOhNa5wqLtdcsL9HXNagaPZlUCqhjcNX1c/m OGZhZROJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w1wAKCRAk1otyXVSH0GPoB/ 41N6+udhNIKyfclpuplMcjQpE4Medy7BQP1spdA3c3PSHB2nlB6id9g5c3aaDNbOG37yHJfw2ZoUN8 sgb3yLFVwhS5JnwHUUpsdwrK7jftwR1SlVArLgVwgyqyz9VuVeZ71IXLIBRhx9kDgK412ZYWfDCUag cByQrhbCoyfJSbprcOy7eNXWxpd4+C4do+G2udTLpptHhhxMqt/0zB6iOYBYk7SaIIXok1V3v/bKNx lQ5mXxZ9LYnsYVcbi9w8SQzJ4VZGzDcECPlEhUS0e2APKEEdGo2EJJ3bjMgkTEPN9C6B03JU9IPhRE A/tP1bU7676ZdvRcI96pstycM97i9L X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In order for EL1 to write to an EL0 GCS it must use the GCSSTTR instruction rather than a normal STTR. Provide a put_user_gcs() which does this. Signed-off-by: Mark Brown --- arch/arm64/include/asm/uaccess.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 22e10e79f56a..e118c3d772c8 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -445,6 +445,24 @@ static inline int gcssttr(unsigned long __user *addr, unsigned long val) return err; } +static inline void put_user_gcs(unsigned long val, unsigned long __user *addr, + int *err) +{ + int ret; + + if (!access_ok((char __user *)addr, sizeof(u64))) { + *err = -EFAULT; + return; + } + + uaccess_ttbr0_enable(); + ret = gcssttr(addr, val); + if (ret != 0) + *err = ret; + uaccess_ttbr0_disable(); +} + + #endif /* CONFIG_ARM64_GCS */ #endif /* __ASM_UACCESS_H */ From patchwork Sat Feb 3 12:25:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769692 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C52663519; Sat, 3 Feb 2024 12:30:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963448; cv=none; b=tI+K4rnUPPX2flGRybDpUC0h3ghk6qpDwvc6xrKfMxHm6q98gDj+ExsIV7sCrS42MHL91IQ4vipouzsvihCqTFGFeLI++0PtT7cLu+C3ILhBXdvAju2vZWRHYeP0e95fjJywQ6nOKH51TwttX5TgJA3s1JkVZFlH1vU+flFu3so= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963448; c=relaxed/simple; bh=+xprcrSIR+r2HgpXmJPi2lCD7uV8JeEsPkL8Hbk9EMs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=V6gKRAMsCPGwVTZ/EV9KDrE3PETMH0buObkiDDUY1yrSr7idRuFyV4LYabfipHSKTBdsiaWaKvbR+NsltYbMFPqnRbcpuBK8Lavu9VYV8+nPi3MCDrI1TtBAmDwc8YRWCns8JhHnxN5tv5o+lVHy2CUn+EHgBF/wS+sbLCHDbxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pF7nqRrZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pF7nqRrZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6C475C43390; Sat, 3 Feb 2024 12:30:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963448; bh=+xprcrSIR+r2HgpXmJPi2lCD7uV8JeEsPkL8Hbk9EMs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pF7nqRrZUhMqdiSGGPRZ4lQFV9LAa39nD4GPEx2pS4SHjfN3SyIUFZR2Ncyf50pbK gcUitWiq8R081mUDfpLYs8f83JqvKeJhtIYHomgIenfRIpeZGr7t37qfR5e7P242C0 yAjJTLv8+DHI2+tBewBy5RlcRObbJsIrkJboK8hy1jh+m5+qlZIdQ01Rx7JyO7FzxB UpP7vAxry8k/EdaVxXUk987ecA9dYoJUu0tj+eqXMr6gkgokd72MQj+oCkT4ExaRqf 2j3DD+Oti0+xH7uSt2T4aXnMyCPvtZRwVl+4ynyOwbpwYTQxarJNo6RxomoQcVoFC1 Cd//sQHU5hr8A== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:36 +0000 Subject: [PATCH v8 10/38] arm64/mm: Allocate PIE slots for EL0 guarded control stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-10-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=2936; i=broonie@kernel.org; h=from:subject:message-id; bh=+xprcrSIR+r2HgpXmJPi2lCD7uV8JeEsPkL8Hbk9EMs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDY6r7GyBRKw7wUgsZ+3nfEbNbAsesepgDtEw2w f6AWxyWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w2AAKCRAk1otyXVSH0KszCA CFSKuiESPpOpCspjr7eC4F35RG5kiLJP0HvJZjycPYcFxtl9dvbsnp0qpk9kwV3sdW0jN2LHxK77XV +PCEGj/fKNBU75dICtXgs2uLlQW8DBTzCDw5TAQzsQIDZwbATaaXD6dwqA2jFnExw/StImkNDeij0B ZRWxF2yKfgGJ1MmZzmUJi94IQSLO3DGmVwXC4q3LN/vT/aVbx9GjC+2f5aNBkpaJgkSsZhpp3nT0Uy qr6/+LduI3OPyrsVtXMpp0GLZ8qhasqr5QNlNBn1LeEEPVIiUmbbNx2PEogj2m82LLNAD5aw0zqR/N Relyii0MDbUbCWfD6Vk60iGQtQM3Vi X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Pages used for guarded control stacks need to be described to the hardware using the Permission Indirection Extension, GCS is not supported without PIE. In order to support copy on write for guarded stacks we allocate two values, one for active GCSs and one for GCS pages marked as read only prior to copy. Since the actual effect is defined using PIE the specific bit pattern used does not matter to the hardware but we choose two values which differ only in PTE_WRITE in order to help share code with non-PIE cases. Signed-off-by: Mark Brown --- arch/arm64/include/asm/pgtable-prot.h | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 483dbfa39c4c..14a33e0bece3 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -129,15 +129,23 @@ extern bool arm64_use_ng_mappings; /* 6: PTE_PXN | PTE_WRITE */ /* 7: PAGE_SHARED_EXEC PTE_PXN | PTE_WRITE | PTE_USER */ /* 8: PAGE_KERNEL_ROX PTE_UXN */ -/* 9: PTE_UXN | PTE_USER */ +/* 9: PAGE_GCS_RO PTE_UXN | PTE_USER */ /* a: PAGE_KERNEL_EXEC PTE_UXN | PTE_WRITE */ -/* b: PTE_UXN | PTE_WRITE | PTE_USER */ +/* b: PAGE_GCS PTE_UXN | PTE_WRITE | PTE_USER */ /* c: PAGE_KERNEL_RO PTE_UXN | PTE_PXN */ /* d: PAGE_READONLY PTE_UXN | PTE_PXN | PTE_USER */ /* e: PAGE_KERNEL PTE_UXN | PTE_PXN | PTE_WRITE */ /* f: PAGE_SHARED PTE_UXN | PTE_PXN | PTE_WRITE | PTE_USER */ +#define _PAGE_GCS (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_WRITE | PTE_USER) +#define _PAGE_GCS_RO (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_USER) + +#define PAGE_GCS __pgprot(_PAGE_GCS) +#define PAGE_GCS_RO __pgprot(_PAGE_GCS_RO) + #define PIE_E0 ( \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_GCS) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_R) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_X_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_RX) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RWX) | \ @@ -145,6 +153,8 @@ extern bool arm64_use_ng_mappings; PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED), PIE_RW)) #define PIE_E1 ( \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_NONE_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW) | \ From patchwork Sat Feb 3 12:25:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769691 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF7DD15E5B5; Sat, 3 Feb 2024 12:31:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963462; cv=none; b=nCrG6Vm0Ko9TjB+2IiIHI7X+dvT2Ei8ZFAkTTENIMuLtpv9rZaVAWmqj72mgdFBdZPpZCUVtgKnSkprVMo6eiTZOceEBxT6miW791YMmuJ+BLiZaK984C4wV6lfw0KOdfAA/f8c6xv4c2m4XtvZVfQ6Pd/Nm/uQg1cFZ6Q7afQg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963462; c=relaxed/simple; bh=s6bCn957vl7MWPN5faZZH74vr8xHhA9oW4MYYSSC+7I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=J08+BL8Qg/G8JvQnD12caSqrxT+w7T51OxCQxJrXgEGzhw3L7Od9xNsDHSEENJZ9XQUSPJwM8JMYXAeAoEpsQAvBwkoiwcgUUrZiQXcHW7gRp8STAsLk1gDMkHIqtk/1RJRTK2Ga9ocXIAHJHYmHix25VaEMvY7UDduBaYgNksE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cXOJ77sc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cXOJ77sc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F7B3C43390; Sat, 3 Feb 2024 12:30:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963461; bh=s6bCn957vl7MWPN5faZZH74vr8xHhA9oW4MYYSSC+7I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cXOJ77sc3cQAIYTgI4yLYEpIC37f03fcaibYvuInwn8di/O4yIt+bHnqL//pUPxDo hYEsFtkP6kQ89PQKm8oK7Wa9WxYQxdLqg+otrN5iczV2csW4rN1aMx8vigCUbn8kDd bmsybYB2hCWtaAxNkqcsA7w7EhpTS+iu98v+OGoXKYYdspFjQn6SR05CQaI5eZRZfT 4Yfc+cN/cWZF8hS+rJhDfqi8J7ujnzpTykA1gdfYbQTXJmJLh9EySsLIC7gaxOoXOc Fc1D2QtP4kY4lu1FgkM5ScscVf9rbilxymkFNDtZTdM+1+0v+4XyIa1x9LC7C0S0sP GlrafKo2iJ7Bg== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:38 +0000 Subject: [PATCH v8 12/38] arm64/mm: Map pages for guarded control stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-12-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=1837; i=broonie@kernel.org; h=from:subject:message-id; bh=s6bCn957vl7MWPN5faZZH74vr8xHhA9oW4MYYSSC+7I=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDaHwr5wrOWEW/xel9E3S7jsHXGClztspo/AIxO WVUvGMiJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w2gAKCRAk1otyXVSH0DmCB/ 95KF+JXdMxHAx9xnGeVA7yTyp0+fMAu3TeB7kS7DtuBsLE4OtsXKtaSYZVp0Hm5moimwKXZTN8sLaR TkcMryVwyRFPeF/KBSacACnQi7Bt9HFPmo9H0eBNRCZuAv9vxu1NFrua3FD7vN0xbAuzCYRC+d53+k 61m3z2A9musH8F9maMwFmrdAW1+mROuXbvvK/LNIGA8xEkvqwNMm7hXoiXIRNdFzWTDBENITY28+dB RAddUPCMDR3nm/ogfYadyteOSUiumCbk85Ejb+ZT+4B6IyxeUQasBjqo+/eSMQVa6Ke6EvIKHHrOQY mUUz2tdyO/wb4T5iv2uYIYRZSrxs8I X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Map pages flagged as being part of a GCS as such rather than using the full set of generic VM flags. This is done using a conditional rather than extending the size of protection_map since that would make for a very sparse array. Signed-off-by: Mark Brown --- arch/arm64/include/asm/mman.h | 9 +++++++++ arch/arm64/mm/mmap.c | 13 ++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h index c21849ffdd88..6d3fe6433a62 100644 --- a/arch/arm64/include/asm/mman.h +++ b/arch/arm64/include/asm/mman.h @@ -61,6 +61,15 @@ static inline bool arch_validate_flags(unsigned long vm_flags) return false; } + if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { + /* + * An executable GCS isn't a good idea, and the mm + * core can't cope with a shared GCS. + */ + if (vm_flags & (VM_EXEC | VM_ARM64_BTI | VM_SHARED)) + return false; + } + return true; } diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index 645fe60d000f..e44ce6fcfad9 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -79,9 +79,20 @@ arch_initcall(adjust_protection_map); pgprot_t vm_get_page_prot(unsigned long vm_flags) { - pteval_t prot = pgprot_val(protection_map[vm_flags & + pteval_t prot; + + /* If this is a GCS then only interpret VM_WRITE. */ + if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { + if (vm_flags & VM_WRITE) + prot = _PAGE_GCS; + else + prot = _PAGE_GCS_RO; + } else { + prot = pgprot_val(protection_map[vm_flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]); + } + /* VM_ARM64_BTI on a GCS is rejected in arch_validate_flags() */ if (vm_flags & VM_ARM64_BTI) prot |= PTE_GP; From patchwork Sat Feb 3 12:25:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769690 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8283E5EE69; Sat, 3 Feb 2024 12:31:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963476; cv=none; b=h82tDslhiacq+zHTKqbZY6vpIi6rYaQ6PgYDxB/T6HU5vJJWRiTOsvbURjtWH2W2Y/oy4ESzgxGTtbaBzs5L8YmSYK0sej/rlL8rZnFnE3Vv20G43dGCQ88YIZlTyTEM9eiOlirxLoexSYUYdH2xJsS8EqpcJe2gCd1wKKpJImA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963476; c=relaxed/simple; bh=D3mrjjzhu1bmO322qeetoeUWjp5BglarJjfdNbXnw/Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LuvQDLChHRO7iBVur6I+j3yOfln3XyskDYgOlf0J8MsLVg33jL9HN35JifghpIdQbVEhuLsTyHOHvRZNB6NlfLo9aSQ4jVgwHIHCWjrAa8IiiDHXLonJPR8ysIhDbNPsyMg+t9fIfWM+tM0AcA5CJEn+6rP7Nn5nIBkcqGUk43I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=b8LtOloG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="b8LtOloG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C56F8C433C7; Sat, 3 Feb 2024 12:31:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963476; bh=D3mrjjzhu1bmO322qeetoeUWjp5BglarJjfdNbXnw/Y=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=b8LtOloGXjSjg88htRVqjKTHNIpv0KYxDjzZZc71ZNTni2PxZT1VAYdJN/gNX8zgU ybOCUuvaxAoBMVShab+E5dZjUyGnTueq0nXdn1xuED1XL3WEEikq8LRu5ULi1VwggL ZgrVxgcgUxtO5uO27W/yK4PC7TpG9BTPJE5/nHaFo2NC8luOoerHgYAqVYROaqSJVa yt4mYZic23yETrP7HcrYgZfC1ZqB2dVzF/Wa6U3eZHxsG4Rp18/77HbM42KQpgvmub rq5RZXLNX70wZiIl1EhPquGNwPdGSPO1GriBmlDJ8EjDf22vLCcozMiMlXJYsxwZlW p/dVV5Ax8HJwA== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:40 +0000 Subject: [PATCH v8 14/38] arm64/gcs: Allow GCS usage at EL0 and EL1 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-14-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=2231; i=broonie@kernel.org; h=from:subject:message-id; bh=D3mrjjzhu1bmO322qeetoeUWjp5BglarJjfdNbXnw/Y=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDc8EHGCknoOBKe6qZKHD+oiq1TQJV0CBg/qGUx v9IrslaJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w3AAKCRAk1otyXVSH0D/LB/ 42988rQEDEd1c4AWNUz5rGufrrvcfn77vTQ2D1Q95lKvScKDoA3SCsg/NCBvD6bTwCQyimBPieLclf vphVOHi8RIWA46cysnMRfhnLSUzUUwJdRhNYL1KASDBbgrGh2nIdUMB1zMuuBP4yJowmA+LBTjroCK YPS9XoZUZIHlFfkTOi0V/a8Fu+HwPn14jxbpiehTXDctJz5EfvwcjTooeg2p6dmcuHzocpcSTVBX5V AZNNDcwvGgtXBrDMV8wVp3ZJk6Iq3ejaqd2L8Iz3zS/sAJBUp7sDi/fCOHvh+iWdCsG4mGqUuqM0F+ LjLYBdtbAn0AUFobSt30jnpFzdqG+D X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB There is a control HCRX_EL2.GCSEn which must be set to allow GCS features to take effect at lower ELs and also fine grained traps for GCS usage at EL0 and EL1. Configure all these to allow GCS usage by EL0 and EL1. Signed-off-by: Mark Brown --- arch/arm64/include/asm/el2_setup.h | 17 +++++++++++++++++ arch/arm64/include/asm/kvm_arm.h | 4 ++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index b7afaa026842..17672563e333 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -27,6 +27,14 @@ ubfx x0, x0, #ID_AA64MMFR1_EL1_HCX_SHIFT, #4 cbz x0, .Lskip_hcrx_\@ mov_q x0, HCRX_HOST_FLAGS + + /* Enable GCS if supported */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_hcrx_\@ + orr x0, x0, #HCRX_EL2_GCSEn + +.Lset_hcrx_\@: msr_s SYS_HCRX_EL2, x0 .Lskip_hcrx_\@: .endm @@ -190,6 +198,15 @@ orr x0, x0, #HFGxTR_EL2_nPIR_EL1 orr x0, x0, #HFGxTR_EL2_nPIRE0_EL1 + /* GCS depends on PIE so we don't check it if PIE is absent */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_fgt_\@ + + /* Disable traps of access to GCS registers at EL0 and EL1 */ + orr x0, x0, #HFGxTR_EL2_nGCS_EL1_MASK + orr x0, x0, #HFGxTR_EL2_nGCS_EL0_MASK + .Lset_fgt_\@: msr_s SYS_HFGRTR_EL2, x0 msr_s SYS_HFGWTR_EL2, x0 diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 3c6f8ba1e479..a9354c237a97 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -103,9 +103,9 @@ #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) #define HCRX_GUEST_FLAGS \ - (HCRX_EL2_SMPME | HCRX_EL2_TCR2En | \ + (HCRX_EL2_SMPME | HCRX_EL2_TCR2En | HCRX_EL2_GCSEn |\ (cpus_have_final_cap(ARM64_HAS_MOPS) ? (HCRX_EL2_MSCEn | HCRX_EL2_MCE2) : 0)) -#define HCRX_HOST_FLAGS (HCRX_EL2_MSCEn | HCRX_EL2_TCR2En) +#define HCRX_HOST_FLAGS (HCRX_EL2_MSCEn | HCRX_EL2_TCR2En | HCRX_EL2_GCSEn) /* TCR_EL2 Registers bits */ #define TCR_EL2_DS (1UL << 32) From patchwork Sat Feb 3 12:25:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769689 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA02667C5F; Sat, 3 Feb 2024 12:31:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963489; cv=none; b=uqvP6RrwTD/sbBVmYlVJw+jPLTY2rQfqbFUsB+jNeqJRSNG30GVKb8aa2HVAuRVSqX+pjWoHOmzhotxbQLhq7dNg2sEn5nHvdnNd7wsP8jAbVVvh1XXJKkZ6I/qO9oecteIBPKpPIZYNBdcynYaPDQiR84pC6geCeSd0VoMZAcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963489; c=relaxed/simple; bh=2cZBAsyDen5t10YKJMqQqRacz55Ix1gmnVNOGcAdVHI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=G84lgowekndlbWHgR4Ic5PZkhc3NzqajGsYg4Ie8aTB/P2JxmnZJw50pV7xNjeMiNz61ajriH63NzrrCYTcPHe5VyNb+Z91gBMnvxGUvdn1w9NeL2WIq2IvloHQFEFBcSRp6tfDYMz10CgFHhO1fibbtQGwtkLMLb3zRAcGNlzs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=G3KODdyX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="G3KODdyX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 23856C433C7; Sat, 3 Feb 2024 12:31:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963489; bh=2cZBAsyDen5t10YKJMqQqRacz55Ix1gmnVNOGcAdVHI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=G3KODdyXV84zp1WCaszFTbVJ2uzuktodWs8f8wULYlN8w99KkxKz/+ezLneV0DugV x/cwvGfsdUC3eW+YYRIwg6HU1tyRVrb7V4Ub/MdYDk6mE4D7XxqcwonHdr44Kp/bvw b2g/KQS9cGH9Cl2Y2GuKcr7ecb32sw2ElksDnlhfkXP2DbnDikvM7MxJ8h8o+cOfkl KvPtjaP0HwY9t6phffOTirc/ieuvAsz8aZwasX1XNp1Cv64JpuCsLj5fu7rvuPL6TP hvuoAl/z3r0yWCjpqndfLm5YGAVsOPfdT2IgvmV9XZi0r9yPQS0W5hdeffXFKXMUnt A9oqQY7azVtTQ== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:42 +0000 Subject: [PATCH v8 16/38] arm64/hwcap: Add hwcap for GCS Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-16-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=2995; i=broonie@kernel.org; h=from:subject:message-id; bh=2cZBAsyDen5t10YKJMqQqRacz55Ix1gmnVNOGcAdVHI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDdArV5/f1ACfon5ADCHhNy9LNTvLl/uQOQRNL0 1A2GhhGJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w3QAKCRAk1otyXVSH0LMLB/ 94kl6jJs+ZdrvXxnmg2VS+Ldw/qKP/sR+00QxgbGMh/uXh1xYFxuDHzgUpcnXqnP08q9VHn3XI0q+9 5SpUX6Qd3Fvtqy1KVotOyPSUoeE7dpPJntzXucH4HzayJCWZ1VbxUaon7nPT6rWirxCE4eqW1rispq HomupbnqiFm9Q5BSDH8gS76+j3FBtUB71bbwothoqHW1avAFYEY/AYeoWmbNMfEsuvtFv4PI5FJ+ZX G4TNA3Mkel4mGN+Y7fQe+nlszt8TNErQCFln4CSh+0S/qH4X75ZBaIvA0HKUTIWPBRrMBDHdpA/Uaq sZb3P7vZZGRDet9V9CpODyO7rtqWzM X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide a hwcap to enable userspace to detect support for GCS. Signed-off-by: Mark Brown --- Documentation/arch/arm64/elf_hwcaps.rst | 3 +++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 1 + arch/arm64/kernel/cpufeature.c | 3 +++ arch/arm64/kernel/cpuinfo.c | 1 + 5 files changed, 9 insertions(+) diff --git a/Documentation/arch/arm64/elf_hwcaps.rst b/Documentation/arch/arm64/elf_hwcaps.rst index ced7b335e2e0..86d4ace9c75c 100644 --- a/Documentation/arch/arm64/elf_hwcaps.rst +++ b/Documentation/arch/arm64/elf_hwcaps.rst @@ -317,6 +317,9 @@ HWCAP2_LRCPC3 HWCAP2_LSE128 Functionality implied by ID_AA64ISAR0_EL1.Atomic == 0b0011. +HWCAP2_GCS + Functionality implied by ID_AA64PFR1_EL1.GCS == 0b1 + 4. Unused AT_HWCAP bits ----------------------- diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h index cd71e09ea14d..e01e6b72a839 100644 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -142,6 +142,7 @@ #define KERNEL_HWCAP_SVE_B16B16 __khwcap2_feature(SVE_B16B16) #define KERNEL_HWCAP_LRCPC3 __khwcap2_feature(LRCPC3) #define KERNEL_HWCAP_LSE128 __khwcap2_feature(LSE128) +#define KERNEL_HWCAP_GCS __khwcap2_feature(GCS) /* * This yields a mask that user programs can use to figure out what diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h index 5023599fa278..996b5b5d4c4e 100644 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -107,5 +107,6 @@ #define HWCAP2_SVE_B16B16 (1UL << 45) #define HWCAP2_LRCPC3 (1UL << 46) #define HWCAP2_LSE128 (1UL << 47) +#define HWCAP2_GCS (1UL << 48) #endif /* _UAPI__ASM_HWCAP_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index b606842ab8c1..1a92c4502a0b 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2867,6 +2867,9 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { HWCAP_CAP(ID_AA64ZFR0_EL1, I8MM, IMP, CAP_HWCAP, KERNEL_HWCAP_SVEI8MM), HWCAP_CAP(ID_AA64ZFR0_EL1, F32MM, IMP, CAP_HWCAP, KERNEL_HWCAP_SVEF32MM), HWCAP_CAP(ID_AA64ZFR0_EL1, F64MM, IMP, CAP_HWCAP, KERNEL_HWCAP_SVEF64MM), +#endif +#ifdef CONFIG_ARM64_GCS + HWCAP_CAP(ID_AA64PFR1_EL1, GCS, IMP, CAP_HWCAP, KERNEL_HWCAP_GCS), #endif HWCAP_CAP(ID_AA64PFR1_EL1, SSBS, SSBS2, CAP_HWCAP, KERNEL_HWCAP_SSBS), #ifdef CONFIG_ARM64_BTI diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c index 47043c0d95ec..b3ec0b89c9e0 100644 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -128,6 +128,7 @@ static const char *const hwcap_str[] = { [KERNEL_HWCAP_SVE_B16B16] = "sveb16b16", [KERNEL_HWCAP_LRCPC3] = "lrcpc3", [KERNEL_HWCAP_LSE128] = "lse128", + [KERNEL_HWCAP_GCS] = "gcs", }; #ifdef CONFIG_COMPAT From patchwork Sat Feb 3 12:25:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769688 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F8766E2A6; Sat, 3 Feb 2024 12:31:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963503; cv=none; b=WxTKPrAE5/HPEYDpeUtPkdnPGrnABYLTbrhSkq3/ExxVa7ne22ytGNgXUbsV+TcnN8o8XMCTQF6qckLhw7jb4tuIpoJIZagWdnOEyibmPHnCRuueKH/6B80HscZFA/3xoEC+uG+5o6gGHJ7SaFjIUAUqJdpp3ZmcFv87sbAMeBk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963503; c=relaxed/simple; bh=Lt1ZeKCFkJyKhq3Sh3OVvMj7LavngE1UwVvkTZX97+I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FhT7iWLAIUDDXOkzjXiTagusHVMJx3n2ptfDOGLcDr4hK4UkFtBcqfPMe7u6qiyjj0IwNYBOq1HpBIIpwTDVOs3tRU9tLt+3+R1Oh3g6sQs5vzj9ZxweN6M4+WLs2M1Rzec+aYoO7m9ZRRaXtia1dFpiS/THwmx65EoVp1SrSss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MnWK2vjJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MnWK2vjJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E116C433C7; Sat, 3 Feb 2024 12:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963502; bh=Lt1ZeKCFkJyKhq3Sh3OVvMj7LavngE1UwVvkTZX97+I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=MnWK2vjJAAkkVC5x18o0vC0rR6nWPKDKY/BfS2I7qnJrpTNZ7R8Qz8N2zz3SBXZAb Plu7/fT9RdJISFL1mqYkz+AgOzFGp+Z+J6i7BmPixmF1AqfN5ZWCgVnvcVYlQBVXra Zd7JbJqQwqRNXBjBqOPOq+FrX6vse5U3wsFEaqFlxMS2YsB/MFdGCtdE/EzAtAWrwi Iauyx9NOR2R+f0J5P+YkzVkJCed3/NPySZSWYyWr4mNPAdtDZGwl/iL9Yj21tzdY82 N3cgnf1ES+7GPIwbvWGD6RSELlUEgvlcb5TSUqRxj+O1zady7WfkC3AeKRZmCZYhji BBDG0y7GXZhJg== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:44 +0000 Subject: [PATCH v8 18/38] arm64/mm: Handle GCS data aborts Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-18-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=5688; i=broonie@kernel.org; h=from:subject:message-id; bh=Lt1ZeKCFkJyKhq3Sh3OVvMj7LavngE1UwVvkTZX97+I=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDfoxOLlYSp5xnxxZxlNSdHvgw6ns5IOY8k6hzh cR1VkbKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w3wAKCRAk1otyXVSH0E4mB/ 95XdtcX8DVyd71GtFByYY8oAvO73P/c6ExMKh3ybTyBN/ABJgVUkX95hDfnSe981nGOLTifnm3U++j IODn0CNQWSR4SnOsuHWTaRfdY6R6YefMgHV9Xw34TcyJ9Sf8K38gWCcgN8RzbHcZubiG7Q813st9Z/ uAYzMmm4wvg7inzgM6OpKlf9gCs1vPs14bJokCn3Df2GKE/Bll+nGBtqdlsz94/2olBqgmbuAbioNx gpDhH9lRdaP6f+aUv86IqJQj3aFYO5AgWNXTxuLYK/IuECgkXeMx9LwawJCLq2NLrfBkEtryw3/QZJ qtgyiU1S1mo+Ym81oIcflatdEV7UEB X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB All GCS operations at EL0 must happen on a page which is marked as having UnprivGCS access, including read operations. If a GCS operation attempts to access a page without this then it will generate a data abort with the GCS bit set in ESR_EL1.ISS2. EL0 may validly generate such faults, for example due to copy on write which will cause the GCS data to be stored in a read only page with no GCS permissions until the actual copy happens. Since UnprivGCS allows both reads and writes to the GCS (though only through GCS operations) we need to ensure that the memory management subsystem handles GCS accesses as writes at all times. Do this by adding FAULT_FLAG_WRITE to any GCS page faults, adding handling to ensure that invalid cases are identfied as such early so the memory management core does not think they will succeed. The core cannot distinguish between VMAs which are generally writeable and VMAs which are only writeable through GCS operations. EL1 may validly write to EL0 GCS for management purposes (eg, while initialising with cap tokens). We also report any GCS faults in VMAs not marked as part of a GCS as access violations, causing a fault to be delivered to userspace if it attempts to do GCS operations outside a GCS. Signed-off-by: Mark Brown --- arch/arm64/mm/fault.c | 79 +++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 71 insertions(+), 8 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 55f6455a8284..5303d4e3457d 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -494,13 +494,30 @@ static void do_bad_area(unsigned long far, unsigned long esr, } } +/* + * Note: not valid for EL1 DC IVAC, but we never use that such that it + * should fault. EL0 cannot issue DC IVAC (undef). + */ +static bool is_write_abort(unsigned long esr) +{ + return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); +} + +static bool is_gcs_fault(unsigned long esr) +{ + if (!esr_is_data_abort(esr)) + return false; + + return ESR_ELx_ISS2(esr) & ESR_ELx_GCS; +} + #define VM_FAULT_BADMAP ((__force vm_fault_t)0x010000) #define VM_FAULT_BADACCESS ((__force vm_fault_t)0x020000) static vm_fault_t __do_page_fault(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned int mm_flags, unsigned long vm_flags, - struct pt_regs *regs) + unsigned long esr, struct pt_regs *regs) { /* * Ok, we have a good vm_area for this memory access, so we can handle @@ -510,6 +527,26 @@ static vm_fault_t __do_page_fault(struct mm_struct *mm, */ if (!(vma->vm_flags & vm_flags)) return VM_FAULT_BADACCESS; + + if (vma->vm_flags & VM_SHADOW_STACK) { + /* + * Writes to a GCS must either be generated by a GCS + * operation or be from EL1. + */ + if (is_write_abort(esr) && + !(is_gcs_fault(esr) || is_el1_data_abort(esr))) + return VM_FAULT_BADACCESS; + } else { + /* + * GCS faults should never happen for pages that are + * not part of a GCS and the operation being attempted + * can never succeed. + */ + if (is_gcs_fault(esr)) + return VM_FAULT_BADACCESS; + } + + return handle_mm_fault(vma, addr, mm_flags, regs); } @@ -518,13 +555,18 @@ static bool is_el0_instruction_abort(unsigned long esr) return ESR_ELx_EC(esr) == ESR_ELx_EC_IABT_LOW; } -/* - * Note: not valid for EL1 DC IVAC, but we never use that such that it - * should fault. EL0 cannot issue DC IVAC (undef). - */ -static bool is_write_abort(unsigned long esr) +static bool is_invalid_el0_gcs_access(struct vm_area_struct *vma, u64 esr) { - return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); + if (!system_supports_gcs()) + return false; + if (likely(!(vma->vm_flags & VM_SHADOW_STACK))) { + if (is_gcs_fault(esr)) + return true; + return false; + } + if (is_gcs_fault(esr)) + return false; + return is_write_abort(esr); } static int __kprobes do_page_fault(unsigned long far, unsigned long esr, @@ -573,6 +615,13 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, /* If EPAN is absent then exec implies read */ if (!alternative_has_cap_unlikely(ARM64_HAS_EPAN)) vm_flags |= VM_EXEC; + /* + * Upgrade read faults to write faults, GCS reads must + * occur on a page marked as GCS so we need to trigger + * copy on write always. + */ + if (is_gcs_fault(esr)) + mm_flags |= FAULT_FLAG_WRITE; } if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { @@ -594,6 +643,20 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, if (!vma) goto lock_mmap; + /* + * We get legitimate write faults for GCS pages from GCS + * operations, even when the initial operation was a read, as + * a result of upgrading GCS accesses to writes for CoW but + * GCS acceses outside of a GCS must fail. Specifically check + * for this since the mm core isn't able to distinguish + * invalid GCS access from valid ones and will try to resolve + * the fault. + */ + if (is_invalid_el0_gcs_access(vma, esr)) { + vma_end_read(vma); + goto lock_mmap; + } + if (!(vma->vm_flags & vm_flags)) { vma_end_read(vma); goto lock_mmap; @@ -625,7 +688,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto done; } - fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, regs); + fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, esr, regs); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { From patchwork Sat Feb 3 12:25:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769687 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB8E45F48F; Sat, 3 Feb 2024 12:31:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963516; cv=none; b=JpVkatNMDU7TDo309anhVpQuM6GN1GuCcUVUMxGFxJImV59WDSSzoOUcev7fc7AKx1CCdAVE0c/rwDSD54hZTZmtatUGvNmdzfwGw3jfJRa1UOQqT3bLCKfhn1eJV2Zo59c+Ya7bQWA3IOrsAyK/hiNku5S1I7P54ySCw/tlw8o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963516; c=relaxed/simple; bh=0Pd0/0g5ocUKEQPs+CZnjpGh8a2yrzXoWJs/48QxMEE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LusQrer36KAoEedHk8WFq/6S77xsMvULmAomJIzg7hly0/cThLGaXZArm/JBIqZ69ZoxXxmFfEKrfQOIM0mNw4lE5bHwgWY2R0ru1HBHkNmBGrFKUjMiOh38fbydSXsM5RcLnKYdRyeHlH8NmEgD0L1yGOp5qVtrx8x/IJnFhQc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cyNcuS0y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cyNcuS0y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BDE6C43609; Sat, 3 Feb 2024 12:31:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963516; bh=0Pd0/0g5ocUKEQPs+CZnjpGh8a2yrzXoWJs/48QxMEE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cyNcuS0yepL5PtUjvbc/4GFBi2f/FzV2Rq68YWIi6IyGPqro5gwLsktAkGRmwPo1a DPZhVCUVajR9H8/X/Mfrr5g/19Z9FqBbjSBiqN49LicQByhoDK78Kzey/i8uaKehHs LTRp/whj0BtqnzkxIl1rHWzXdbCik8Y0arm5N5cnVYOH4p9zdydt9tsy7G5jSIPUlE ghVEwmHGsW1EjfuxRTurzDKjJr6RXkxUft3/jf/K1OSaiK4+nwoWVbcIiXmp8OuW1J s8rV2N0mpTA5rWd/gCd4jovg5NSF9MQOzC4oohJwqa/2pI0gR/8eV2YU/LnDYnWV6Q JBz+FjYiklaJA== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:46 +0000 Subject: [PATCH v8 20/38] arm64/gcs: Ensure that new threads have a GCS Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-20-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=8134; i=broonie@kernel.org; h=from:subject:message-id; bh=0Pd0/0g5ocUKEQPs+CZnjpGh8a2yrzXoWJs/48QxMEE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDgHmbpnJ6Kt5FNSJQp2DtC/IJQdbRg85cvhx1K Miz2+d6JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w4AAKCRAk1otyXVSH0Nt4B/ wL1XPKLA8oon7Ntls/p18FPxhxgITjHFhOhCLATvZLdx2/CGCM3XeV/oZsdqUKAMfvNSiqdk2Hge2L fXSpLOz8lwnFEMhBV3cPt2SQCcib5865B65V4ujM7VbyryF0RdOZBJ41g7KrO1xusFXtASE/2LgTRg cLOGH05SSH2Iikwkk6HQbyru9QOvtzsuefI+bj077rEszYry0kVn8SgYKDtyKel1hhgX47wO6gRvy/ LH1qXI2PclfTBlVo+H8Uqr5qA2Vnn4yZpnxjUdpiKQnpuBPZKk3k3h+YZur6WqivBxp086HvEi+kZZ zA7yjuAcOJULaSipntqexmuqh00q3A X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB When a new thread is created by a thread with GCS enabled the GCS needs to be specified along with the regular stack. clone3() has been extended to support this case, allowing userspace to explicitly specify the size and location of the GCS. The specified GCS must have a valid GCS token at the top of the stack, as though userspace were pivoting to the new GCS. This will be consumed on use. At present we do not atomically consume the token, this will be addressed in a future revision. Unfortunately plain clone() is not extensible and existing clone3() users will not specify a stack so all existing code would be broken if we mandated specifying the stack explicitly. For compatibility with these cases and also x86 (which did not initially implement clone3() support for shadow stacks) if no GCS is specified we will allocate one thread so when a thread is created which has GCS enabled allocate one for it. We follow the extensively discussed x86 implementation and allocate min(RLIMIT_STACK, 4G). Since the GCS only stores the call stack and not any variables this should be more than sufficient for most applications. GCSs allocated via this mechanism then it will be freed when the thread exits, those explicitly configured by the user will not. Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 9 ++++ arch/arm64/kernel/process.c | 29 +++++++++++ arch/arm64/mm/gcs.c | 117 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 155 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 04594ef59dad..c1f274fdb9c0 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -8,6 +8,8 @@ #include #include +struct kernel_clone_args; + static inline void gcsb_dsync(void) { asm volatile(".inst 0xd503227f" : : : "memory"); @@ -58,6 +60,8 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) void gcs_set_el0_mode(struct task_struct *task); void gcs_free(struct task_struct *task); void gcs_preserve_current_state(void); +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); #else @@ -69,6 +73,11 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) static inline void gcs_set_el0_mode(struct task_struct *task) { } static inline void gcs_free(struct task_struct *task) { } static inline void gcs_preserve_current_state(void) { } +static inline unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + return -ENOTSUPP; +} #endif diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index fd80b43c2969..8bd66cde0a86 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -285,9 +285,32 @@ static void flush_gcs(void) write_sysreg_s(0, SYS_GCSPR_EL0); } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + unsigned long gcs; + + gcs = gcs_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(gcs)) + return PTR_ERR((void *)gcs); + + p->thread.gcs_el0_mode = current->thread.gcs_el0_mode; + p->thread.gcs_el0_locked = current->thread.gcs_el0_locked; + + /* Ensure the current state of the GCS is seen by CoW */ + gcsb_dsync(); + + return 0; +} + #else static void flush_gcs(void) { } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + return 0; +} #endif @@ -303,6 +326,7 @@ void flush_thread(void) void arch_release_task_struct(struct task_struct *tsk) { fpsimd_release_task(tsk); + gcs_free(tsk); } int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) @@ -369,6 +393,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long stack_start = args->stack; unsigned long tls = args->tls; struct pt_regs *childregs = task_pt_regs(p); + int ret; memset(&p->thread.cpu_context, 0, sizeof(struct cpu_context)); @@ -410,6 +435,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.uw.tp_value = tls; p->thread.tpidr2_el0 = 0; } + + ret = copy_thread_gcs(p, args); + if (ret != 0) + return ret; } else { /* * A kthread has no context to ERET to, so ensure any buggy diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index b0a67efc522b..3cbc3a3d4bc7 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -8,6 +8,113 @@ #include #include +static unsigned long alloc_gcs(unsigned long addr, unsigned long size, + unsigned long token_offset, bool set_res_tok) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE; + struct mm_struct *mm = current->mm; + unsigned long mapped_addr, unused; + + if (addr) + flags |= MAP_FIXED_NOREPLACE; + + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ | PROT_WRITE, flags, + VM_SHADOW_STACK, 0, &unused, NULL); + mmap_write_unlock(mm); + + return mapped_addr; +} + +static unsigned long gcs_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + /* Allocate RLIMIT_STACK/2 with limits of PAGE_SIZE..2G */ + size = PAGE_ALIGN(min_t(unsigned long long, + rlimit(RLIMIT_STACK) / 2, SZ_2G)); + return max(PAGE_SIZE, size); +} + +static bool gcs_consume_token(struct task_struct *tsk, unsigned long user_addr) +{ + unsigned long expected = GCS_CAP(user_addr); + unsigned long val; + int ret = 0; + + /* This should really be an atomic cpmxchg. It is not. */ + __get_user_error(val, (__user unsigned long *)user_addr, ret); + if (ret != 0) + return false; + + if (val != expected) + return false; + + put_user_gcs(0, (__user unsigned long*)user_addr, &ret); + + return ret == 0; +} + +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size, gcspr_el0; + + /* If the user specified a GCS use it. */ + if (args->shadow_stack_size) { + if (!system_supports_gcs()) + return (unsigned long)ERR_PTR(-EINVAL); + + addr = args->shadow_stack; + size = args->shadow_stack_size; + + /* + * There should be a token, there might be an end of + * stack marker. + */ + gcspr_el0 = addr + size - (2 * sizeof(u64)); + if (!gcs_consume_token(tsk, gcspr_el0)) { + gcspr_el0 += sizeof(u64); + if (!gcs_consume_token(tsk, gcspr_el0)) + return (unsigned long)ERR_PTR(-EINVAL); + } + + /* Userspace is responsible for unmapping */ + tsk->thread.gcspr_el0 = gcspr_el0 + sizeof(u64); + } else { + + /* + * Otherwise fall back to legacy clone() support and + * implicitly allocate a GCS if we need a new one. + */ + + if (!system_supports_gcs()) + return 0; + + if (!task_gcs_el0_enabled(tsk)) + return 0; + + if ((args->flags & (CLONE_VFORK | CLONE_VM)) != CLONE_VM) { + tsk->thread.gcspr_el0 = read_sysreg_s(SYS_GCSPR_EL0); + return 0; + } + + size = args->stack_size; + + size = gcs_size(size); + addr = alloc_gcs(0, size, 0, 0); + if (IS_ERR_VALUE(addr)) + return addr; + + tsk->thread.gcs_base = addr; + tsk->thread.gcs_size = size; + tsk->thread.gcspr_el0 = addr + size - sizeof(u64); + } + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. @@ -30,6 +137,16 @@ void gcs_set_el0_mode(struct task_struct *task) void gcs_free(struct task_struct *task) { + + /* + * When fork() with CLONE_VM fails, the child (tsk) already + * has a GCS allocated, and exit_thread() calls this function + * to free it. In this case the parent (current) and the + * child share the same mm struct. + */ + if (!task->mm || task->mm != current->mm) + return; + if (task->thread.gcs_base) vm_munmap(task->thread.gcs_base, task->thread.gcs_size); From patchwork Sat Feb 3 12:25:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769686 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DF21745C0; Sat, 3 Feb 2024 12:32:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963530; cv=none; b=VVeGoCCIuRtY62z+lFroVbvqKoytfX+j0iENte8vmEDpErsYFzJQtq2Y2whAVqmymevNLXhoCQDNH/CKuujMNFJpXyNcAz0eVsD9+zYNqihNtcRzOa3zTpWdQySNjFe4COOMhG96gxYNDZ66flIlJG0s0dvAiYzjN7AyCUL2EnA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963530; c=relaxed/simple; bh=Aw31IGKjr1UrJDLWxkb2b/BK0n5+HbIF3Im1kdrhlF4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AQNAzkT7DnQojg94ZsYdO6T5e3NJJJ8OIrkPRRWG/TN6WCG2wdrzBdk29mXcQ8mDABb7jcMpj3A4lRLPAjxhmX+cVnzZ+FddDFaC4I1os429a23rn2vNY1yh373N2SDLcWKb7g1bLt3bAkbp2j68Htxp9DtsGzmf2hCe45cqXgE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GKUG+zpZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GKUG+zpZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A16F9C433F1; Sat, 3 Feb 2024 12:32:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963529; bh=Aw31IGKjr1UrJDLWxkb2b/BK0n5+HbIF3Im1kdrhlF4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=GKUG+zpZ2VILKpS5XDiWYdz2oomc5hvV3q397oN3cKwrqvjN8JKf9ZMOB+6zZRcY1 iTUhw186L7BkGkiyEnDwbA2D1HfTzsgjtdPEhb3OSGyDDEL4L3JlqFz1AWtIBAQSY6 9dm9WMbF7/+oCBcxK3fFKSGyTaH2t0ukjuHqcX8qIGbanmqsR109W7myEvSS2ptTye FSH1gSgaA3tm4tUie8cuI9rQF5s39SGkMpyheiPi+onw9b6y8icdbNSgLC+iGX/nJe CxyxwaJOf9c9pdGmuwR4iuMbniVufpOG9YnrXm3OemlI0rqHYkzLn2mzlr33eNnXwL 0mDazwlGIzVkQ== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:48 +0000 Subject: [PATCH v8 22/38] arm64/mm: Implement map_shadow_stack() Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-22-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=2931; i=broonie@kernel.org; h=from:subject:message-id; bh=Aw31IGKjr1UrJDLWxkb2b/BK0n5+HbIF3Im1kdrhlF4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDiF619KL+pXnFncRboG77ij7BrGr7OUVw8KD54 W94MjnmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w4gAKCRAk1otyXVSH0Jx+B/ 4v6RH+W/BD/bwrjhMex1EknsuooW4f/43TzROGdgQf5ZleIEHLx7aQHusweh6WwRiwnNYYsQ47/1EM BBeH8SXHb4LF1Foh1DTm5I9/a3wa3noh+LUCiGbbO1uvAOAcrsconNTQ4XfZmOTHEVomxOV9JdONTR mrghCkklix5q/XA7ublf7dKYPmkBrd4XpgT/S80mmzPh5YFsOt3wpu+UCpqeqcgyCacBE7C/v284GH zAJjljdn6oB0SlBacil3LPR+nO9vOcxZaN1lT2ae59T0BR5pE8p7S+qcMIYriIalD501OnOZhMaNTr rVijbRZWUs3jqHufM94IsGOg3p/jRe X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB As discussed extensively in the changelog for the addition of this syscall on x86 ("x86/shstk: Introduce map_shadow_stack syscall") the existing mmap() and madvise() syscalls do not map entirely well onto the security requirements for guarded control stacks since they lead to windows where memory is allocated but not yet protected or stacks which are not properly and safely initialised. Instead a new syscall map_shadow_stack() has been defined which allocates and initialises a shadow stack page. Implement this for arm64. Two flags are provided, allowing applications to request that the stack be initialised with a valid cap token at the top of the stack and optionally also an end of stack marker above that. We support requesting an end of stack marker alone but since this is a NULL pointer it is indistinguishable from not initialising anything by itself. Signed-off-by: Mark Brown --- arch/arm64/mm/gcs.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index 95f5cf599bc6..f34821d98d85 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -115,6 +115,67 @@ unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, return addr; } +SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) +{ + unsigned long alloc_size; + unsigned long __user *cap_ptr; + unsigned long cap_val; + int ret = 0; + int cap_offset; + + if (!system_supports_gcs()) + return -EOPNOTSUPP; + + if (flags & ~(SHADOW_STACK_SET_TOKEN | SHADOW_STACK_SET_MARKER)) + return -EINVAL; + + if (addr && (addr % PAGE_SIZE)) + return -EINVAL; + + if (size == 8 || size % 8) + return -EINVAL; + + /* + * An overflow would result in attempting to write the restore token + * to the wrong location. Not catastrophic, but just return the right + * error code and block it. + */ + alloc_size = PAGE_ALIGN(size); + if (alloc_size < size) + return -EOVERFLOW; + + addr = alloc_gcs(addr, alloc_size, 0, false); + if (IS_ERR_VALUE(addr)) + return addr; + + /* + * Put a cap token at the end of the allocated region so it + * can be switched to. + */ + if (flags & SHADOW_STACK_SET_TOKEN) { + /* Leave an extra empty frame as a top of stack marker? */ + if (flags & SHADOW_STACK_SET_MARKER) + cap_offset = 2; + else + cap_offset = 1; + + cap_ptr = (unsigned long __user *)(addr + size - + (cap_offset * sizeof(unsigned long))); + cap_val = GCS_CAP(cap_ptr); + + put_user_gcs(cap_val, cap_ptr, &ret); + if (ret != 0) { + vm_munmap(addr, size); + return -EFAULT; + } + + /* Ensure the new cap is viaible for GCS */ + gcsb_dsync(); + } + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. From patchwork Sat Feb 3 12:25:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769685 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEB8C5DF19; Sat, 3 Feb 2024 12:32:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963544; cv=none; b=m3M1RN4cCpLyJq77fy0VfTQa2QexiWVPddbOsPF6BGmFHRKpNam2ZOD36eGBvE8b0bifyoP5AwVm5/+JB4+5OefwnsADw7k0Vk+K14l1Hf1/Ku+GcsZQ/uh/hYqjvounSYDjRg6P7A5G0OpL2v9l0lYtR3/nqmW3BTQH4Brwdhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963544; c=relaxed/simple; bh=l7eNMpwW/Zrho/p1JMAQudrls7xr4SXcPsP5pQLneX4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Q5CSFmfk+dLa8SP14vJoKL83pwNJ8X+tkk/uGmdzzaczYnYNruezCrwAftmH5KIjrHMecADhaSvZtzCmQa8sZYQc4Y6WRfItPT/TsYCGfcKC+pzYq63ALzXh7qqrQ5XME4XBJphSyvY4s/Z4I4RgBFP8CqOf2d4EKLhLiy2KC0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SpICFxO+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SpICFxO+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 29685C433F1; Sat, 3 Feb 2024 12:32:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963543; bh=l7eNMpwW/Zrho/p1JMAQudrls7xr4SXcPsP5pQLneX4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SpICFxO+jw19mzMfV6cfMNixQkq7VMvu1AkUejmRe86YQ0H1QNUy8v+j8JNd2tTog Jyb1HUt2TnXkefBJRdGTuIuyNXjKyqzW2EtKrhyDBHm9Nqbt1a7/tBHPHGbr8kjpCY 7F98IjW72F3y5+HkKcTMzRhV8deyVVr2/tHR4Br+VJ0Qr3z61hbkS9thLp2OKi1Jkc Mh/vneAbXIDRKtcTa8T0JsflVT+4tkjywTeYTvwE95+IjxhJ8D9hrOQxUsB4N/9ceC idlg8LiudKW0E/5Kv1gnN/nZtZmt7nK53l66xuuPKHiwNsXpJcBW0CCVz1qLdSk/3A gzrIpWoFBrSLQ== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:50 +0000 Subject: [PATCH v8 24/38] arm64/signal: Expose GCS state in signal frames Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-24-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=6022; i=broonie@kernel.org; h=from:subject:message-id; bh=l7eNMpwW/Zrho/p1JMAQudrls7xr4SXcPsP5pQLneX4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDknrRNNIJOduAyD1EyJZboeRdf0F/nRG9a5SH5 BPws6vCJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w5AAKCRAk1otyXVSH0A6/B/ 9oXh58vXfiNiy88b0dMTBDLISUs38B0Uk3dMhOeVwEmjjKQt3sm27kbGcsLXvpOr+r+MJL9VWV0Sus N+Od2IoGJF0yPgAAVxWdNbao1BIT4fyRMJFB8ZbW2lcNTEBuS1nyDx2l3cmDf61+5TikKrPilnubIU LhOijbvx2cOhZnhT/PVcfCCY9Lx56KnRj99oXS8qOs2mJAIrKS4bBNHJnbEljhzpE8njZoOYv2XcXQ K4yI+zAozwOmELTo29JufFWqG3jNOxBGJhkCcSjyPmkCitRY/A2TdCd8WASkUYVtN9NBgIkp9ru0Yf k1eyar6n/pQxNv9n3aUd+RlzikIsTB X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add a context for the GCS state and include it in the signal context when running on a system that supports GCS. We reuse the same flags that the prctl() uses to specify which GCS features are enabled and also provide the current GCS pointer. We do not support enabling GCS via signal return, there is a conflict between specifying GCSPR_EL0 and allocation of a new GCS and this is not an ancticipated use case. We also enforce GCS configuration locking on signal return. Signed-off-by: Mark Brown --- arch/arm64/include/uapi/asm/sigcontext.h | 9 +++ arch/arm64/kernel/signal.c | 108 +++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+) diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h index f23c1dc3f002..7b66d245f2d2 100644 --- a/arch/arm64/include/uapi/asm/sigcontext.h +++ b/arch/arm64/include/uapi/asm/sigcontext.h @@ -168,6 +168,15 @@ struct zt_context { __u16 __reserved[3]; }; +#define GCS_MAGIC 0x47435300 + +struct gcs_context { + struct _aarch64_ctx head; + __u64 gcspr; + __u64 features_enabled; + __u64 reserved; +}; + #endif /* !__ASSEMBLY__ */ #include diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 1cca646a7479..91bfb315bfd3 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -88,6 +88,7 @@ struct rt_sigframe_user_layout { unsigned long fpsimd_offset; unsigned long esr_offset; + unsigned long gcs_offset; unsigned long sve_offset; unsigned long tpidr2_offset; unsigned long za_offset; @@ -214,6 +215,8 @@ struct user_ctxs { u32 za_size; struct zt_context __user *zt; u32 zt_size; + struct gcs_context __user *gcs; + u32 gcs_size; }; static int preserve_fpsimd_context(struct fpsimd_context __user *ctx) @@ -606,6 +609,83 @@ extern int restore_zt_context(struct user_ctxs *user); #endif /* ! CONFIG_ARM64_SME */ +#ifdef CONFIG_ARM64_GCS + +static int preserve_gcs_context(struct gcs_context __user *ctx) +{ + int err = 0; + u64 gcspr; + + /* + * We will add a cap token to the frame, include it in the + * GCSPR_EL0 we report to support stack switching via + * sigreturn. + */ + gcs_preserve_current_state(); + gcspr = current->thread.gcspr_el0; + if (task_gcs_el0_enabled(current)) + gcspr -= 8; + + __put_user_error(GCS_MAGIC, &ctx->head.magic, err); + __put_user_error(sizeof(*ctx), &ctx->head.size, err); + __put_user_error(gcspr, &ctx->gcspr, err); + __put_user_error(0, &ctx->reserved, err); + __put_user_error(current->thread.gcs_el0_mode, + &ctx->features_enabled, err); + + return err; +} + +static int restore_gcs_context(struct user_ctxs *user) +{ + u64 gcspr, enabled; + int err = 0; + + if (user->gcs_size != sizeof(*user->gcs)) + return -EINVAL; + + __get_user_error(gcspr, &user->gcs->gcspr, err); + __get_user_error(enabled, &user->gcs->features_enabled, err); + if (err) + return err; + + /* Don't allow unknown modes */ + if (enabled & ~PR_SHADOW_STACK_SUPPORTED_STATUS_MASK) + return -EINVAL; + + err = gcs_check_locked(current, enabled); + if (err != 0) + return err; + + /* Don't allow enabling */ + if (!task_gcs_el0_enabled(current) && + (enabled & PR_SHADOW_STACK_ENABLE)) + return -EINVAL; + + /* If we are disabling disable everything */ + if (!(enabled & PR_SHADOW_STACK_ENABLE)) + enabled = 0; + + current->thread.gcs_el0_mode = enabled; + + /* + * We let userspace set GCSPR_EL0 to anything here, we will + * validate later in gcs_restore_signal(). + */ + current->thread.gcspr_el0 = gcspr; + write_sysreg_s(current->thread.gcspr_el0, SYS_GCSPR_EL0); + + return 0; +} + +#else /* ! CONFIG_ARM64_GCS */ + +/* Turn any non-optimised out attempts to use these into a link error: */ +extern int preserve_gcs_context(void __user *ctx); +extern int restore_gcs_context(struct user_ctxs *user); + +#endif /* ! CONFIG_ARM64_GCS */ + static int parse_user_sigframe(struct user_ctxs *user, struct rt_sigframe __user *sf) { @@ -622,6 +702,7 @@ static int parse_user_sigframe(struct user_ctxs *user, user->tpidr2 = NULL; user->za = NULL; user->zt = NULL; + user->gcs = NULL; if (!IS_ALIGNED((unsigned long)base, 16)) goto invalid; @@ -716,6 +797,17 @@ static int parse_user_sigframe(struct user_ctxs *user, user->zt_size = size; break; + case GCS_MAGIC: + if (!system_supports_gcs()) + goto invalid; + + if (user->gcs) + goto invalid; + + user->gcs = (struct gcs_context __user *)head; + user->gcs_size = size; + break; + case EXTRA_MAGIC: if (have_extra_context) goto invalid; @@ -835,6 +927,9 @@ static int restore_sigframe(struct pt_regs *regs, err = restore_fpsimd_context(&user); } + if (err == 0 && system_supports_gcs() && user.gcs) + err = restore_gcs_context(&user); + if (err == 0 && system_supports_tpidr2() && user.tpidr2) err = restore_tpidr2_context(&user); @@ -954,6 +1049,13 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user, return err; } + if (system_supports_gcs()) { + err = sigframe_alloc(user, &user->gcs_offset, + sizeof(struct gcs_context)); + if (err) + return err; + } + if (system_supports_sve() || system_supports_sme()) { unsigned int vq = 0; @@ -1047,6 +1149,12 @@ static int setup_sigframe(struct rt_sigframe_user_layout *user, __put_user_error(current->thread.fault_code, &esr_ctx->esr, err); } + if (system_supports_gcs() && err == 0 && user->gcs_offset) { + struct gcs_context __user *gcs_ctx = + apply_user_offset(user, user->gcs_offset); + err |= preserve_gcs_context(gcs_ctx); + } + /* Scalable Vector Extension state (including streaming), if present */ if ((system_supports_sve() || system_supports_sme()) && err == 0 && user->sve_offset) { From patchwork Sat Feb 3 12:25:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769684 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5680C7762E; Sat, 3 Feb 2024 12:32:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963557; cv=none; b=AcSoHrn0wlCXoR9yjsSQGLfO1T9qMwkwiXzg+TPYNlq6IfJt+m1IGm4oXCSWUiqjUhbntdM80Vo9hfs1PvURbtWdNlFyCUYFdA9SZ0j/6wUXp4f4a6DDJaQ9xVKzIdGrTozCQN8Khsv8rptfeMxG5YKSVhsEsmx30x3qhSKVdeE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963557; c=relaxed/simple; bh=RGzSIrmEW7Jf6evuT6565KZp1tpVwBC25PeiKV/XCIM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AclgKjta+zioEIyu8IYfUJ6Wn3CSDh6NJGRZ6LqG7FFfqIOL4Sjlp1GkfEYrphudNBXzXy4r/U6JaLZBuz2yJXsfVnnq7V/ESX5PLJTCaK0ndso9JO4PLwv+xynMM8uFJeuz4obEXd4j1y+HERLr4qbzAOpybJ0dOR7KLKnbY+o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Rr0NlCUf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Rr0NlCUf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76DADC433F1; Sat, 3 Feb 2024 12:32:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963556; bh=RGzSIrmEW7Jf6evuT6565KZp1tpVwBC25PeiKV/XCIM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Rr0NlCUfUX2j3myBULSLJPalVM826rFJl0vTY3fYPNIAoGjsaWyYKxtBifaNm6sDv R3k3Z+e3/jgWq5+5PInC1Osi52ku2mCr6D9o1aURAOeLt+R2Zkk5U6R2wCML7xz6zk PG81g8AMxl2rVOrJUNnz2N35u5ho9Xg89aJIJj9OoWN9wFUIZ/qvSjm3ZM8I4h9hFC AbMSyA++bvK+sdEPPG3qzzEnAj62MLnGKSS32CcHHeeK/jUMKkKjE6RdyAZ+tvsviN dDdf/wFFm15nQghsG5q3BA7MRI+OHsHHN8xmcmlC7HDsvlRem/d2ChYl+uIz2gqkTZ rhMiIq5b7EusQ== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:52 +0000 Subject: [PATCH v8 26/38] arm64: Add Kconfig for Guarded Control Stack (GCS) Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-26-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=1372; i=broonie@kernel.org; h=from:subject:message-id; bh=RGzSIrmEW7Jf6evuT6565KZp1tpVwBC25PeiKV/XCIM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDlwn8OBL1okRa937HDPu7LnBF8+tPg3W5TVRmQ TyRiYLiJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w5QAKCRAk1otyXVSH0GddB/ 94PnXcnxhx9d3Igx3+8wyrY4sdF5KYIjmDuj3squkXqrrZ2tiirA6Q62mQp8vp+ezqY1jRXa4sYuJ6 S4br3364VBFwZURGrMoqdURAvnmTRIVSNGGYe6TRIoFURRVmXmsSb1Zx9GeUJE+887P2YThSn1hX4u UYhHuNxxIuI+/hGpYdkS35l+885fx4nMPUte+GNr+FvdEa6vqVjtRScIfSuKLmDgZ/N4wSajZL/k1q rBPuJY2ifU59ibgG/D9dxZieNsgex3LyQf29V2Ibhl6yWuOIeyY9K15eIuhbvVWh0jnwxw8K8dMwt4 yP307QaPCE4iW0R2nFKNxCYMzLGxW/ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide a Kconfig option allowing the user to select if GCS support is built into the kernel. Signed-off-by: Mark Brown --- arch/arm64/Kconfig | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index aa7c1d435139..e0048e4660cf 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2098,6 +2098,26 @@ config ARM64_EPAN if the cpu does not implement the feature. endmenu # "ARMv8.7 architectural features" +menu "v9.4 architectural features" + +config ARM64_GCS + bool "Enable support for Guarded Control Stack (GCS)" + default y + select ARCH_HAS_USER_SHADOW_STACK + select ARCH_USES_HIGH_VMA_FLAGS + help + Guarded Control Stack (GCS) provides support for a separate + stack with restricted access which contains only return + addresses. This can be used to harden against some attacks + by comparing return address used by the program with what is + stored in the GCS, and may also be used to efficiently obtain + the call stack for applications such as profiling. + + The feature is detected at runtime, and will remain disabled + if the system does not implement the feature. + +endmenu # "v9.4 architectural features" + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y From patchwork Sat Feb 3 12:25:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769683 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97A0D79DB0; Sat, 3 Feb 2024 12:32:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963570; cv=none; b=mGiAlkw7T1bmYPEO1BgWtUWHL41hA5u1FqdtBlS0ddjVfWo2y5djrm5XPGAKuy7b/wXXVgElyQMaq4jqq+7tn/hgtbqF6CfZLRKa3syXh19Qby+Ramu+Vzwf4YDkSBs1EUsvIpChbLSnO1afYh4bdqrOxJuqHUxuhfob+NKI3hE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963570; c=relaxed/simple; bh=Go+3eKanODPQVKnV1QnMIrC75/lXwO2PkPnkMfOn2oA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bFTPEWvMwRQU6jJ5T4Pyq/DQkadCilquxZVyCmiLMH6wRTEnW3dQIAnhItx2dT50LpANXtncoKhd6JfOU+OwmjC467Y7ve6Vq4/0ChUtcodsJPe61Gg3LQE45/bLUU1Kl4sV3RWCSy41Zr/pdTg7WeEX6kj2Bk/KXixEp9ae0Ps= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DX3pD5b6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DX3pD5b6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D77D1C43390; Sat, 3 Feb 2024 12:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963570; bh=Go+3eKanODPQVKnV1QnMIrC75/lXwO2PkPnkMfOn2oA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=DX3pD5b66LMn5s9XCPzNiW8Ohd0be3iU+tGL+cyFljRDq6foySmJjsTZF1YctTL+3 s1odq9u5fazA79LfV23TfXoDxAtZ6G40eneN6LwjBBYmJ5WJD9mun8qf0d6dtAkQa0 0NQ03qC16gpi8tG7LZcW4KBDLgGBtNj2OLhPTLCa25Ia7G48F3rARD0QaEPgNJ9NMq QNs6oA55JJ1CqAy51xQaC+t4SE50oVOtTfR5Uwu+ukSBphjlu+rJuTznNA/znOKVi9 6x3lT2ibIvsjY+VRHM2uE7Tc9JFxWeSkWXYS4epLtICQ13T/OlHd0vfdNUyLQyCe4e w0J+0hm3/1drQ== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:54 +0000 Subject: [PATCH v8 28/38] kselftest/arm64: Add GCS as a detected feature in the signal tests Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-28-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=1828; i=broonie@kernel.org; h=from:subject:message-id; bh=Go+3eKanODPQVKnV1QnMIrC75/lXwO2PkPnkMfOn2oA=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDnv4va7Xw8q+9IPj8NpQvVrRWDKQAKGymqiuH9 vVzGLcWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w5wAKCRAk1otyXVSH0La0B/ 4lCc/SwjJvxVcIvBZtR884QVIBbVQHrqffIBeJOToitxCKeral+Vi14paZlCsB+z4lQqHnwXqSmU1O LgytJMK+lHJNHXVwX6qBDliivP28+wkHYFGM5dVzlnuaHLYaWCvcrTt9FuxcycfdKrGxh6Yrlcns6P 5cO/ZComrZ+hmwB9ih+IDFWZhyVXl+X/T9oGJpK1OYbUkWxkE5eWPOqIViydw/Yy1G5D0LcoqvAA+0 nXqkFzSboRRYecAvqq/m1WCqmXukbEeCLSo7TT9ptOZVWoaztMIhE98hlZzzBVTupuJs1gusBll95R l9oXxqT+rgpyun7GAjAkca6C9qG6hz X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for testing GCS related signal handling add it as a feature we check for in the signal handling support code. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/signal/test_signals.h | 2 ++ tools/testing/selftests/arm64/signal/test_signals_utils.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/tools/testing/selftests/arm64/signal/test_signals.h b/tools/testing/selftests/arm64/signal/test_signals.h index 1e6273d81575..7ada43688c02 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.h +++ b/tools/testing/selftests/arm64/signal/test_signals.h @@ -35,6 +35,7 @@ enum { FSME_BIT, FSME_FA64_BIT, FSME2_BIT, + FGCS_BIT, FMAX_END }; @@ -43,6 +44,7 @@ enum { #define FEAT_SME (1UL << FSME_BIT) #define FEAT_SME_FA64 (1UL << FSME_FA64_BIT) #define FEAT_SME2 (1UL << FSME2_BIT) +#define FEAT_GCS (1UL << FGCS_BIT) /* * A descriptor used to describe and configure a test case. diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c index 0dc948db3a4a..89ef95c1af0e 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.c +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c @@ -30,6 +30,7 @@ static char const *const feats_names[FMAX_END] = { " SME ", " FA64 ", " SME2 ", + " GCS ", }; #define MAX_FEATS_SZ 128 @@ -329,6 +330,8 @@ int test_init(struct tdescr *td) td->feats_supported |= FEAT_SME_FA64; if (getauxval(AT_HWCAP2) & HWCAP2_SME2) td->feats_supported |= FEAT_SME2; + if (getauxval(AT_HWCAP2) & HWCAP2_GCS) + td->feats_supported |= FEAT_GCS; if (feats_ok(td)) { if (td->feats_required & td->feats_supported) fprintf(stderr, From patchwork Sat Feb 3 12:25:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769682 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05B9D7C094; Sat, 3 Feb 2024 12:33:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963584; cv=none; b=kllUY4sORyQS7GTYbVhn/GV6kFBjnkAoAld/gRnnryvy6kRaJxlMFtUG430pWszrGDH+ybhbZyeMXNsf5LUVjCDsyAgnQBAeoI9NpiiG0gnyFiomYDdoNpcwJSpzM469mHqIhUrdjx9T3/5pr7IxLn5r17IkSQ0kBiHa2pPMKAk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963584; c=relaxed/simple; bh=JCh6BoCIOxePO8zZIr/FO/CERnc7ZRYHEz9dpM/Qpg4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=vAnnEMyda31T70R5pSi5dHyVkYgiEpQyPXiQ+5fkfgWNBPm4x01CL5DGNXMpwz0G6jQl/Px8c1Mv7BU/wzBxUFshyHGnXeb+Pq5IkWczy2PaPdOS+/Axxg7XuiSLqnkU5sLfBu+xG1eEFwOtYYudkaYuAcV7qdjxs1GIE2dJ2t8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kqkNRTd3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kqkNRTd3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4CF52C43390; Sat, 3 Feb 2024 12:32:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963583; bh=JCh6BoCIOxePO8zZIr/FO/CERnc7ZRYHEz9dpM/Qpg4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kqkNRTd3I3lO3QHFD5UjLkvOmXQImqFY3hWHEUllrYLLdpP6+BXaYy4YzYJpIQfiI 2kHLBae1c3Xq/sQpeqs2cTCFeqKAzm2GWG98antWcjX06gw7CwAcB8sGoWImSmbV/w wyflXPYe7lInBio/hiz7dnB6MaDG5vFzEnie76b+PFqOpjJwi04lDvhCirR0sYEstt Osi7TWKazNTZlnXlSoBwbArXQxLqLF+EcZfBxzWRLpsvG2QYf3yRTP1bdczvlPmBcF FWwLyJCk+HVbPq+4OqXrI/U2oQFk8+dEj1kiPmSnxfaxgaNatAxo25FP09LCxaN/sf Dd2IR+IMxzJ5g== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:56 +0000 Subject: [PATCH v8 30/38] kselftest/arm64: Allow signals tests to specify an expected si_code Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-30-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=2627; i=broonie@kernel.org; h=from:subject:message-id; bh=JCh6BoCIOxePO8zZIr/FO/CERnc7ZRYHEz9dpM/Qpg4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDo38gIcK9LJf7YTIjfUSjpLDV2o1lpaAkCiIJC kt2+aTiJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w6AAKCRAk1otyXVSH0ALzB/ 4mRIohJMqHEtEnxH9wC0dGcy1NF3GJGVuohQBQ1SeayArCc3f3SjqKV52kokunFJEEUuSE49WPF9fJ i1lvqhrAg971iuSllKwjK15OdqSMq8uAVlbyXSeh1UpRu978IpinABVAJmFzVeBhsxRlAdH9j88WBx kAF5vqmLYu3dBH1x0Lf2hMVMye0R21jtWPC2adssJAG5oI2lzq7kPZH0P2HbAvcBqegDeN0fHuVdA2 55NxduDCGp22NvYO4Jj4QstU5qHqGmFAw/t1E7RoLgStYY9aKRX0ItbjaUV03z2h4aOqP+0Wy4Puax E6I7vX3cLSpZLvELwVpM5UmmmNf1rV X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently we ignore si_code unless the expected signal is a SIGSEGV, in which case we enforce it being SEGV_ACCERR. Allow test cases to specify exactly which si_code should be generated so we can validate this, and test for other segfault codes. Signed-off-by: Mark Brown --- .../testing/selftests/arm64/signal/test_signals.h | 4 +++ .../selftests/arm64/signal/test_signals_utils.c | 29 ++++++++++++++-------- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/arm64/signal/test_signals.h b/tools/testing/selftests/arm64/signal/test_signals.h index 7ada43688c02..ee75a2c25ce7 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.h +++ b/tools/testing/selftests/arm64/signal/test_signals.h @@ -71,6 +71,10 @@ struct tdescr { * Zero when no signal is expected on success */ int sig_ok; + /* + * expected si_code for sig_ok, or 0 to not check + */ + int sig_ok_code; /* signum expected on unsupported CPU features. */ int sig_unsupp; /* a timeout in second for test completion */ diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c index 89ef95c1af0e..63deca32b0df 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.c +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c @@ -143,16 +143,25 @@ static bool handle_signal_ok(struct tdescr *td, "current->token ZEROED...test is probably broken!\n"); abort(); } - /* - * Trying to narrow down the SEGV to the ones generated by Kernel itself - * via arm64_notify_segfault(). This is a best-effort check anyway, and - * the si_code check may need to change if this aspect of the kernel - * ABI changes. - */ - if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { - fprintf(stdout, - "si_code != SEGV_ACCERR...test is probably broken!\n"); - abort(); + if (td->sig_ok_code) { + if (si->si_code != td->sig_ok_code) { + fprintf(stdout, "si_code is %d not %d\n", + si->si_code, td->sig_ok_code); + abort(); + } + } else { + /* + * Trying to narrow down the SEGV to the ones + * generated by Kernel itself via + * arm64_notify_segfault(). This is a best-effort + * check anyway, and the si_code check may need to + * change if this aspect of the kernel ABI changes. + */ + if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { + fprintf(stdout, + "si_code != SEGV_ACCERR...test is probably broken!\n"); + abort(); + } } td->pass = 1; /* From patchwork Sat Feb 3 12:25:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769681 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2173E7D401; Sat, 3 Feb 2024 12:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963601; cv=none; b=bN5TMg7hGapYr/b0sjqb8ADY6m2ZwJdTNmN3UxAEHO6YanaLaMmXbis7ejXqkWN4TDTPHkMBsfeWsfJJ7tiR45id6ouYgAbvodY2tk2x6tQeUHiJJwLEdRHn1WeDLeu184TJlU0YghjVvPfZEd5PL6St6NBALfcXo4LfLGo0+UA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963601; c=relaxed/simple; bh=ybwVQt+4Fd4Hh9j4CJYVgeQo5WwMX/HVaBmXstiKNFM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=K8zIlMwhUVXSXgIXSBTAHpTiMz8Xr9CPnI+trCEoLmEAaVQkRp8evbT9Q1YvjNsqA3ZmYAhyUMjTtNc1dRWiVL/7No2dfMTYTgyhAvzzvlsEUgx/kUx09Xw+8nhBBDRAz92U9+g9os/OruZSjSsmWxaCmlU/eqaRMX58fFWKdv0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CGfK5dcd; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CGfK5dcd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 92F5BC433C7; Sat, 3 Feb 2024 12:33:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963600; bh=ybwVQt+4Fd4Hh9j4CJYVgeQo5WwMX/HVaBmXstiKNFM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=CGfK5dcd4VLwXjYN9T3hD+uXAGUW4515O2wxiBuq70g1l4pOvDGP8fl1CoO3JbZ7r dr6hggKkIxTO/0jVBovhRFL8geYc+6Dle9jaVoWTUtLTpgSvrxKjSiYF6M8KdSkwfc 4g46Lt850l6imjIPm1jgTR5t0JaSK46GymfWhtb4EQcOgwCK0LGII9sr0Vb5w+RLJz KQ8ajonFJeKgAsZA+qbYrPoCk1EkG3KJqJE2WLo4QOrIPidYC8Mh72TOjNcJefYjT0 K9xZcHw7ja0CjkLc/cxUmo98dGPyHwXvBLBUpH+UspBNBsGyBOcJN7Xy9UWUDH4CNK NskyP0jv5K8nQ== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:58 +0000 Subject: [PATCH v8 32/38] kselftest/arm64: Add very basic GCS test program Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-32-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=14970; i=broonie@kernel.org; h=from:subject:message-id; bh=ybwVQt+4Fd4Hh9j4CJYVgeQo5WwMX/HVaBmXstiKNFM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDqOndUREajYz9qo7ymvrITUA3lHYtCvP1xnyVj w4L6YKaJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w6gAKCRAk1otyXVSH0ESjB/ 9DvlDkzJVO4w7I8UJxuMtOvg2qOu3uEOrNOZSqkLtrgaLbV3cFsOU+YLEyZW7a6VdsDPTI7ZY4DFka x6260nt8xNnDGKmSkHqRPubmsHz8IVfiyLSMoGNOT+EMnOQ5fMB9OD0jAp9Q6Jy15ByUr+mA7gTmYa 5FCRMV/CfVgYD5wNbZmYbiVa532otV0lnf7v8AZuro4eR7zFkiip2vrNQlImDA0FfzoT1/7LnPYfq6 s0gxM2QEJSuDB6z+Q42dszuvPdZFF3eugFyDbeUOz1lieOBC8JeTb77+1x6gHL4J8eeO5dse2WYpV5 B4xr0ZQZBOQGk3mVX6c8or0TxDUCYV X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB This test program just covers the basic GCS ABI, covering aspects of the ABI as standalone features without attempting to integrate things. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/Makefile | 2 +- tools/testing/selftests/arm64/gcs/.gitignore | 1 + tools/testing/selftests/arm64/gcs/Makefile | 18 ++ tools/testing/selftests/arm64/gcs/basic-gcs.c | 428 ++++++++++++++++++++++++++ tools/testing/selftests/arm64/gcs/gcs-util.h | 90 ++++++ 5 files changed, 538 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile index 28b93cab8c0d..22029e60eff3 100644 --- a/tools/testing/selftests/arm64/Makefile +++ b/tools/testing/selftests/arm64/Makefile @@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs else ARM64_SUBTARGETS := endif diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore new file mode 100644 index 000000000000..0e5e695ecba5 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -0,0 +1 @@ +basic-gcs diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile new file mode 100644 index 000000000000..61a30f483429 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 ARM Limited +# +# In order to avoid interaction with the toolchain and dynamic linker the +# portions of these tests that interact with the GCS are implemented using +# nolibc. +# + +TEST_GEN_PROGS := basic-gcs + +include ../../lib.mk + +$(OUTPUT)/basic-gcs: basic-gcs.c + $(CC) -g -fno-asynchronous-unwind-tables -fno-ident -s -Os -nostdlib \ + -static -include ../../../../include/nolibc/nolibc.h \ + -I../../../../../usr/include \ + -std=gnu99 -I../.. -g \ + -ffreestanding -Wall $^ -o $@ -lgcc diff --git a/tools/testing/selftests/arm64/gcs/basic-gcs.c b/tools/testing/selftests/arm64/gcs/basic-gcs.c new file mode 100644 index 000000000000..3ef957e0065f --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/basic-gcs.c @@ -0,0 +1,428 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Limited. + */ + +#include +#include + +#include + +#include +#include +#include + +#include "kselftest.h" +#include "gcs-util.h" + +/* nolibc doesn't have sysconf(), just hard code the maximum */ +static size_t page_size = 65536; + +static __attribute__((noinline)) void valid_gcs_function(void) +{ + /* Do something the compiler can't optimise out */ + my_syscall1(__NR_prctl, PR_SVE_GET_VL); +} + +static inline int gcs_set_status(unsigned long mode) +{ + bool enabling = mode & PR_SHADOW_STACK_ENABLE; + int ret; + unsigned long new_mode; + + /* + * The prctl takes 1 argument but we need to ensure that the + * other 3 values passed in registers to the syscall are zero + * since the kernel validates them. + */ + ret = my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, mode, + 0, 0, 0); + + if (ret == 0) { + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &new_mode, 0, 0, 0); + if (ret == 0) { + if (new_mode != mode) { + ksft_print_msg("Mode set to %x not %x\n", + new_mode, mode); + ret = -EINVAL; + } + } else { + ksft_print_msg("Failed to validate mode: %d\n", ret); + } + + if (enabling != chkfeat_gcs()) { + ksft_print_msg("%senabled by prctl but %senabled in CHKFEAT\n", + enabling ? "" : "not ", + chkfeat_gcs() ? "" : "not "); + ret = -EINVAL; + } + } + + return ret; +} + +/* Try to read the status */ +static bool read_status(void) +{ + unsigned long state; + int ret; + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &state, 0, 0, 0); + if (ret != 0) { + ksft_print_msg("Failed to read state: %d\n", ret); + return false; + } + + return state & PR_SHADOW_STACK_ENABLE; +} + +/* Just a straight enable */ +static bool base_enable(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE failed %d\n", ret); + return false; + } + + return true; +} + +/* Check we can read GCSPR_EL0 when GCS is enabled */ +static bool read_gcspr_el0(void) +{ + unsigned long *gcspr_el0; + + ksft_print_msg("GET GCSPR\n"); + gcspr_el0 = get_gcspr(); + ksft_print_msg("GCSPR_EL0 is %p\n", gcspr_el0); + + return true; +} + +/* Also allow writes to stack */ +static bool enable_writeable(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE writeable failed: %d\n", ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +/* Also allow writes to stack */ +static bool enable_push_pop(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE with push failed: %d\n", + ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +/* Enable GCS and allow everything */ +static bool enable_all(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH | + PR_SHADOW_STACK_WRITE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE with everything failed: %d\n", + ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +static bool enable_invalid(void) +{ + int ret = gcs_set_status(ULONG_MAX); + if (ret == 0) { + ksft_print_msg("GCS_SET_STATUS %lx succeeded\n", ULONG_MAX); + return false; + } + + return true; +} + +/* Map a GCS */ +static bool map_guarded_stack(void) +{ + int ret; + uint64_t *buf; + uint64_t expected_cap; + int elem; + bool pass = true; + + buf = (void *)my_syscall3(__NR_map_shadow_stack, 0, page_size, + SHADOW_STACK_SET_MARKER | + SHADOW_STACK_SET_TOKEN); + if (buf == MAP_FAILED) { + ksft_print_msg("Failed to map %d byte GCS: %d\n", + page_size, errno); + return false; + } + ksft_print_msg("Mapped GCS at %p-%p\n", buf, + (uint64_t)buf + page_size); + + /* The top of the newly allocated region should be 0 */ + elem = (page_size / sizeof(uint64_t)) - 1; + if (buf[elem]) { + ksft_print_msg("Last entry is 0x%lx not 0x0\n", buf[elem]); + pass = false; + } + + /* Then a valid cap token */ + elem--; + expected_cap = ((uint64_t)buf + page_size - 16); + expected_cap &= GCS_CAP_ADDR_MASK; + expected_cap |= GCS_CAP_VALID_TOKEN; + if (buf[elem] != expected_cap) { + ksft_print_msg("Cap entry is 0x%lx not 0x%lx\n", + buf[elem], expected_cap); + pass = false; + } + ksft_print_msg("cap token is 0x%lx\n", buf[elem]); + + /* The rest should be zeros */ + for (elem = 0; elem < page_size / sizeof(uint64_t) - 2; elem++) { + if (!buf[elem]) + continue; + ksft_print_msg("GCS slot %d is 0x%lx not 0x0\n", + elem, buf[elem]); + pass = false; + } + + ret = munmap(buf, page_size); + if (ret != 0) { + ksft_print_msg("Failed to unmap %d byte GCS: %d\n", + page_size, errno); + pass = false; + } + + return pass; +} + +/* A fork()ed process can run */ +static bool test_fork(void) +{ + unsigned long child_mode; + int ret, status; + pid_t pid; + bool pass = true; + + pid = fork(); + if (pid == -1) { + ksft_print_msg("fork() failed: %d\n", errno); + pass = false; + goto out; + } + if (pid == 0) { + /* In child, make sure we can call a function, read + * the GCS pointer and status and then exit */ + valid_gcs_function(); + get_gcspr(); + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &child_mode, 0, 0, 0); + if (ret == 0 && !(child_mode & PR_SHADOW_STACK_ENABLE)) { + ksft_print_msg("GCS not enabled in child\n"); + ret = -EINVAL; + } + + exit(ret); + } + + /* + * In parent, check we can still do function calls then block + * for the child. + */ + valid_gcs_function(); + + ksft_print_msg("Waiting for child %d\n", pid); + + ret = waitpid(pid, &status, 0); + if (ret == -1) { + ksft_print_msg("Failed to wait for child: %d\n", + errno); + return false; + } + + if (!WIFEXITED(status)) { + ksft_print_msg("Child exited due to signal %d\n", + WTERMSIG(status)); + pass = false; + } else { + if (WEXITSTATUS(status)) { + ksft_print_msg("Child exited with status %d\n", + WEXITSTATUS(status)); + pass = false; + } + } + +out: + + return pass; +} + +/* Check that we can explicitly specify a GCS via clone3() */ +static bool test_clone3(void) +{ + struct clone_args args; + unsigned long child_mode; + pid_t pid = -1; + int status, ret; + bool pass; + + memset(&args, 0, sizeof(args)); + args.flags = CLONE_VM; + args.shadow_stack_size = page_size; + + pid = my_syscall2(__NR_clone3, &args, sizeof(args)); + if (pid < 0) { + ksft_print_msg("clone3() failed: %d\n", errno); + pass = false; + goto out; + } + + /* In child? */ + if (pid == 0) { + /* Do we have GCS enabled? */ + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &child_mode, 0, 0, 0); + if (ret != 0) { + ksft_print_msg("PR_GET_SHADOW_STACK_STATUS failed: %d\n", + ret); + exit(EXIT_FAILURE); + } + + if (!(child_mode & PR_SHADOW_STACK_ENABLE)) { + ksft_print_msg("GCS not enabled in child\n"); + exit(EXIT_FAILURE); + } + + ksft_print_msg("GCS enabled in child\n"); + + /* We've probably already called a function but make sure */ + valid_gcs_function(); + + exit(EXIT_SUCCESS); + } + + if (waitpid(-1, &status, __WALL) < 0) { + ksft_print_msg("waitpid() failed %d\n", errno); + pass = false; + goto out; + } + if (WIFEXITED(status)) { + if (WEXITSTATUS(status) == EXIT_SUCCESS) { + pass = true; + } else { + ksft_print_msg("Child returned status %d\n", + WEXITSTATUS(status)); + pass = false; + } + } else if (WIFSIGNALED(status)) { + ksft_print_msg("Child exited due to signal %d\n", + WTERMSIG(status)); + pass = false; + } else { + ksft_print_msg("Child exited uncleanly\n"); + pass = false; + } + +out: + return pass; +} + +typedef bool (*gcs_test)(void); + +static struct { + char *name; + gcs_test test; + bool needs_enable; +} tests[] = { + { "read_status", read_status }, + { "base_enable", base_enable, true }, + { "read_gcspr_el0", read_gcspr_el0 }, + { "enable_writeable", enable_writeable, true }, + { "enable_push_pop", enable_push_pop, true }, + { "enable_all", enable_all, true }, + { "enable_invalid", enable_invalid, true }, + { "map_guarded_stack", map_guarded_stack }, + { "fork", test_fork }, + { "clone3", test_clone3 }, +}; + +int main(void) +{ + int i, ret; + unsigned long gcs_mode; + + ksft_print_header(); + + /* + * We don't have getauxval() with nolibc so treat a failure to + * read GCS state as a lack of support and skip. + */ + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &gcs_mode, 0, 0, 0); + if (ret != 0) + ksft_exit_skip("Failed to read GCS state: %d\n", ret); + + if (!(gcs_mode & PR_SHADOW_STACK_ENABLE)) { + gcs_mode = PR_SHADOW_STACK_ENABLE; + ret = my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + gcs_mode, 0, 0, 0); + if (ret != 0) + ksft_exit_fail_msg("Failed to enable GCS: %d\n", ret); + } + + ksft_set_plan(ARRAY_SIZE(tests)); + + for (i = 0; i < ARRAY_SIZE(tests); i++) { + ksft_test_result((*tests[i].test)(), "%s\n", tests[i].name); + } + + /* One last test: disable GCS, we can do this one time */ + my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, 0, 0, 0, 0); + if (ret != 0) + ksft_print_msg("Failed to disable GCS: %d\n", ret); + + ksft_finished(); + + return 0; +} diff --git a/tools/testing/selftests/arm64/gcs/gcs-util.h b/tools/testing/selftests/arm64/gcs/gcs-util.h new file mode 100644 index 000000000000..b37801c95604 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-util.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2023 ARM Limited. + */ + +#ifndef GCS_UTIL_H +#define GCS_UTIL_H + +#include + +#ifndef __NR_map_shadow_stack +#define __NR_map_shadow_stack 453 +#endif + +#ifndef __NR_prctl +#define __NR_prctl 167 +#endif + +/* Shadow Stack/Guarded Control Stack interface */ +#define PR_GET_SHADOW_STACK_STATUS 71 +#define PR_SET_SHADOW_STACK_STATUS 72 +#define PR_LOCK_SHADOW_STACK_STATUS 73 + +# define PR_SHADOW_STACK_ENABLE (1UL << 0) +# define PR_SHADOW_STACK_WRITE (1UL << 1) +# define PR_SHADOW_STACK_PUSH (1UL << 2) + +#define PR_SHADOW_STACK_ALL_MODES \ + PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE | PR_SHADOW_STACK_PUSH + +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ +#define SHADOW_STACK_SET_MARKER (1ULL << 1) /* Set up a top of stack merker in the shadow stack */ + +#define GCS_CAP_ADDR_MASK (0xfffffffffffff000UL) +#define GCS_CAP_TOKEN_MASK (0x0000000000000fffUL) +#define GCS_CAP_VALID_TOKEN 1 +#define GCS_CAP_IN_PROGRESS_TOKEN 5 + +#define GCS_CAP(x) (((unsigned long)(x) & GCS_CAP_ADDR_MASK) | \ + GCS_CAP_VALID_TOKEN) + +static inline unsigned long *get_gcspr(void) +{ + unsigned long *gcspr; + + asm volatile( + "mrs %0, S3_3_C2_C5_1" + : "=r" (gcspr) + : + : "cc"); + + return gcspr; +} + +static inline void __attribute__((always_inline)) gcsss1(unsigned long *Xt) +{ + asm volatile ( + "sys #3, C7, C7, #2, %0\n" + : + : "rZ" (Xt) + : "memory"); +} + +static inline unsigned long __attribute__((always_inline)) *gcsss2(void) +{ + unsigned long *Xt; + + asm volatile( + "SYSL %0, #3, C7, C7, #3\n" + : "=r" (Xt) + : + : "memory"); + + return Xt; +} + +static inline bool chkfeat_gcs(void) +{ + register long val __asm__ ("x16") = 1; + + /* CHKFEAT x16 */ + asm volatile( + "hint #0x28\n" + : "=r" (val) + : "r" (val)); + + return val != 1; +} + +#endif From patchwork Sat Feb 3 12:26:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769680 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A1437E590; Sat, 3 Feb 2024 12:33:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963614; cv=none; b=dLW+ZH5bZR2DqQE+t47U6ds8vb5n+iSFapOHODEZqBpdENJZzKsRRqNpHxUjdUB6V+XTNTtlTt/isdqXzL6CLOYyEegm2Qpv3H9P/JrBwZZGvNb2Ee8qUUvWYiypMGpQiQ0PeAJ55cQoJ1K27TjUAmnu0IDymFs02L1qeUenOSc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963614; c=relaxed/simple; bh=IP7maWyjZ0dh+q8gDfrxLaUB03tkGEoqGc+9eYXpRWY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=hkWjA9zPgxJCIGqWHA3ZPVjUHxWxrRRFHrbedHN8/o/vuCi79RxKu9F3SCkY5wGdRJ4CiRZnvk3C5asMhpG4OmgM+gUHX5GbQt4Hj6BefaGV6HYK7AMqE5KLPyEAIPlb3lXiDTzWbhLlzWQgfSp6qcZYnQ6zbPPWwW+eknTPQ08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lUf867/I; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lUf867/I" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34C1BC43390; Sat, 3 Feb 2024 12:33:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963614; bh=IP7maWyjZ0dh+q8gDfrxLaUB03tkGEoqGc+9eYXpRWY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lUf867/IYTnlmfivw2vcu+gZCvSb7GVG5VRnmQDFmQg6rYyaC9H9noFVLBOwGRPQI FTgk6uVmgHEzoBo5+x//TTBxchZUYb8rWezBMc+hvg0rx8WxU39eU51BdH8BD9k7sW 8qodxPwPm0Jx+45GESf1OyDZGmNm12Q/SOwOnkRXNo42ZE+L3jayyH5Iff8Zqbyid6 WOqq7+RRoF5xs92MjUIQbI91kvt2KIm26eh8schxQ7aYO6kR56T3+85QU+sOh8oxC5 T4bIi+jyGxhMDvseI96rjz0oMMKWn5cboQVu28GB8sxKQ37hXTjdputCgIq5Ev23PO TxMYEyrbPaoAQ== From: Mark Brown Date: Sat, 03 Feb 2024 12:26:00 +0000 Subject: [PATCH v8 34/38] kselftest/arm64: Add test coverage for GCS mode locking Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-34-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=7316; i=broonie@kernel.org; h=from:subject:message-id; bh=IP7maWyjZ0dh+q8gDfrxLaUB03tkGEoqGc+9eYXpRWY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDrVwGlTHFqGQFU5FQOcbAFNfn+d/ZZXmK+epCj TesXUFaJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w6wAKCRAk1otyXVSH0Dp6B/ 427Q1kr1wSJrS9lqpLcIsIugUm1F07V9fnO08lyEDhIoUIji9R5YD/X5W6x86fryWV8QRrEURJNnch ff8OKTlfNgiK7I+P3kCBoH4XrthYuaP8TkHOZWBmVbzMcoiSqUWfsNrOWFSjkmUr5cl4XIvtsqBQXw 2eWAzZ4hG2rS9zSe2MeiuPOuV7Hlthlni0UXqgxDv7tzVpqG/DE3TCaglGxznDe+3HUvzVY2eqCaV1 ph4usWgUpK6+iQ7es7F1ve/a05w0HiDlk0fYq7Mo4hT2RFBko+oU+XW9NLjekEWFgSQk9bxV2EFvpl UhzbHjSupx5rz7/Dq/m7m00Y2RCq71 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Verify that we can lock individual GCS mode bits, that other modes aren't affected and as a side effect also that every combination of modes can be enabled. Normally the inability to reenable GCS after disabling it would be an issue with testing but fortunately the kselftest_harness runs each test within a fork()ed child. This can be inconvenient for some kinds of testing but here it means that each test is in a separate thread and therefore won't be affected by other tests in the suite. Once we get toolchains with support for enabling GCS by default we will need to take care to not do that in the build system but there are no such toolchains yet so it is not yet an issue. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/gcs/.gitignore | 1 + tools/testing/selftests/arm64/gcs/Makefile | 2 +- tools/testing/selftests/arm64/gcs/gcs-locking.c | 200 ++++++++++++++++++++++++ 3 files changed, 202 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore index 5810c4a163d4..0c86f53f68ad 100644 --- a/tools/testing/selftests/arm64/gcs/.gitignore +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -1,2 +1,3 @@ basic-gcs libc-gcs +gcs-locking diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile index a8fdf21e9a47..2173d6275956 100644 --- a/tools/testing/selftests/arm64/gcs/Makefile +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -6,7 +6,7 @@ # nolibc. # -TEST_GEN_PROGS := basic-gcs libc-gcs +TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking LDLIBS+=-lpthread diff --git a/tools/testing/selftests/arm64/gcs/gcs-locking.c b/tools/testing/selftests/arm64/gcs/gcs-locking.c new file mode 100644 index 000000000000..f6a73254317e --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-locking.c @@ -0,0 +1,200 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Limited. + * + * Tests for GCS mode locking. These tests rely on both having GCS + * unconfigured on entry and on the kselftest harness running each + * test in a fork()ed process which will have it's own mode. + */ + +#include + +#include +#include + +#include + +#include "kselftest_harness.h" + +#include "gcs-util.h" + +#define my_syscall2(num, arg1, arg2) \ +({ \ + register long _num __asm__ ("x8") = (num); \ + register long _arg1 __asm__ ("x0") = (long)(arg1); \ + register long _arg2 __asm__ ("x1") = (long)(arg2); \ + register long _arg3 __asm__ ("x2") = 0; \ + register long _arg4 __asm__ ("x3") = 0; \ + register long _arg5 __asm__ ("x4") = 0; \ + \ + __asm__ volatile ( \ + "svc #0\n" \ + : "=r"(_arg1) \ + : "r"(_arg1), "r"(_arg2), \ + "r"(_arg3), "r"(_arg4), \ + "r"(_arg5), "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +/* No mode bits are rejected for locking */ +TEST(lock_all_modes) +{ + int ret; + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, ULONG_MAX, 0, 0, 0); + ASSERT_EQ(ret, 0); +} + +FIXTURE(valid_modes) +{ +}; + +FIXTURE_VARIANT(valid_modes) +{ + unsigned long mode; +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable) +{ + .mode = PR_SHADOW_STACK_ENABLE, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_write) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_push) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_write_push) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE | + PR_SHADOW_STACK_PUSH, +}; + +FIXTURE_SETUP(valid_modes) +{ +} + +FIXTURE_TEARDOWN(valid_modes) +{ +} + +/* We can set the mode at all */ +TEST_F(valid_modes, set) +{ + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + _exit(0); +} + +/* Enabling, locking then disabling is rejected */ +TEST_F(valid_modes, enable_lock_disable) +{ + unsigned long mode; + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, 0); + ASSERT_EQ(ret, -EBUSY); + + _exit(0); +} + +/* Locking then enabling is rejected */ +TEST_F(valid_modes, lock_enable) +{ + unsigned long mode; + int ret; + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, -EBUSY); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, 0); + + _exit(0); +} + +/* Locking then changing other modes is fine */ +TEST_F(valid_modes, lock_enable_disable_others) +{ + unsigned long mode; + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + PR_SHADOW_STACK_ALL_MODES); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, PR_SHADOW_STACK_ALL_MODES); + + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + _exit(0); +} + +int main(int argc, char **argv) +{ + unsigned long mode; + int ret; + + if (!(getauxval(AT_HWCAP2) & HWCAP2_GCS)) + ksft_exit_skip("SKIP GCS not supported\n"); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + if (ret) { + ksft_print_msg("Failed to read GCS state: %d\n", ret); + return EXIT_FAILURE; + } + + if (mode & PR_SHADOW_STACK_ENABLE) { + ksft_print_msg("GCS was enabled, test unsupported\n"); + return KSFT_SKIP; + } + + return test_harness_run(argc, argv); +} From patchwork Sat Feb 3 12:26:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769679 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C7197A723; Sat, 3 Feb 2024 12:33:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963628; cv=none; b=fozyUz0kCFyc/CJffYQfhXlG0UY+7AdrWUmItN2wZ+UJgT5G+BnLxMrHgasQ5/ebS+EmDJXcKVUfOhan8QIb8OdfaxKGoRGQFVtodpAA5hG7aaS4NRR/jm+v0p+yXqdtiq3U+0cPnCySxcLxloJN4GVT7grNDr/1OTB4mkL4U98= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963628; c=relaxed/simple; bh=abP0OxQ39yBw88Uk/mZez3jim3UVxuH5Q3zwc7pr9ZE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WjTecl8ka2Sh7fvPaLnxqcC1BWXnIaan35YPD3Gmr9u7WcabeVpZZ/25zRCGRGuS1fuDSD3qd+FKwckX6t4PmUCIhXeqYvfXYuZXy9J9bO1JX3IfR9xOU0feGGqV/kr8JUY7a9kOOjztkjr8uaQ7Dbvat5fZZl6HxV1ABHhPYAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KcEvtstn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KcEvtstn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9A940C433C7; Sat, 3 Feb 2024 12:33:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963628; bh=abP0OxQ39yBw88Uk/mZez3jim3UVxuH5Q3zwc7pr9ZE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=KcEvtstng6Yv6LMO1Y0y2MuMyuTaVPBXWSGu3TjzDD/lKrnxvjSjL5FxGnQWVDYBP tSlGhP4EEEppcBMkZPSUo5scwpocVKO8QxahYCLJAhjiFMFBdeJ2OHOp1PAiW0QTfe nVpevBnSR29leggI/OI/kl/KtH+5SYvih0+DPAXP6yJ1gljVu2mWQFFREJFODVP+hV hInJvPWCdcGMGq4/a3cGTi4J+XK+AXM7ERU83ADsCNt9al1uelKR17SyPeWjZiWmgm +yZS7WcByyTVUMj6h71PtbvYFi/urqdutpLZm4IN9iaFd7Pi/a0qyy1TTKPsA3FpM6 PkWugelAgzpCw== From: Mark Brown Date: Sat, 03 Feb 2024 12:26:02 +0000 Subject: [PATCH v8 36/38] kselftest/arm64: Add a GCS stress test Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-36-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=21146; i=broonie@kernel.org; h=from:subject:message-id; bh=abP0OxQ39yBw88Uk/mZez3jim3UVxuH5Q3zwc7pr9ZE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDtyrNyZQ55VaPTu7vstnlD+Qg7ntn5oVLklrA/ iTlo9AWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w7QAKCRAk1otyXVSH0C60B/ 9B+N9fAtIc8rM4bR1zVzPPUuTULn5DHPJzJ7ToQHNfE9g4tpf9G/wRs5DS0WyclGWu9SVML50OC9jQ tJ1nqaXLZC4WugY8+IpdOyeGB+rh1MaZrFup7SQLXnWFKVdq5OYiVbTqT3y4n/hiJnFCmGpE0z2T/j dz4Vsn3aU0Vx9QTCkOeyk37sQEvCkxGpWVV7zgBK1dhry80mt2UA3+CJDYk10AxQnXVNAzN1kCvd/E fCKIEXiLXN0OzS11DkRUzbE7is7BL/uAln4wjNI6iM7rZn1vkRZ4kV1yR6MLDihbvjbqqnj+mTe4lt cWNJ4wcQ9gzbxxLBU9TADvD6QmyfpV X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add a stress test which runs one more process than we have CPUs spinning through a very recursive function with frequent syscalls immediately prior to return and signals being injected every 100ms. The goal is to flag up any scheduling related issues, for example failure to ensure that barriers are inserted when moving a GCS using task to another CPU. The test runs for a configurable amount of time, defaulting to 10 seconds. Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/gcs/.gitignore | 2 + tools/testing/selftests/arm64/gcs/Makefile | 6 +- tools/testing/selftests/arm64/gcs/asm-offsets.h | 0 .../selftests/arm64/gcs/gcs-stress-thread.S | 311 ++++++++++++ tools/testing/selftests/arm64/gcs/gcs-stress.c | 532 +++++++++++++++++++++ 5 files changed, 850 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore index 0c86f53f68ad..1e8d1f6b27f2 100644 --- a/tools/testing/selftests/arm64/gcs/.gitignore +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -1,3 +1,5 @@ basic-gcs libc-gcs gcs-locking +gcs-stress +gcs-stress-thread diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile index 2173d6275956..d8b06ca51e22 100644 --- a/tools/testing/selftests/arm64/gcs/Makefile +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -6,7 +6,8 @@ # nolibc. # -TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking +TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking gcs-stress +TEST_GEN_PROGS_EXTENDED := gcs-stress-thread LDLIBS+=-lpthread @@ -18,3 +19,6 @@ $(OUTPUT)/basic-gcs: basic-gcs.c -I../../../../../usr/include \ -std=gnu99 -I../.. -g \ -ffreestanding -Wall $^ -o $@ -lgcc + +$(OUTPUT)/gcs-stress-thread: gcs-stress-thread.S + $(CC) -nostdlib $^ -o $@ diff --git a/tools/testing/selftests/arm64/gcs/asm-offsets.h b/tools/testing/selftests/arm64/gcs/asm-offsets.h new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S b/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S new file mode 100644 index 000000000000..2a08d6bf1ced --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S @@ -0,0 +1,311 @@ +// Program that loops for ever doing lots of recursions and system calls, +// intended to be used as part of a stress test for GCS context switching. +// +// Copyright 2015-2023 Arm Ltd + +#include + +#define sa_sz 32 +#define sa_flags 8 +#define sa_handler 0 +#define sa_mask_sz 8 + +#define si_code 8 + +#define SIGINT 2 +#define SIGABRT 6 +#define SIGUSR1 10 +#define SIGSEGV 11 +#define SIGUSR2 12 +#define SIGTERM 15 +#define SEGV_CPERR 10 + +#define SA_NODEFER 1073741824 +#define SA_SIGINFO 4 +#define ucontext_regs 184 + +#define PR_SET_SHADOW_STACK_STATUS 72 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) + +#define GCSPR_EL0 S3_3_C2_C5_1 + +.macro function name + .macro endfunction + .type \name, @function + .purgem endfunction + .endm +\name: +.endm + +// Print a single character x0 to stdout +// Clobbers x0-x2,x8 +function putc + str x0, [sp, #-16]! + + mov x0, #1 // STDOUT_FILENO + mov x1, sp + mov x2, #1 + mov x8, #__NR_write + svc #0 + + add sp, sp, #16 + ret +endfunction +.globl putc + +// Print a NUL-terminated string starting at address x0 to stdout +// Clobbers x0-x3,x8 +function puts + mov x1, x0 + + mov x2, #0 +0: ldrb w3, [x0], #1 + cbz w3, 1f + add x2, x2, #1 + b 0b + +1: mov w0, #1 // STDOUT_FILENO + mov x8, #__NR_write + svc #0 + + ret +endfunction +.globl puts + +// Utility macro to print a literal string +// Clobbers x0-x4,x8 +.macro puts string + .pushsection .rodata.str1.1, "aMS", @progbits, 1 +.L__puts_literal\@: .string "\string" + .popsection + + ldr x0, =.L__puts_literal\@ + bl puts +.endm + +// Print an unsigned decimal number x0 to stdout +// Clobbers x0-x4,x8 +function putdec + mov x1, sp + str x30, [sp, #-32]! // Result can't be > 20 digits + + mov x2, #0 + strb w2, [x1, #-1]! // Write the NUL terminator + + mov x2, #10 +0: udiv x3, x0, x2 // div-mod loop to generate the digits + msub x0, x3, x2, x0 + add w0, w0, #'0' + strb w0, [x1, #-1]! + mov x0, x3 + cbnz x3, 0b + + ldrb w0, [x1] + cbnz w0, 1f + mov w0, #'0' // Print "0" for 0, not "" + strb w0, [x1, #-1]! + +1: mov x0, x1 + bl puts + + ldr x30, [sp], #32 + ret +endfunction +.globl putdec + +// Print an unsigned decimal number x0 to stdout, followed by a newline +// Clobbers x0-x5,x8 +function putdecn + mov x5, x30 + + bl putdec + mov x0, #'\n' + bl putc + + ret x5 +endfunction +.globl putdecn + +// Fill x1 bytes starting at x0 with 0. +// Clobbers x1, x2. +function memclr + mov w2, #0 +endfunction +.globl memclr + // fall through to memfill + +// Trivial memory fill: fill x1 bytes starting at address x0 with byte w2 +// Clobbers x1 +function memfill + cmp x1, #0 + b.eq 1f + +0: strb w2, [x0], #1 + subs x1, x1, #1 + b.ne 0b + +1: ret +endfunction +.globl memfill + +// w0: signal number +// x1: sa_action +// w2: sa_flags +// Clobbers x0-x6,x8 +function setsignal + str x30, [sp, #-((sa_sz + 15) / 16 * 16 + 16)]! + + mov w4, w0 + mov x5, x1 + mov w6, w2 + + add x0, sp, #16 + mov x1, #sa_sz + bl memclr + + mov w0, w4 + add x1, sp, #16 + str w6, [x1, #sa_flags] + str x5, [x1, #sa_handler] + mov x2, #0 + mov x3, #sa_mask_sz + mov x8, #__NR_rt_sigaction + svc #0 + + cbz w0, 1f + + puts "sigaction failure\n" + b abort + +1: ldr x30, [sp], #((sa_sz + 15) / 16 * 16 + 16) + ret +endfunction + + +function tickle_handler + // Perhaps collect GCSPR_EL0 here in future? + ret +endfunction + +function terminate_handler + mov w21, w0 + mov x20, x2 + + puts "Terminated by signal " + mov w0, w21 + bl putdec + puts ", no error\n" + + mov x0, #0 + mov x8, #__NR_exit + svc #0 +endfunction + +function segv_handler + // stash the siginfo_t * + mov x20, x1 + + // Disable GCS, we don't want additional faults logging things + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x1, xzr + mov x2, xzr + mov x3, xzr + mov x4, xzr + mov x5, xzr + mov x8, #__NR_prctl + svc #0 + + puts "Got SIGSEGV code " + + ldr x21, [x20, #si_code] + mov x0, x21 + bl putdec + + // GCS faults should have si_code SEGV_CPERR + cmp x21, #SEGV_CPERR + bne 1f + + puts " (GCS violation)" +1: + mov x0, '\n' + bl putc + b abort +endfunction + +// Recurse x20 times +.macro recurse id +function recurse\id + stp x29, x30, [sp, #-16]! + mov x29, sp + + cmp x20, 0 + beq 1f + sub x20, x20, 1 + bl recurse\id + +1: + ldp x29, x30, [sp], #16 + + // Do a syscall immediately prior to returning to try to provoke + // scheduling and migration at a point where coherency issues + // might trigger. + mov x8, #__NR_getpid + svc #0 + + ret +endfunction +.endm + +// Generate and use two copies so we're changing the GCS contents +recurse 1 +recurse 2 + +.globl _start +function _start + // Run with GCS + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x1, PR_SHADOW_STACK_ENABLE + mov x2, xzr + mov x3, xzr + mov x4, xzr + mov x5, xzr + mov x8, #__NR_prctl + svc #0 + cbz x0, 1f + puts "Failed to enable GCS\n" + b abort +1: + + mov w0, #SIGTERM + adr x1, terminate_handler + mov w2, #SA_SIGINFO + bl setsignal + + mov w0, #SIGUSR1 + adr x1, tickle_handler + mov w2, #SA_SIGINFO + orr w2, w2, #SA_NODEFER + bl setsignal + + mov w0, #SIGSEGV + adr x1, segv_handler + mov w2, #SA_SIGINFO + orr w2, w2, #SA_NODEFER + bl setsignal + + puts "Running\n" + +loop: + // Small recursion depth so we're frequently flipping between + // the two recursors and changing what's on the stack + mov x20, #5 + bl recurse1 + mov x20, #5 + bl recurse2 + b loop +endfunction + +abort: + mov x0, #255 + mov x8, #__NR_exit + svc #0 diff --git a/tools/testing/selftests/arm64/gcs/gcs-stress.c b/tools/testing/selftests/arm64/gcs/gcs-stress.c new file mode 100644 index 000000000000..23fd8ec37bdc --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-stress.c @@ -0,0 +1,532 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2022-3 ARM Limited. + */ + +#define _GNU_SOURCE +#define _POSIX_C_SOURCE 199309L + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../kselftest.h" + +struct child_data { + char *name, *output; + pid_t pid; + int stdout; + bool output_seen; + bool exited; + int exit_status; + int exit_signal; +}; + +static int epoll_fd; +static struct child_data *children; +static struct epoll_event *evs; +static int tests; +static int num_children; +static bool terminate; + +static int startup_pipe[2]; + +static int num_processors(void) +{ + long nproc = sysconf(_SC_NPROCESSORS_CONF); + if (nproc < 0) { + perror("Unable to read number of processors\n"); + exit(EXIT_FAILURE); + } + + return nproc; +} + +static void start_thread(struct child_data *child) +{ + int ret, pipefd[2], i; + struct epoll_event ev; + + ret = pipe(pipefd); + if (ret != 0) + ksft_exit_fail_msg("Failed to create stdout pipe: %s (%d)\n", + strerror(errno), errno); + + child->pid = fork(); + if (child->pid == -1) + ksft_exit_fail_msg("fork() failed: %s (%d)\n", + strerror(errno), errno); + + if (!child->pid) { + /* + * In child, replace stdout with the pipe, errors to + * stderr from here as kselftest prints to stdout. + */ + ret = dup2(pipefd[1], 1); + if (ret == -1) { + fprintf(stderr, "dup2() %d\n", errno); + exit(EXIT_FAILURE); + } + + /* + * Duplicate the read side of the startup pipe to + * FD 3 so we can close everything else. + */ + ret = dup2(startup_pipe[0], 3); + if (ret == -1) { + fprintf(stderr, "dup2() %d\n", errno); + exit(EXIT_FAILURE); + } + + /* + * Very dumb mechanism to clean open FDs other than + * stdio. We don't want O_CLOEXEC for the pipes... + */ + for (i = 4; i < 8192; i++) + close(i); + + /* + * Read from the startup pipe, there should be no data + * and we should block until it is closed. We just + * carry on on error since this isn't super critical. + */ + ret = read(3, &i, sizeof(i)); + if (ret < 0) + fprintf(stderr, "read(startp pipe) failed: %s (%d)\n", + strerror(errno), errno); + if (ret > 0) + fprintf(stderr, "%d bytes of data on startup pipe\n", + ret); + close(3); + + ret = execl("gcs-stress-thread", "gcs-stress-thread", NULL); + fprintf(stderr, "execl(gcs-stress-thread) failed: %d (%s)\n", + errno, strerror(errno)); + + exit(EXIT_FAILURE); + } else { + /* + * In parent, remember the child and close our copy of the + * write side of stdout. + */ + close(pipefd[1]); + child->stdout = pipefd[0]; + child->output = NULL; + child->exited = false; + child->output_seen = false; + + ev.events = EPOLLIN | EPOLLHUP; + ev.data.ptr = child; + + ret = asprintf(&child->name, "Thread-%d", child->pid); + if (ret == -1) + ksft_exit_fail_msg("asprintf() failed\n"); + + ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, child->stdout, &ev); + if (ret < 0) { + ksft_exit_fail_msg("%s EPOLL_CTL_ADD failed: %s (%d)\n", + child->name, strerror(errno), errno); + } + } + + ksft_print_msg("Started %s\n", child->name); + num_children++; +} + +static bool child_output_read(struct child_data *child) +{ + char read_data[1024]; + char work[1024]; + int ret, len, cur_work, cur_read; + + ret = read(child->stdout, read_data, sizeof(read_data)); + if (ret < 0) { + if (errno == EINTR) + return true; + + ksft_print_msg("%s: read() failed: %s (%d)\n", + child->name, strerror(errno), + errno); + return false; + } + len = ret; + + child->output_seen = true; + + /* Pick up any partial read */ + if (child->output) { + strncpy(work, child->output, sizeof(work) - 1); + cur_work = strnlen(work, sizeof(work)); + free(child->output); + child->output = NULL; + } else { + cur_work = 0; + } + + cur_read = 0; + while (cur_read < len) { + work[cur_work] = read_data[cur_read++]; + + if (work[cur_work] == '\n') { + work[cur_work] = '\0'; + ksft_print_msg("%s: %s\n", child->name, work); + cur_work = 0; + } else { + cur_work++; + } + } + + if (cur_work) { + work[cur_work] = '\0'; + ret = asprintf(&child->output, "%s", work); + if (ret == -1) + ksft_exit_fail_msg("Out of memory\n"); + } + + return false; +} + +static void child_output(struct child_data *child, uint32_t events, + bool flush) +{ + bool read_more; + + if (events & EPOLLIN) { + do { + read_more = child_output_read(child); + } while (read_more); + } + + if (events & EPOLLHUP) { + close(child->stdout); + child->stdout = -1; + flush = true; + } + + if (flush && child->output) { + ksft_print_msg("%s: %s\n", child->name, child->output); + free(child->output); + child->output = NULL; + } +} + +static void child_tickle(struct child_data *child) +{ + if (child->output_seen && !child->exited) + kill(child->pid, SIGUSR1); +} + +static void child_stop(struct child_data *child) +{ + if (!child->exited) + kill(child->pid, SIGTERM); +} + +static void child_cleanup(struct child_data *child) +{ + pid_t ret; + int status; + bool fail = false; + + if (!child->exited) { + do { + ret = waitpid(child->pid, &status, 0); + if (ret == -1 && errno == EINTR) + continue; + + if (ret == -1) { + ksft_print_msg("waitpid(%d) failed: %s (%d)\n", + child->pid, strerror(errno), + errno); + fail = true; + break; + } + + if (WIFEXITED(status)) { + child->exit_status = WEXITSTATUS(status); + child->exited = true; + } + + if (WIFSIGNALED(status)) { + child->exit_signal = WTERMSIG(status); + ksft_print_msg("%s: Exited due to signal %d\n", + child->name); + fail = true; + child->exited = true; + } + } while (!child->exited); + } + + if (!child->output_seen) { + ksft_print_msg("%s no output seen\n", child->name); + fail = true; + } + + if (child->exit_status != 0) { + ksft_print_msg("%s exited with error code %d\n", + child->name, child->exit_status); + fail = true; + } + + ksft_test_result(!fail, "%s\n", child->name); +} + +static void handle_child_signal(int sig, siginfo_t *info, void *context) +{ + int i; + bool found = false; + + for (i = 0; i < num_children; i++) { + if (children[i].pid == info->si_pid) { + children[i].exited = true; + children[i].exit_status = info->si_status; + found = true; + break; + } + } + + if (!found) + ksft_print_msg("SIGCHLD for unknown PID %d with status %d\n", + info->si_pid, info->si_status); +} + +static void handle_exit_signal(int sig, siginfo_t *info, void *context) +{ + int i; + + /* If we're already exiting then don't signal again */ + if (terminate) + return; + + ksft_print_msg("Got signal, exiting...\n"); + + terminate = true; + + /* + * This should be redundant, the main loop should clean up + * after us, but for safety stop everything we can here. + */ + for (i = 0; i < num_children; i++) + child_stop(&children[i]); +} + +/* Handle any pending output without blocking */ +static void drain_output(bool flush) +{ + int ret = 1; + int i; + + while (ret > 0) { + ret = epoll_wait(epoll_fd, evs, tests, 0); + if (ret < 0) { + if (errno == EINTR) + continue; + ksft_print_msg("epoll_wait() failed: %s (%d)\n", + strerror(errno), errno); + } + + for (i = 0; i < ret; i++) + child_output(evs[i].data.ptr, evs[i].events, flush); + } +} + +static const struct option options[] = { + { "timeout", required_argument, NULL, 't' }, + { } +}; + +int main(int argc, char **argv) +{ + int seen_children; + bool all_children_started = false; + int gcs_threads; + int timeout = 10; + int ret, cpus, i, c; + struct sigaction sa; + + while ((c = getopt_long(argc, argv, "t:", options, NULL)) != -1) { + switch (c) { + case 't': + ret = sscanf(optarg, "%d", &timeout); + if (ret != 1) + ksft_exit_fail_msg("Failed to parse timeout %s\n", + optarg); + break; + default: + ksft_exit_fail_msg("Unknown argument\n"); + } + } + + cpus = num_processors(); + tests = 0; + + if (getauxval(AT_HWCAP2) & HWCAP2_GCS) { + /* One extra thread, trying to trigger migrations */ + gcs_threads = cpus + 1; + tests += gcs_threads; + } else { + gcs_threads = 0; + } + + ksft_print_header(); + ksft_set_plan(tests); + + ksft_print_msg("%d CPUs, %d GCS threads\n", + cpus, gcs_threads); + + if (!tests) + ksft_exit_skip("No tests scheduled\n"); + + if (timeout > 0) + ksft_print_msg("Will run for %ds\n", timeout); + else + ksft_print_msg("Will run until terminated\n"); + + children = calloc(sizeof(*children), tests); + if (!children) + ksft_exit_fail_msg("Unable to allocate child data\n"); + + ret = epoll_create1(EPOLL_CLOEXEC); + if (ret < 0) + ksft_exit_fail_msg("epoll_create1() failed: %s (%d)\n", + strerror(errno), ret); + epoll_fd = ret; + + /* Create a pipe which children will block on before execing */ + ret = pipe(startup_pipe); + if (ret != 0) + ksft_exit_fail_msg("Failed to create startup pipe: %s (%d)\n", + strerror(errno), errno); + + /* Get signal handers ready before we start any children */ + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = handle_exit_signal; + sa.sa_flags = SA_RESTART | SA_SIGINFO; + sigemptyset(&sa.sa_mask); + ret = sigaction(SIGINT, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGINT handler: %s (%d)\n", + strerror(errno), errno); + ret = sigaction(SIGTERM, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGTERM handler: %s (%d)\n", + strerror(errno), errno); + sa.sa_sigaction = handle_child_signal; + ret = sigaction(SIGCHLD, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGCHLD handler: %s (%d)\n", + strerror(errno), errno); + + evs = calloc(tests, sizeof(*evs)); + if (!evs) + ksft_exit_fail_msg("Failed to allocated %d epoll events\n", + tests); + + for (i = 0; i < gcs_threads; i++) + start_thread(&children[i]); + + /* + * All children started, close the startup pipe and let them + * run. + */ + close(startup_pipe[0]); + close(startup_pipe[1]); + + timeout *= 10; + for (;;) { + /* Did we get a signal asking us to exit? */ + if (terminate) + break; + + /* + * Timeout is counted in 100ms with no output, the + * tests print during startup then are silent when + * running so this should ensure they all ran enough + * to install the signal handler, this is especially + * useful in emulation where we will both be slow and + * likely to have a large set of VLs. + */ + ret = epoll_wait(epoll_fd, evs, tests, 100); + if (ret < 0) { + if (errno == EINTR) + continue; + ksft_exit_fail_msg("epoll_wait() failed: %s (%d)\n", + strerror(errno), errno); + } + + /* Output? */ + if (ret > 0) { + for (i = 0; i < ret; i++) { + child_output(evs[i].data.ptr, evs[i].events, + false); + } + continue; + } + + /* Otherwise epoll_wait() timed out */ + + /* + * If the child processes have not produced output they + * aren't actually running the tests yet. + */ + if (!all_children_started) { + seen_children = 0; + + for (i = 0; i < num_children; i++) + if (children[i].output_seen || + children[i].exited) + seen_children++; + + if (seen_children != num_children) { + ksft_print_msg("Waiting for %d children\n", + num_children - seen_children); + continue; + } + + all_children_started = true; + } + + ksft_print_msg("Sending signals, timeout remaining: %d00ms\n", + timeout); + + for (i = 0; i < num_children; i++) + child_tickle(&children[i]); + + /* Negative timeout means run indefinitely */ + if (timeout < 0) + continue; + if (--timeout == 0) + break; + } + + ksft_print_msg("Finishing up...\n"); + terminate = true; + + for (i = 0; i < tests; i++) + child_stop(&children[i]); + + drain_output(false); + + for (i = 0; i < tests; i++) + child_cleanup(&children[i]); + + drain_output(true); + + ksft_print_cnts(); + + return 0; +} From patchwork Sat Feb 3 12:26:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 769678 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9AEC7F7C4; Sat, 3 Feb 2024 12:34:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963642; cv=none; b=KCDltrnfJjDSuKJKxUME2mqNEaFYvtvQ5YDoluWBOUmF4AWeHtpZg7GgJXN17nTgQSDBEhbeEWPyTxkVaftCpcm34CkBSrj6e27Fj2jpsGqfxQ/VQ3pm9cw3VJecrsEFuAzp7Dl9C/11aI2p5fgdveDSiTX/z6KSzMJLUqn5pHk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706963642; c=relaxed/simple; bh=WXqbYsgYYOXFEeMs+roJ3AXRYBhuL60ozPF+o7O4bAk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=J0x4Q2/WSGuE4Gggb/RW1lKOpYVeFpOhQk2Pnpk/rt7+MXSnnUuRNbd5TtnexsVURHUPatuj4INhCIVWkXDTFk++qA/j55HOY4505t/GNtqwKMCbSfoIa46L/DZYc0cwsZQbFX4jkO7jTHNwo6UkcqN/02EJYmIlUuNIQaeoGIQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=F78PhGGK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="F78PhGGK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35D85C43394; Sat, 3 Feb 2024 12:33:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963641; bh=WXqbYsgYYOXFEeMs+roJ3AXRYBhuL60ozPF+o7O4bAk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=F78PhGGK2tKTlc1p2op6t75h6sAseztGvobKzgGSABUqePNhCLv6h7b0MXYZozVvf TMoM8VKK0ZO5F2e6/66jeGgtskS0UMHEiCAZUXqg+6vxYh1xgH6tpBdIJiNPgo7YpK EnaGYjMB/A//4+Sjcv0gbyN2txDslqcYGoLlVah+34dTkNIBFZPTZDpK5wR3pQL70I Dzxx0jU1Bjxx/D0JoI4d/HNBk5LzI9/lGYtAoE48s/fM91juriLFdFQwpik4auJ+g4 1D/HN4P1oVK7bX97yoTav7bFx3jIr53qvhASYz8zdLRC/yDEo0s7d2QHFhc9ilv97S gLO6O0zcUVi6Q== From: Mark Brown Date: Sat, 03 Feb 2024 12:26:04 +0000 Subject: [PATCH v8 38/38] kselftest: Provide shadow stack enable helpers for arm64 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-38-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=2372; i=broonie@kernel.org; h=from:subject:message-id; bh=WXqbYsgYYOXFEeMs+roJ3AXRYBhuL60ozPF+o7O4bAk=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDvM7ObtGoCDxxiAl89DYa6RzxdLnWQy+FkForz aUSU5kiJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w7wAKCRAk1otyXVSH0NEICA CCvybEtXLigY0Lceu5RSwEZ8U+9W6svq5+knPV3mZQY4TSxy9Yy4lyeVk8Pv21v/12qFvp6uqzLp0g k6+WV9GRZ/Pqfacx9BQ00Oc/rfhKVczK0MsQz3mDGVuLKbJ4qSNpb8/dqtFqZr2ObsOcGe2ZV7T/CG ALJt/tNjptHMxvEWIyQI3z/Smo1ZEdn4C7yaVXufiGkwDmFEB8wquT7O7Dh3npGDpxL78q4rAtfJy+ 8dPSdobFohDZMcnrMhjgw8Klf2bbVp8Sut5M7DVJ/t3nRkqxRoYhyVFTjjG6NdjDWhhL+IKQdltpt4 8U4+ItNFhfAZrHX27/JjU9V30WRy/h X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Allow test programs to use the shadow stack helpers on arm64. Signed-off-by: Mark Brown --- tools/testing/selftests/ksft_shstk.h | 37 ++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/tools/testing/selftests/ksft_shstk.h b/tools/testing/selftests/ksft_shstk.h index 85d0747c1802..223e24b4eb80 100644 --- a/tools/testing/selftests/ksft_shstk.h +++ b/tools/testing/selftests/ksft_shstk.h @@ -50,6 +50,43 @@ static inline __attribute__((always_inline)) void enable_shadow_stack(void) #endif +#ifdef __aarch64__ +#define PR_SET_SHADOW_STACK_STATUS 72 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) + +#define my_syscall2(num, arg1, arg2) \ +({ \ + register long _num __asm__ ("x8") = (num); \ + register long _arg1 __asm__ ("x0") = (long)(arg1); \ + register long _arg2 __asm__ ("x1") = (long)(arg2); \ + register long _arg3 __asm__ ("x2") = 0; \ + register long _arg4 __asm__ ("x3") = 0; \ + register long _arg5 __asm__ ("x4") = 0; \ + \ + __asm__ volatile ( \ + "svc #0\n" \ + : "=r"(_arg1) \ + : "r"(_arg1), "r"(_arg2), \ + "r"(_arg3), "r"(_arg4), \ + "r"(_arg5), "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +#define ENABLE_SHADOW_STACK +static inline __attribute__((always_inline)) void enable_shadow_stack(void) +{ + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + PR_SHADOW_STACK_ENABLE); + if (ret == 0) + shadow_stack_enabled = true; +} + +#endif + #ifndef __NR_map_shadow_stack #define __NR_map_shadow_stack 453 #endif