From patchwork Tue Oct 1 22:58:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832112 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41AC81CB513; Tue, 1 Oct 2024 23:00:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823620; cv=none; b=Uo1B3VHwPrh9raKeBkS6Ddtya8Y1gTpP/nnJzjBXkxOw8JFmgXZbGehQ8UuPujT168uOHeYpRlUjVkHlI8W0vaPOuuyQzcFl0s3oqE1+a+lHhJRM16q4nFkAXpm5gFOKDyqE138YG0KgNGNryFiVrx/ag0qNdlAcN1lafTZd6Q4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823620; c=relaxed/simple; bh=E+STpDluPni1qqn1pN34uc0md3Dbr7p50mwxPtfPVv4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=F9ETBH2cjjxg1ytk4CJCqvkJpiE1xb8UPmZzc8sjosAofgOgPHa1RllmwfHVr+hGKoXeTwugpkyzYQcQQl2Hse8j5lvnBnXBl3YiRlDAJcG9zL6btcdbN/VOHZEBEYeLCoT5Qhee/KcMCNCtVdY1DW65QC9V88SI/5dDfqGBvew= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=l96NGn7T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="l96NGn7T" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 00FADC4CECD; Tue, 1 Oct 2024 23:00:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823620; bh=E+STpDluPni1qqn1pN34uc0md3Dbr7p50mwxPtfPVv4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=l96NGn7TPurtYSYyEpu0VnZYlAC2GuD2+RbdqPXib+mqQAeBSqqkmnezOKTnlqVjr HDbSwGS+mgTjMLa07JGqmUfBnR0ypexjQGKJaNBgwajGZwKwRsxdv0AKZ0xRiwe56F Ao8ZG+9U5OS5Ve6nVTGHYWROxrqGeMoeMc71uZS0QrY0VZUEYy5W2+NhFjwXiCtCim +Nct63DmkECUz2AZK7Oiyx9s45YrUR/vEgDrJPLoW4fLqNroyzIth5HIga6UfMOvMe sAugk1xN/dOaAabMo8cz3b3m2WGmTmp5NFmB0lKgERg/V11rLMz3zxPeL1oHAHXyVs XDfHSOswPd4sQ== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:41 +0100 Subject: [PATCH v13 02/40] mm: Define VM_HIGH_ARCH_6 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-2-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=1285; i=broonie@kernel.org; h=from:subject:message-id; bh=E+STpDluPni1qqn1pN34uc0md3Dbr7p50mwxPtfPVv4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7FqkbqHSry4P1gTmihgOrWZBFx7sDukTznRbrU 2uCRGoCJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+xQAKCRAk1otyXVSH0F4rB/ 9amfpCTqeJr4oMpmSRZ4U6Ditl6XVFokj/OXg+LAKqsq6EhoA3N+8gdc/aS5cnW13+1ac1jZeWXWrg hp3oVPQPuVmteub5aFl3SQzh/BKX8YHEq56dKgJA1udA1tiAkKjASjDqBpIkUH0AFjU0kv0u4Xh+ND dJfTfnKT0IzpOCcfSpCSGhidKn7KUnuBFjPPmw40/aRUZF069RKTCk0USiZqlY1TSUbe33MhAVt3hr nZ3RYPMnOpZs00/ydAeSFTD1hwI9QE9t6Ah0uenxM/o06Cd1v9n2mS9NMPMdLJpFbndB0W4fQmIG0E 3CMqg/lvoOg4PUH/jEwsL0eXZfDCF9 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The addition of protection keys means that on arm64 we now use all of the currently defined VM_HIGH_ARCH_x bits. In order to allow us to allocate a new flag for GCS pages define VM_HIGH_ARCH_6. Signed-off-by: Mark Brown --- include/linux/mm.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index ecf63d2b0582..182bad0c55df 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -329,12 +329,14 @@ extern unsigned int kobjsize(const void *objp); #define VM_HIGH_ARCH_BIT_3 35 /* bit only usable on 64-bit architectures */ #define VM_HIGH_ARCH_BIT_4 36 /* bit only usable on 64-bit architectures */ #define VM_HIGH_ARCH_BIT_5 37 /* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_6 38 /* bit only usable on 64-bit architectures */ #define VM_HIGH_ARCH_0 BIT(VM_HIGH_ARCH_BIT_0) #define VM_HIGH_ARCH_1 BIT(VM_HIGH_ARCH_BIT_1) #define VM_HIGH_ARCH_2 BIT(VM_HIGH_ARCH_BIT_2) #define VM_HIGH_ARCH_3 BIT(VM_HIGH_ARCH_BIT_3) #define VM_HIGH_ARCH_4 BIT(VM_HIGH_ARCH_BIT_4) #define VM_HIGH_ARCH_5 BIT(VM_HIGH_ARCH_BIT_5) +#define VM_HIGH_ARCH_6 BIT(VM_HIGH_ARCH_BIT_6) #endif /* CONFIG_ARCH_USES_HIGH_VMA_FLAGS */ #ifdef CONFIG_ARCH_HAS_PKEYS From patchwork Tue Oct 1 22:58:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832111 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 629111CF29B; Tue, 1 Oct 2024 23:00:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823642; cv=none; b=fRisgdjlMc9uFiFCAUyd3UD7vUU6uocU4TIc40Pp//cGTsDpBfUYxn7kIUEu5aq09Ws1zU2RhI3Uzo+uXVeuSsWoHryWtRo1Q8g/6WKR9VsF1g2uimcyohRJuj4I9zXBaWHUb9yQAN2K1UZ9PfzBfy+to3uOZ/GJXW3IwgMwITI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823642; c=relaxed/simple; bh=QzHtuVcSDaL4Yx9qwbzgbMXiTjYft30F6+HnraYB7I8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SuLqqPBVc23wt0wy66dfpvNBAhzk30EOwvJ18HDj2wHVY6a9y3k3XZcb56aoyicP2+LmHbgmI4MlsfzkbOHcaQwYFtek0jYjgKHaPaotkgY7uBOdXjk1UuzJNDZ4UobdEVDMcfSr+85W5KEzRArBAaFnzFY4nz/TKWqFVW0Trdo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uZYeamrV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uZYeamrV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E63EC4CECF; Tue, 1 Oct 2024 23:00:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823641; bh=QzHtuVcSDaL4Yx9qwbzgbMXiTjYft30F6+HnraYB7I8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=uZYeamrVp2rmyMCQnluYwOnDm4n+Ff//3VOK9nh7fhijWFZSBsIrKKDjAGk/o56ph TWtU7syb9yzkhscU782HgCKBmfTe8umJChkH7pbpdmrFkFnZKDzEigjtf2MH4Q6h+p 03C4A6BM0Yce+YT95YTjNz7OxrDfGQse/whSczqePqSDu3R0HqcStk54Lkhz+mQEEG /g6Czt7bYJqEty6YzhPgcA7W0BO47QcNZGVJ0ID0LvsSGOrdMO4RtMtauP6xRfcUf3 TqTDwfiQYc3f0nkcAQpIFVMpBhb6kHSmTpoI4+YPQ4WcHzjP2Dcp9wdpT1PpFQ1JyR PZHhD65WC/Q5Q== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:43 +0100 Subject: [PATCH v13 04/40] prctl: arch-agnostic prctl for shadow stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-4-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=4975; i=broonie@kernel.org; h=from:subject:message-id; bh=QzHtuVcSDaL4Yx9qwbzgbMXiTjYft30F6+HnraYB7I8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7HehyHHZF6+cj8DJo6NxY+7EGNDDBamzVyHc5d IEYLGoqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+xwAKCRAk1otyXVSH0JhiB/ wO8/ZF+SkRw9d5n/x/2zuP6kLyixcTZ3zAcm+iDEAW8JePX4d66upSGjkAXNAIOYliNpblNkRnl9lW /pqQ7K6QEeMennahYz0YD8lnH71QpGSZoYU4Q3TSyJu1ZMKTYlrBAd4xIFAjtTa1dsVtHWMbgc7EYT GrpwH5cRB82KnH5mEWaCdoBopZsgFWmMql5OS4Dhig4w/zyRuEBRoxmlgvQxdF3TWFrXD9i1lNVpHx CCKN9fjW0cokoWtvsnRvMxrlBAUb+FeHgux5A6Y+EjNCDrNy3dUev6djW59zf6sdiH/eXUqnNhum7i C/teDyY09NlgqHrpdzubJx7yX0COoy X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Three architectures (x86, aarch64, riscv) have announced support for shadow stacks with fairly similar functionality. While x86 is using arch_prctl() to control the functionality neither arm64 nor riscv uses that interface so this patch adds arch-agnostic prctl() support to get and set status of shadow stacks and lock the current configuation to prevent further changes, with support for turning on and off individual subfeatures so applications can limit their exposure to features that they do not need. The features are: - PR_SHADOW_STACK_ENABLE: Tracking and enforcement of shadow stacks, including allocation of a shadow stack if one is not already allocated. - PR_SHADOW_STACK_WRITE: Writes to specific addresses in the shadow stack. - PR_SHADOW_STACK_PUSH: Push additional values onto the shadow stack. These features are expected to be inherited by new threads and cleared on exec(), unknown features should be rejected for enable but accepted for locking (in order to allow for future proofing). This is based on a patch originally written by Deepak Gupta but modified fairly heavily, support for indirect landing pads is removed, additional modes added and the locking interface reworked. The set status prctl() is also reworked to just set flags, if setting/reading the shadow stack pointer is required this could be a separate prctl. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Acked-by: Yury Khrustalev Signed-off-by: Mark Brown --- include/linux/mm.h | 4 ++++ include/uapi/linux/prctl.h | 22 ++++++++++++++++++++++ kernel/sys.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 182bad0c55df..56654306a832 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4221,4 +4221,8 @@ static inline void pgalloc_tag_copy(struct folio *new, struct folio *old) } #endif /* CONFIG_MEM_ALLOC_PROFILING */ +int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status); +int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); +int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); + #endif /* _LINUX_MM_H */ diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 35791791a879..557a3d2ac1d4 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -328,4 +328,26 @@ struct prctl_mm_map { # define PR_PPC_DEXCR_CTRL_CLEAR_ONEXEC 0x10 /* Clear the aspect on exec */ # define PR_PPC_DEXCR_CTRL_MASK 0x1f +/* + * Get the current shadow stack configuration for the current thread, + * this will be the value configured via PR_SET_SHADOW_STACK_STATUS. + */ +#define PR_GET_SHADOW_STACK_STATUS 74 + +/* + * Set the current shadow stack configuration. Enabling the shadow + * stack will cause a shadow stack to be allocated for the thread. + */ +#define PR_SET_SHADOW_STACK_STATUS 75 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) +# define PR_SHADOW_STACK_WRITE (1UL << 1) +# define PR_SHADOW_STACK_PUSH (1UL << 2) + +/* + * Prevent further changes to the specified shadow stack + * configuration. All bits may be locked via this call, including + * undefined bits. + */ +#define PR_LOCK_SHADOW_STACK_STATUS 76 + #endif /* _LINUX_PRCTL_H */ diff --git a/kernel/sys.c b/kernel/sys.c index 4da31f28fda8..3d38a9c7c5c9 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -2324,6 +2324,21 @@ int __weak arch_prctl_spec_ctrl_set(struct task_struct *t, unsigned long which, return -EINVAL; } +int __weak arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status) +{ + return -EINVAL; +} + +int __weak arch_set_shadow_stack_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + +int __weak arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status) +{ + return -EINVAL; +} + #define PR_IO_FLUSHER (PF_MEMALLOC_NOIO | PF_LOCAL_THROTTLE) #ifdef CONFIG_ANON_VMA_NAME @@ -2784,6 +2799,21 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, case PR_RISCV_SET_ICACHE_FLUSH_CTX: error = RISCV_SET_ICACHE_FLUSH_CTX(arg2, arg3); break; + case PR_GET_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_get_shadow_stack_status(me, (unsigned long __user *) arg2); + break; + case PR_SET_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_set_shadow_stack_status(me, arg2); + break; + case PR_LOCK_SHADOW_STACK_STATUS: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = arch_lock_shadow_stack_status(me, arg2); + break; default: error = -EINVAL; break; From patchwork Tue Oct 1 22:58:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832110 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BFEC19AD89; Tue, 1 Oct 2024 23:00:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823654; cv=none; b=MmWcn+PP37zjRkqSlQQ2flkncLWMO4ioFOeOCqDVH/LXUiETHZZbg2b38mJ09CzkFH+yuQiAl3wsldgL4JRYceygBojdNYz/KE1n58gCL4fvIU17QGmpmsjIrNM97E1nSqYR4ZUsiMinwqUjOBAuZ+YSy1D+zdKI7DIaIWafYy0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823654; c=relaxed/simple; bh=vL+b/h3WOPYO7SwFl92JA9c0VbV77YwkdcIPJjlNFTs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ANbPldvrG5bbpLFI9vwZZ1hQZiKIyiC79x38vo7LORZyXFeQDq4CC+x6pfIaR6DDQRRdQFGS2ew+wb5C6zyht2b43krNARmR7NPfwz7P8jVuTE24GtTEShfY3PWdbaCwiPmCM0Y/Z6/Cl3ZtepadDs4v5u6cU4dIWpFDVYrPV4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q9BEblIa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q9BEblIa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 73BCEC4CECD; Tue, 1 Oct 2024 23:00:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823653; bh=vL+b/h3WOPYO7SwFl92JA9c0VbV77YwkdcIPJjlNFTs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Q9BEblIa6MmA5Xaow7wFPzfAM9zttSOUnO+EtJzN/PX7Huf9lzr++3lNcVEnhbBqz 6dMkhlxPMElbdJeNlV158SBKPlOnHk5LQmhwlqFXihJgYSoqGxl3GLqUDM7DzAG5zc lvLDejujVe26CNwqEFjY9RC9WxGXrSNkF5G9RyuOCrURHXGYY7+8AA1BJ1SMrCIjZt BJvDkqRMfZF1cudVdwGq4CDhB5w9OLNa4NOrghTmOJJzwygsOzNNOFDRDlXzt7kKtH IpBNBwRkNrS1h0eaDjRqb0/mVRIcjwZiyqBZn+jatUlWQ0I77tCcOgDFflvH0cvLXI 3XrtVqCX11mdg== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:44 +0100 Subject: [PATCH v13 05/40] mman: Add map_shadow_stack() flags Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-5-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=1935; i=broonie@kernel.org; h=from:subject:message-id; bh=vL+b/h3WOPYO7SwFl92JA9c0VbV77YwkdcIPJjlNFTs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7Ig3nICM+W9H3bCT71LrQPx0TmwN2XrzWIKusU qfRtVgmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+yAAKCRAk1otyXVSH0EIMB/ 9VAnLOcjD+99k7ouK69WkUZZje02S2hR6fRnCSftnhm+4bz+XAyRnRMAjPkiv0ecgHaTuBtP+MmN6L M1BTqr15gkX62ORCMeKqgMig0l5Zm08WyrOD4f/Vf4Zb4phDfeQkEhHthQYNAOzPiw9zjQIJGaypMM PJHFuO2ykfazrzqINBCj0D1SCeQt3+5cESSO4XqcDnyaOrpnKPqwWdoWVkNzB98GHx999EQw2iWsOp pEi9Zg8ZJFAhJSTayVv1NDY9QzD6ngW3nJCCx7rD/3cbiZd2/Whg25ERj0YbAqlldQVHJfqrX3RCBL guH+PVB5PQbczVYykW+F+llt/dznd7 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for adding arm64 GCS support make the map_shadow_stack() SHADOW_STACK_SET_TOKEN flag generic and add _SET_MARKER. The existing flag indicates that a token usable for stack switch should be added to the top of the newly mapped GCS region while the new flag indicates that a top of stack marker suitable for use by unwinders should be added above that. For arm64 the top of stack marker is all bits 0. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Acked-by: Yury Khrustalev Signed-off-by: Mark Brown --- arch/x86/include/uapi/asm/mman.h | 3 --- include/uapi/asm-generic/mman.h | 4 ++++ 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/uapi/asm/mman.h b/arch/x86/include/uapi/asm/mman.h index 46cdc941f958..ac1e6277212b 100644 --- a/arch/x86/include/uapi/asm/mman.h +++ b/arch/x86/include/uapi/asm/mman.h @@ -5,9 +5,6 @@ #define MAP_32BIT 0x40 /* only give out 32bit addresses */ #define MAP_ABOVE4G 0x80 /* only map above 4GB */ -/* Flags for map_shadow_stack(2) */ -#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ - #include #endif /* _ASM_X86_MMAN_H */ diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h index 57e8195d0b53..5e3d61ddbd8c 100644 --- a/include/uapi/asm-generic/mman.h +++ b/include/uapi/asm-generic/mman.h @@ -19,4 +19,8 @@ #define MCL_FUTURE 2 /* lock all future mappings */ #define MCL_ONFAULT 4 /* lock all pages that are faulted in */ +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ +#define SHADOW_STACK_SET_MARKER (1ULL << 1) /* Set up a top of stack marker in the shadow stack */ + + #endif /* __ASM_GENERIC_MMAN_H */ From patchwork Tue Oct 1 22:58:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832109 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8B6D1CEAD9; Tue, 1 Oct 2024 23:01:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823679; cv=none; b=CnwOQzfoUucTReqMn8AbQ6SF+lOAkhQzSXXGH+BKgn8ETTC+huQmGhwOU0gdh84cJLjdurSogbt6Txr/66ZudttLdOc/B/nP54EOfTvsFam8E//n+eTIPykQeK+dSFDoaYCMkedcZ/yJckGBLU6/NDLWr8+h5V/NuVsYYBv2N/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823679; c=relaxed/simple; bh=Vf2uNluHz2rca3mw8TcTX+NsJV/Qh5T4cYUs2i7TXxM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=U7IfYFqZTMzRU/Aj2U76rP1eyovvHBrHGICDFF5PxoqbRM7XP7o2BdSMjTzMn08/xY+hqPoYOiLcMOK9m1NAtc+rTwAkfw2EhEQUlbz0dfdt6tLXA0ghLFc/evI8zKsCq3y/x3q3niNM2GY0y1nakxOubbm0MeHJF9/XHQ+mNmk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jVZM9w2T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jVZM9w2T" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1CB2CC4CECF; Tue, 1 Oct 2024 23:01:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823679; bh=Vf2uNluHz2rca3mw8TcTX+NsJV/Qh5T4cYUs2i7TXxM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=jVZM9w2TFPYRXcHqVrP0Qzet5ZimUajVBnGH0EhpBkabdMfLXa+RzCA+jUnjyGM1m CJ8EFXdP1PgHrC1tKvVGnJWlxMjAFQMP2gFRgwEqZO8N6qSYimU8ejOpwwGHnFZuXY 86DXvpsvB3FWiEMk+ntP0Lha7xlDlV5qpMAZOggrQuJm3djAJfUnCgt713hfxPM+Ev 0BXqsx8xr/0zc257E+hP6isHgW4XTx+znlwISB5hvczbAi1lqyvDXPzJ5/yi8ANzdt jNLailjxmqcTsawgpffDetmjcAUOgQOmif3/O5yhtJeanlELsfO43OyWb9ZUezbd+v tN8vKA+l6a/Pg== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:46 +0100 Subject: [PATCH v13 07/40] arm64/gcs: Document the ABI for Guarded Control Stacks Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-7-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=10660; i=broonie@kernel.org; h=from:subject:message-id; bh=Vf2uNluHz2rca3mw8TcTX+NsJV/Qh5T4cYUs2i7TXxM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7Ju1pToDWaUGEp3Hkm5trwDDFmJAt21ggli5wt 1wpFq4aJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+yQAKCRAk1otyXVSH0HYwB/ 91P3Xkg8BBGXxh3Fgc894UqDHkhXWVd3t+Pksyai5obwhEVa85s8Wg9ISM+HTQkh460hHADA/xDbFF 2YjD56YVZP5Dg+ORNWhRRVdTKX/eo1sLcWdJvt9xONO7FI9kLgypXqSp630Xos/2fq/AHJy62H/9l8 q9RceRohuQOKkPBoI66VYnysha3TgtYDEMxt4978KiJe/JB0aYFC1ixMNEa8hdorjk8s7JUw0RK8wd MbGVvFdRIm33dFehcmpQPYMvJSvBGHAq3ED4DFY8dttj9OB30BGbU64SftZehNJudxSMnvy8NHs8NB dI+RpTjveoc1pWc7Ojpl1gXEDkyEIa X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add some documentation of the userspace ABI for Guarded Control Stacks. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Acked-by: Yury Khrustalev Signed-off-by: Mark Brown --- Documentation/arch/arm64/gcs.rst | 230 +++++++++++++++++++++++++++++++++++++ Documentation/arch/arm64/index.rst | 1 + 2 files changed, 231 insertions(+) diff --git a/Documentation/arch/arm64/gcs.rst b/Documentation/arch/arm64/gcs.rst new file mode 100644 index 000000000000..af58d9151cb7 --- /dev/null +++ b/Documentation/arch/arm64/gcs.rst @@ -0,0 +1,230 @@ +=============================================== +Guarded Control Stack support for AArch64 Linux +=============================================== + +This document outlines briefly the interface provided to userspace by Linux in +order to support use of the ARM Guarded Control Stack (GCS) feature. + +This is an outline of the most important features and issues only and not +intended to be exhaustive. + + + +1. General +----------- + +* GCS is an architecture feature intended to provide greater protection + against return oriented programming (ROP) attacks and to simplify the + implementation of features that need to collect stack traces such as + profiling. + +* When GCS is enabled a separate guarded control stack is maintained by the + PE which is writeable only through specific GCS operations. This + stores the call stack only, when a procedure call instruction is + performed the current PC is pushed onto the GCS and on RET the + address in the LR is verified against that on the top of the GCS. + +* When active the current GCS pointer is stored in the system register + GCSPR_EL0. This is readable by userspace but can only be updated + via specific GCS instructions. + +* The architecture provides instructions for switching between guarded + control stacks with checks to ensure that the new stack is a valid + target for switching. + +* The functionality of GCS is similar to that provided by the x86 Shadow + Stack feature, due to sharing of userspace interfaces the ABI refers to + shadow stacks rather than GCS. + +* Support for GCS is reported to userspace via HWCAP_GCS in the aux vector + AT_HWCAP2 entry. + +* GCS is enabled per thread. While there is support for disabling GCS + at runtime this should be done with great care. + +* GCS memory access faults are reported as normal memory access faults. + +* GCS specific errors (those reported with EC 0x2d) will be reported as + SIGSEGV with a si_code of SEGV_CPERR (control protection error). + +* GCS is supported only for AArch64. + +* On systems where GCS is supported GCSPR_EL0 is always readable by EL0 + regardless of the GCS configuration for the thread. + +* The architecture supports enabling GCS without verifying that return values + in LR match those in the GCS, the LR will be ignored. This is not supported + by Linux. + + + +2. Enabling and disabling Guarded Control Stacks +------------------------------------------------- + +* GCS is enabled and disabled for a thread via the PR_SET_SHADOW_STACK_STATUS + prctl(), this takes a single flags argument specifying which GCS features + should be used. + +* When set PR_SHADOW_STACK_ENABLE flag allocates a Guarded Control Stack + and enables GCS for the thread, enabling the functionality controlled by + GCSCRE0_EL1.{nTR, RVCHKEN, PCRSEL}. + +* When set the PR_SHADOW_STACK_PUSH flag enables the functionality controlled + by GCSCRE0_EL1.PUSHMEn, allowing explicit GCS pushes. + +* When set the PR_SHADOW_STACK_WRITE flag enables the functionality controlled + by GCSCRE0_EL1.STREn, allowing explicit stores to the Guarded Control Stack. + +* Any unknown flags will cause PR_SET_SHADOW_STACK_STATUS to return -EINVAL. + +* PR_LOCK_SHADOW_STACK_STATUS is passed a bitmask of features with the same + values as used for PR_SET_SHADOW_STACK_STATUS. Any future changes to the + status of the specified GCS mode bits will be rejected. + +* PR_LOCK_SHADOW_STACK_STATUS allows any bit to be locked, this allows + userspace to prevent changes to any future features. + +* There is no support for a process to remove a lock that has been set for + it. + +* PR_SET_SHADOW_STACK_STATUS and PR_LOCK_SHADOW_STACK_STATUS affect only the + thread that called them, any other running threads will be unaffected. + +* New threads inherit the GCS configuration of the thread that created them. + +* GCS is disabled on exec(). + +* The current GCS configuration for a thread may be read with the + PR_GET_SHADOW_STACK_STATUS prctl(), this returns the same flags that + are passed to PR_SET_SHADOW_STACK_STATUS. + +* If GCS is disabled for a thread after having previously been enabled then + the stack will remain allocated for the lifetime of the thread. At present + any attempt to reenable GCS for the thread will be rejected, this may be + revisited in future. + +* It should be noted that since enabling GCS will result in GCS becoming + active immediately it is not normally possible to return from the function + that invoked the prctl() that enabled GCS. It is expected that the normal + usage will be that GCS is enabled very early in execution of a program. + + + +3. Allocation of Guarded Control Stacks +---------------------------------------- + +* When GCS is enabled for a thread a new Guarded Control Stack will be + allocated for it of half the standard stack size or 2 gigabytes, + whichever is smaller. + +* When a new thread is created by a thread which has GCS enabled then a + new Guarded Control Stack will be allocated for the new thread with + half the size of the standard stack. + +* When a stack is allocated by enabling GCS or during thread creation then + the top 8 bytes of the stack will be initialised to 0 and GCSPR_EL0 will + be set to point to the address of this 0 value, this can be used to + detect the top of the stack. + +* Additional Guarded Control Stacks can be allocated using the + map_shadow_stack() system call. + +* Stacks allocated using map_shadow_stack() can optionally have an end of + stack marker and cap placed at the top of the stack. If the flag + SHADOW_STACK_SET_TOKEN is specified a cap will be placed on the stack, + if SHADOW_STACK_SET_MARKER is not specified the cap will be the top 8 + bytes of the stack and if it is specified then the cap will be the next + 8 bytes. While specifying just SHADOW_STACK_SET_MARKER by itself is + valid since the marker is all bits 0 it has no observable effect. + +* Stacks allocated using map_shadow_stack() must have a size which is a + multiple of 8 bytes larger than 8 bytes and must be 8 bytes aligned. + +* An address can be specified to map_shadow_stack(), if one is provided then + it must be aligned to a page boundary. + +* When a thread is freed the Guarded Control Stack initially allocated for + that thread will be freed. Note carefully that if the stack has been + switched this may not be the stack currently in use by the thread. + + +4. Signal handling +-------------------- + +* A new signal frame record gcs_context encodes the current GCS mode and + pointer for the interrupted context on signal delivery. This will always + be present on systems that support GCS. + +* The record contains a flag field which reports the current GCS configuration + for the interrupted context as PR_GET_SHADOW_STACK_STATUS would. + +* The signal handler is run with the same GCS configuration as the interrupted + context. + +* When GCS is enabled for the interrupted thread a signal handling specific + GCS cap token will be written to the GCS, this is an architectural GCS cap + with the token type (bits 0..11) all clear. The GCSPR_EL0 reported in the + signal frame will point to this cap token. + +* The signal handler will use the same GCS as the interrupted context. + +* When GCS is enabled on signal entry a frame with the address of the signal + return handler will be pushed onto the GCS, allowing return from the signal + handler via RET as normal. This will not be reported in the gcs_context in + the signal frame. + + +5. Signal return +----------------- + +When returning from a signal handler: + +* If there is a gcs_context record in the signal frame then the GCS flags + and GCSPR_EL0 will be restored from that context prior to further + validation. + +* If there is no gcs_context record in the signal frame then the GCS + configuration will be unchanged. + +* If GCS is enabled on return from a signal handler then GCSPR_EL0 must + point to a valid GCS signal cap record, this will be popped from the + GCS prior to signal return. + +* If the GCS configuration is locked when returning from a signal then any + attempt to change the GCS configuration will be treated as an error. This + is true even if GCS was not enabled prior to signal entry. + +* GCS may be disabled via signal return but any attempt to enable GCS via + signal return will be rejected. + + +6. ptrace extensions +--------------------- + +* A new regset NT_ARM_GCS is defined for use with PTRACE_GETREGSET and + PTRACE_SETREGSET. + +* Due to the complexity surrounding allocation and deallocation of stacks and + lack of practical application it is not possible to enable GCS via ptrace. + GCS may be disabled via the ptrace interface. + +* Other GCS modes may be configured via ptrace. + +* Configuration via ptrace ignores locking of GCS mode bits. + + +7. ELF coredump extensions +--------------------------- + +* NT_ARM_GCS notes will be added to each coredump for each thread of the + dumped process. The contents will be equivalent to the data that would + have been read if a PTRACE_GETREGSET of the corresponding type were + executed for each thread when the coredump was generated. + + + +8. /proc extensions +-------------------- + +* Guarded Control Stack pages will include "ss" in their VmFlags in + /proc//smaps. diff --git a/Documentation/arch/arm64/index.rst b/Documentation/arch/arm64/index.rst index 78544de0a8a9..056f6a739d25 100644 --- a/Documentation/arch/arm64/index.rst +++ b/Documentation/arch/arm64/index.rst @@ -15,6 +15,7 @@ ARM64 Architecture cpu-feature-registers cpu-hotplug elf_hwcaps + gcs hugetlbpage kdump legacy_instructions From patchwork Tue Oct 1 22:58:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832108 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDE061CEEA2; Tue, 1 Oct 2024 23:01:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823701; cv=none; b=dmr9kuxCegpugqHVuBX+sRbnf93QCkWg5VS1U4uX9g9oMd5wXgWKnsh5p1ewZNGTLC3uZ5CgbeRkEqtuTx+5x9IlJ/XyFCbSSntIZCD6sU9d4G1MET4j1Bj+uj/QWRDQNEtqh8E7qMfywwqhOqI28+a1zvjuZbILQY7iS4Y+hHc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823701; c=relaxed/simple; bh=aU7VcPlcAVWfKwhYAkwXPGJufVjITRO3ag+Sd8hmpYI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=o2LVCrX6uxoxPlkKscriZ90t1Qt/0nRYsjiOO3fOc97EOQCKhblpM5yiIazPBO6ulJbLkc9ogsKmAAQPyholPWTYZ/pJw7sxveu8uzuVSm0hM5mHiUtrxCj3taj8pB9g5cq9stAiTdIFFB+xO3k4thlYeZpSW01rs1j8Hg886MI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dV73+Ea5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dV73+Ea5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C4FB6C4CEC6; Tue, 1 Oct 2024 23:01:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823700; bh=aU7VcPlcAVWfKwhYAkwXPGJufVjITRO3ag+Sd8hmpYI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dV73+Ea5guYEgEPkX2usmY5HovH63TaRUUqIwoMNSA8BkIKeLm6OjU6ojGIhqUs7U CI4eZP3fJvKmp72PKtaa1G9lqkuhfBysgZZ0qKnEexY1eHOGB3Wk+tulPLg5Jg0P36 Hc8mFDYughfODnEmyIL1gcZ0e8muPL+dbbReHwNwaOhZFDbjLkWGxGi0dYmUma4+79 KZbyL+ci4niWQ4Id2qQvRbPrKZ+brVlwDGekRKlMUrBctMUoKMbroW8lGFxIOufKyu jBwOaIwxcBD5wuj4f67t5hU5ck2cfXAE6J4IOCSHxTLi4SvTFohIYdhVVrnhwqBp2l LAK4QXY++l20g== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:48 +0100 Subject: [PATCH v13 09/40] arm64/gcs: Add manual encodings of GCS instructions Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-9-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=2696; i=broonie@kernel.org; h=from:subject:message-id; bh=aU7VcPlcAVWfKwhYAkwXPGJufVjITRO3ag+Sd8hmpYI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7LQGCFxhZ4o1Gnqs6EwqfSZFzmIWEbnCPdGfbE Mko0y22JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+ywAKCRAk1otyXVSH0PuqB/ 9/M5q7I6ky13ADECCsEXG802M1nKDkIbHJiCzaQA3xg86SIi+RIEw095I/kklJkR9rb4ewkuQSixwm WzRFIoPzNau9qpeYn49pBgf/H6+jWXMYiDWsHwQw39LojZetSpTA2ZgueG9CSsFa+tWUt4t1rJbFht K38wv2zESM+vYvtJJwvMObmkjO+9sCNAejKH9dWgFmQr8bTZwIoxeSFJgOB4mMqNeIrq7XOxYeNvvG xSWu5Rt60zUdGJl2yX2CaOTMX7RVE75meQaswAPVsFcXlzMYOLNPnnx4uTNw/w+8jCq+BmN6J2qWIM B9zpnCJRJ/uZH+tWhZp+wTblV/dZQs X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Define C callable functions for GCS instructions used by the kernel. In order to avoid ambitious toolchain requirements for GCS support these are manually encoded, this means we have fixed register numbers which will be a bit limiting for the compiler but none of these should be used in sufficiently fast paths for this to be a problem. Note that GCSSTTR is used to store to EL0. Reviewed-by: Thiago Jung Bauermann Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 51 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/include/asm/uaccess.h | 22 +++++++++++++++++ 2 files changed, 73 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h new file mode 100644 index 000000000000..7c5e95218db6 --- /dev/null +++ b/arch/arm64/include/asm/gcs.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2023 ARM Ltd. + */ +#ifndef __ASM_GCS_H +#define __ASM_GCS_H + +#include +#include + +static inline void gcsb_dsync(void) +{ + asm volatile(".inst 0xd503227f" : : : "memory"); +} + +static inline void gcsstr(u64 *addr, u64 val) +{ + register u64 *_addr __asm__ ("x0") = addr; + register long _val __asm__ ("x1") = val; + + /* GCSSTTR x1, x0 */ + asm volatile( + ".inst 0xd91f1c01\n" + : + : "rZ" (_val), "r" (_addr) + : "memory"); +} + +static inline void gcsss1(u64 Xt) +{ + asm volatile ( + "sys #3, C7, C7, #2, %0\n" + : + : "rZ" (Xt) + : "memory"); +} + +static inline u64 gcsss2(void) +{ + u64 Xt; + + asm volatile( + "SYSL %0, #3, C7, C7, #3\n" + : "=r" (Xt) + : + : "memory"); + + return Xt; +} + +#endif diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 1aa4ecb73429..0db494b24dd0 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -502,4 +502,26 @@ static inline size_t probe_subpage_writeable(const char __user *uaddr, #endif /* CONFIG_ARCH_HAS_SUBPAGE_FAULTS */ +#ifdef CONFIG_ARM64_GCS + +static inline int gcssttr(unsigned long __user *addr, unsigned long val) +{ + register unsigned long __user *_addr __asm__ ("x0") = addr; + register unsigned long _val __asm__ ("x1") = val; + int err = 0; + + /* GCSSTTR x1, x0 */ + asm volatile( + "1: .inst 0xd91f1c01\n" + "2: \n" + _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0) + : "+r" (err) + : "rZ" (_val), "r" (_addr) + : "memory"); + + return err; +} + +#endif /* CONFIG_ARM64_GCS */ + #endif /* __ASM_UACCESS_H */ From patchwork Tue Oct 1 22:58:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832107 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F2DE1BDA9B; Tue, 1 Oct 2024 23:01:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823718; cv=none; b=gH6zI1rOdZjpDiYXeGQ1Qk4zdmVE2E22Bxa2f5w59YPBgsM2JF4w03q+PprC/rsCBE339wNj+nodurWNLbJD8nGrZIceDh38bATryqfQotyUa7Rn9aMyon3L2B2/tpE7YTsU24xIrqNikEwSypAUXher5KoGqpL4y9ZLlxKk1dQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823718; c=relaxed/simple; bh=Ks8MFARHCgoYvKlZU6GLbHhVfaqXF8TPA2joPbZMDJ0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HNOq3aW6aynYM6hGSwDGFtN+dNMyzUiGTKw9CoNgvFKPL9DHhlX5pN98Rlc8ID3xKJS1KibFsfTTAtILuSkcBv8Nl0JtRI/JwrlyKhCkgs4A1elysKHIcecwjWvu8CLcitY/gvYx4ag2+yx8r5udq5XDmx9aWrAdEYzJtrUgojk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cn0MHg3e; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cn0MHg3e" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3D4AFC4CECF; Tue, 1 Oct 2024 23:01:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823718; bh=Ks8MFARHCgoYvKlZU6GLbHhVfaqXF8TPA2joPbZMDJ0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cn0MHg3eO0Yo4Or//lQRmgonz5ejCUv+6594M8VlW6g8B5NIWlg860BJ6D8qI2YgV GV+I3lNe9kuZP1vrCgqbzVS6hK9e+nc0txyMnJ9T8BT9Uh+OEmcDYrfNNz7bzBOn6S oRFXhyJvXgEpzBJ5KVKYMioEvdK+KG96dF0Km5qHym4e1l9SU/X5va2Xey3Dk154O4 hMljQY4YA3oBYMuXQLOUoRZaCvlmileovyiCzDXvVRsJaA8NHELWgzYESuf5NthmQY Z+sByWmrf/S56XQALexiOpbAyIVq6QTCeeCuG3iOdod/wVu9gWyN5eQUa+dfllnXpK SUfOmuhLGsQ+w== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:50 +0100 Subject: [PATCH v13 11/40] arm64/gcs: Provide basic EL2 setup to allow GCS usage at EL0 and EL1 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-11-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=2363; i=broonie@kernel.org; h=from:subject:message-id; bh=Ks8MFARHCgoYvKlZU6GLbHhVfaqXF8TPA2joPbZMDJ0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7Mo7StH9NeEdFx4e51cOj/G4fz2GUpoEOK3dVF ihqzuxKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+zAAKCRAk1otyXVSH0E4VB/ 44Y/DcxPyyJpegtnqk2Xgu4LfuVN9YN36+8Wa/TWW/x1dk/OsduoSR0EhzeC2naITe38OnI2yNsg2r HN/GaAXzc9BwXtFDMkIxufE38zZBxSH6tb7DRcyR4t+sY0wc8QE2X3AAAG9PSGMiA56nu5zdYlC7XP PLgYHgUmzw+YfzPsfZBVE2jh1vAbXfixjIO03HSI1Rbp1UsVnoZspSK5JVdwcA5R6nZcMCt8dA9tEp bAEQuk72ftCeVy1fIrWWJlYMyaTZ2W23ceUAsMF+g+Ef8tFQJxR/QpzGSrU3J/a3W2zLuuMjMuopKW 7ReB4Wzd9fUI3XBDKdquXjtl43L8R+ X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB There is a control HCRX_EL2.GCSEn which must be set to allow GCS features to take effect at lower ELs and also fine grained traps for GCS usage at EL0 and EL1. Configure all these to allow GCS usage by EL0 and EL1. We also initialise GCSCR_EL1 and GCSCRE0_EL1 to ensure that we can execute function call instructions without faulting regardless of the state when the kernel is started. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/el2_setup.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index e0ffdf13a18b..27086a81eae3 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -27,6 +27,14 @@ ubfx x0, x0, #ID_AA64MMFR1_EL1_HCX_SHIFT, #4 cbz x0, .Lskip_hcrx_\@ mov_q x0, HCRX_HOST_FLAGS + + /* Enable GCS if supported */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_hcrx_\@ + orr x0, x0, #HCRX_EL2_GCSEn + +.Lset_hcrx_\@: msr_s SYS_HCRX_EL2, x0 .Lskip_hcrx_\@: .endm @@ -200,6 +208,16 @@ orr x0, x0, #HFGxTR_EL2_nPOR_EL0 .Lskip_poe_fgt_\@: + /* GCS depends on PIE so we don't check it if PIE is absent */ + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lset_fgt_\@ + + /* Disable traps of access to GCS registers at EL0 and EL1 */ + orr x0, x0, #HFGxTR_EL2_nGCS_EL1_MASK + orr x0, x0, #HFGxTR_EL2_nGCS_EL0_MASK + +.Lset_fgt_\@: msr_s SYS_HFGRTR_EL2, x0 msr_s SYS_HFGWTR_EL2, x0 msr_s SYS_HFGITR_EL2, xzr @@ -215,6 +233,17 @@ .Lskip_fgt_\@: .endm +.macro __init_el2_gcs + mrs_s x1, SYS_ID_AA64PFR1_EL1 + ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 + cbz x1, .Lskip_gcs_\@ + + /* Ensure GCS is not enabled when we start trying to do BLs */ + msr_s SYS_GCSCR_EL1, xzr + msr_s SYS_GCSCRE0_EL1, xzr +.Lskip_gcs_\@: +.endm + .macro __init_el2_nvhe_prepare_eret mov x0, #INIT_PSTATE_EL1 msr spsr_el2, x0 @@ -240,6 +269,7 @@ __init_el2_nvhe_idregs __init_el2_cptr __init_el2_fgt + __init_el2_gcs .endm #ifndef __KVM_NVHE_HYPERVISOR__ From patchwork Tue Oct 1 22:58:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832106 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E1F61CF5DC; Tue, 1 Oct 2024 23:02:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823738; cv=none; b=aIkKvNH+fe/K4asBNTbWI7OsT/77eIZGnfPmvHtDe2Pg9sF2v3sx1N8pr5+Hmogbfo0ui2GrjLt8nBXlIbYhNOgXM4eB7obkB9fsc2iBsXXoSUiS992gFOQ2VpLtlx9N52vkvHgPg+mQec1dxnop/v98Cg1D3Wam50Xv6QEUzl4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823738; c=relaxed/simple; bh=vaxVaycm7ebuteD+83+ohSTITiaiPgd3wzWiS8vbKhY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pIhlj/FWq4CkVQ37pmH0lNrPcOlhHlUgTfF7y83Xz9R7eyi/YOX7w8hkx5vMrtTdoMZRFatl6H4FEoox4QRYCeQ+2xFtC5hGzDkDjKXaYuVnItE5qx4dwP19o/sQfN6X2ONsIA3gYyGxknWM4ddhB3g/QyInDvawRzH8mW/Sp8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rJUOqUeC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rJUOqUeC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A8B8C4CED9; Tue, 1 Oct 2024 23:02:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823738; bh=vaxVaycm7ebuteD+83+ohSTITiaiPgd3wzWiS8vbKhY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=rJUOqUeCp2KVjBBFXojZ/9SF6t/enxvGS6axJO57zNZy1jrB1lGqVlGRQCupS4fEC ohBq8aVleb5r7ZKGie5EI/2CJcaIhDRwb/r883erqa+zwMVLT9ucKZg7sPeYaKEllJ SJvjiMRWcPran3zzscDbr6RmpxmVIQflvNnfVfzjtkl3bMx6Q3VMYHtIvsAaF4Vv6N Dk0XWFy7M1ygsTK8ufANb1hagsfnWUP92/j9wVhjGsDcHiCoJToP7SQQRhJcbiZYqV NA/B+lI3wX5YMhr7mPMTD9VFEQPrfr/ia1nQWZLRLjWq6J7N7Cw5KQfHfoA+fvYTYL p7AZrwKzbL4LA== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:52 +0100 Subject: [PATCH v13 13/40] arm64/mm: Allocate PIE slots for EL0 guarded control stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-13-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=3090; i=broonie@kernel.org; h=from:subject:message-id; bh=vaxVaycm7ebuteD+83+ohSTITiaiPgd3wzWiS8vbKhY=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7OFsx7rjfggosVPd6CTMLpKPa20DpvMtc4pKI/ zV30kZWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+zgAKCRAk1otyXVSH0CG6CA CBp7hBCPEkMdZ/Nsef6Zwahuw5MDiInv9NS8uCA+9eVJb3v6hj99Gpjsxs0lJXz/2vwsFfecjDCu4+ BDPdelwOdutPr2FkIdL6d5B0pkTQZqFAl9r0SQf6CMC9HsteqnYHrP4f+SkVNxs+CKKMI7zlGa3y4r K0CCRmb+0VQ3NBkfhpU9qOS2rwuiS5D27GesuGwNzo62aWURmN/vqcrUWRaGFugEiicbaIX9fp0tud JdY6ias/O9Vbthbm8USjjYy1lLVkmZphsfLmtAuXoL2+9o0RaPhlGkPWXVWADwCIfOudmUVzgmPGGE MVqCzHKUuxHfesEWw2BTBuW17fC2gx X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Pages used for guarded control stacks need to be described to the hardware using the Permission Indirection Extension, GCS is not supported without PIE. In order to support copy on write for guarded stacks we allocate two values, one for active GCSs and one for GCS pages marked as read only prior to copy. Since the actual effect is defined using PIE the specific bit pattern used does not matter to the hardware but we choose two values which differ only in PTE_WRITE in order to help share code with non-PIE cases. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/pgtable-prot.h | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 2a11d0c10760..4e4bcd676f4c 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -144,15 +144,23 @@ static inline bool __pure lpa2_is_enabled(void) /* 6: PTE_PXN | PTE_WRITE */ /* 7: PAGE_SHARED_EXEC PTE_PXN | PTE_WRITE | PTE_USER */ /* 8: PAGE_KERNEL_ROX PTE_UXN */ -/* 9: PTE_UXN | PTE_USER */ +/* 9: PAGE_GCS_RO PTE_UXN | PTE_USER */ /* a: PAGE_KERNEL_EXEC PTE_UXN | PTE_WRITE */ -/* b: PTE_UXN | PTE_WRITE | PTE_USER */ +/* b: PAGE_GCS PTE_UXN | PTE_WRITE | PTE_USER */ /* c: PAGE_KERNEL_RO PTE_UXN | PTE_PXN */ /* d: PAGE_READONLY PTE_UXN | PTE_PXN | PTE_USER */ /* e: PAGE_KERNEL PTE_UXN | PTE_PXN | PTE_WRITE */ /* f: PAGE_SHARED PTE_UXN | PTE_PXN | PTE_WRITE | PTE_USER */ +#define _PAGE_GCS (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_WRITE | PTE_USER) +#define _PAGE_GCS_RO (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_USER) + +#define PAGE_GCS __pgprot(_PAGE_GCS) +#define PAGE_GCS_RO __pgprot(_PAGE_GCS_RO) + #define PIE_E0 ( \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_GCS) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_R) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_X_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_RX_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RWX_O) | \ @@ -160,6 +168,8 @@ static inline bool __pure lpa2_is_enabled(void) PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED), PIE_RW_O)) #define PIE_E1 ( \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_NONE_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW) | \ From patchwork Tue Oct 1 22:58:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832105 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6FB02207A; Tue, 1 Oct 2024 23:02:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823758; cv=none; b=l14c6ZnJGzewzYPBgR6nzWBaUk9P1Q+DiADCRvWPjWrn42AF0ApurND5GeBL4XCXhg8pLknxSm/GPRO6f+u0HpZGD4HjDD112DMttiJ4Zr1UhHGTXtMZlHRp/lTPms7bI0xgrAYdG4P7m5LabEHoHGl7QN1f1F9ac+I3wnvNzCo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823758; c=relaxed/simple; bh=iaOo/eHnt7nVAYYY04s0CJoGcZR71Wh0L0z0tfAjj5s=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JVun44LcetMuAWsYEnGDdj2rX9Pbt+e7D1SPBfTXjQUTvC03Tx46BXJ0QVV/gvYNAb7zEDOi/NBePE+92u8wCXJin4tpQCHTolMHjxwjRibQKheKenWXun/hBlH1qrig9uYrLujhljcHMS/a3VFYLTso8es5dfjwSF5MXbiTZ1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gUqqh7nY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gUqqh7nY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32691C4CEC6; Tue, 1 Oct 2024 23:02:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823758; bh=iaOo/eHnt7nVAYYY04s0CJoGcZR71Wh0L0z0tfAjj5s=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=gUqqh7nY8DifOXx4W85pnErZS2D1gnTtAy+3DgFxNMmQtLboM/sTyjbKi0UGUvxYE MnPV6ZCsIIJ6o7qGezeh+gI3YMVvc9Jb6VGhEpr3Ph1WOsyCPsnWO/QMsj4eHdpx20 P5298HT7y+dw1g1P5hyBAQlmS4D34zxnf2h9aYrrJGJtHSPGXczECZxbvomv0JSMEI 2brqU6kp7yqRC63sx+2NCcxw7ZEYrBATr3AqXdUwFLg8qnmQ0HQYdQ5eBj99fc3UTK 3WHErdya4+QHcjBF9VZIutYVj3Booae9dNtS22ZpYGEV1wpuiYpI32CInMnHzD7Phl eZsvlvNUlqAyQ== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:54 +0100 Subject: [PATCH v13 15/40] arm64/mm: Map pages for guarded control stack Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-15-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=1824; i=broonie@kernel.org; h=from:subject:message-id; bh=iaOo/eHnt7nVAYYY04s0CJoGcZR71Wh0L0z0tfAjj5s=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7P1EJJZsPfYIE0QBq/tINqmBg6uzvjA9MNO4G2 WFi9f7CJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+zwAKCRAk1otyXVSH0IY5B/ 9I4+JyQBe97ebsZEscKo1qo7/m+PT10LjgI7Sgz+xOEzztCrZMsOVvxpJOfRz9LSx3HXo7l/r2fMhO o70EbKlG3Ndfo8ukorzVw/0u/VlVIE+M0+uxJE46CJS+gaJLEvLlTr/mr9roUwkqHeI5YJ3u1qCxDg sfdiVWbdejibK6kkPIsuDcMl7qLDgPPgfGyCaCEVo58dYkq5niIfdflsydhtntcNM/C56Pa+c0aUx3 c1CWAKMjvXY1Y+G6FwbVbqON1ejk1JFXdCKgSslWxH6sD1MB8vblGF7f9LD5Jkj1AV8uIM49MW5V/u qCNkmAnsAgQFFEoUsAkeMHuk7yKJ/0 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Map pages flagged as being part of a GCS as such rather than using the full set of generic VM flags. This is done using a conditional rather than extending the size of protection_map since that would make for a very sparse array. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/mman.h | 9 +++++++++ arch/arm64/mm/mmap.c | 9 ++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h index 03b790fd0ad8..f6d784f8e6e0 100644 --- a/arch/arm64/include/asm/mman.h +++ b/arch/arm64/include/asm/mman.h @@ -71,6 +71,15 @@ static inline bool arch_validate_flags(unsigned long vm_flags) return false; } + if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { + /* An executable GCS isn't a good idea. */ + if (vm_flags & VM_EXEC) + return false; + + /* The memory management core should prevent this */ + VM_WARN_ON(vm_flags & VM_SHARED); + } + return true; } diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index 7e3ad97e27d8..07aeab8a7606 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -83,8 +83,15 @@ arch_initcall(adjust_protection_map); pgprot_t vm_get_page_prot(unsigned long vm_flags) { - pteval_t prot = pgprot_val(protection_map[vm_flags & + pteval_t prot; + + /* Short circuit GCS to avoid bloating the table. */ + if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { + prot = _PAGE_GCS_RO; + } else { + prot = pgprot_val(protection_map[vm_flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]); + } if (vm_flags & VM_ARM64_BTI) prot |= PTE_GP; From patchwork Tue Oct 1 22:58:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832104 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 776D51CFEA1; Tue, 1 Oct 2024 23:03:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823781; cv=none; b=o/C2SQQKcE6DV/I3D/BIeKAmx9pSKR4G2NoeIF7gCzRHs2ivjv1OTqsi2LniEVI5hod00Am79zkZxfjY+ZY0wjjkGliO2HSnAfmNDioHBP9mpZt99pyRys8O3mBmEqqlXFSpaS2Y2W7QqMachEZ5qqv+1tKVHJ7G+Q4cYol7neg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823781; c=relaxed/simple; bh=zAbUzr1ODeYIke6nRWrxiS99ZR09MpEN2O+JxNln/KQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=n4ZIjwXz1wYr7mtXqj7zPQY8BlEd1g7BgPgwzgdD+Q/XB/YTGbxNMBm7ZOCUacmkUyGnQGAa6aDT5UPyvRefrW+LNVyt1+qBwYk2PHAOG0xQnJ4fULOEZqmx1emO35It5EvZzbR0MDOdFvTOjRtp7YfLca77WTEevpAoITLq9+4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nwsAukUu; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nwsAukUu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F414AC4CEC6; Tue, 1 Oct 2024 23:02:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823781; bh=zAbUzr1ODeYIke6nRWrxiS99ZR09MpEN2O+JxNln/KQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nwsAukUuBWfza0uW/rUXY3JuaOEaCfpGhqWfdkW4N1gDIkUidRK/hasGJbxinc8n9 aJuU34SmkjIp5dTX+VS2pz6KWmffzyTp6vcj+BHHUSPAxm4MvcCE0UpxaD7Y5fKPkn ZIouN8Bo132zSXBrDXzBs9KdMFWMGBAL+8NcAJOHtPBextX0bEy24+65ZtcHu2XBQh 4ExV9wc+5W+7nYeRkqGn3uWRbIMhfHZWT4rIQWmnRpTLfUw142l34HvjeorA2Ftp5z LQR24dMGjVHOKiE8geEw2Dk+GA2M8PK0kQgzok44ktT6EG3tYjPymmKJsyQtX5RU9K rq1mVjQibGiCQ== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:56 +0100 Subject: [PATCH v13 17/40] arm64/idreg: Add overrride for GCS Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-17-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=2008; i=broonie@kernel.org; h=from:subject:message-id; bh=zAbUzr1ODeYIke6nRWrxiS99ZR09MpEN2O+JxNln/KQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7REUmBbOjHIbjRTXTBou6dlj2jf+JoQilSMLfb 4oamPWaJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+0QAKCRAk1otyXVSH0FP1B/ 4sD5XnZ55qcZbmL8d240mUgjPiUILjDGICiY1VAzT57XN7KmS4yKXFIBxESjVgvnC8PpI/Y9Zb7sLt BoUFcV+Q7U8gNjyR9hza7jtoxvCKcCe9Fn68gwmkX0Q4K0iHv87BggWlo0hChy9JQMvrDNyrRwBnMY BQAtnYj8LUX9i5RQKhYnVKqYIz6KT6wIZL3OCytHzA4lDd0VcqnaXlWy8vM+ahKTXCJCH1oxk8LAzG 7YIm6HPFMIia0KQmWqlfB//G7jzb5amS/Mc8t2m3DcdvgO2PX0nljBPoA5AgKnG93CQFNxYmlBQ5kG 7zXw7oMDda+u457zCFPWYThTod14ro X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Hook up an override for GCS, allowing it to be disabled from the command line by specifying arm64.nogcs in case there are problems. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- Documentation/admin-guide/kernel-parameters.txt | 3 +++ arch/arm64/kernel/pi/idreg-override.c | 2 ++ 2 files changed, 5 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 1518343bbe22..c1b00f709734 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -446,6 +446,9 @@ arm64.nobti [ARM64] Unconditionally disable Branch Target Identification support + arm64.nogcs [ARM64] Unconditionally disable Guarded Control Stack + support + arm64.nomops [ARM64] Unconditionally disable Memory Copy and Memory Set instructions support diff --git a/arch/arm64/kernel/pi/idreg-override.c b/arch/arm64/kernel/pi/idreg-override.c index 29d4b6244a6f..2bb709d78405 100644 --- a/arch/arm64/kernel/pi/idreg-override.c +++ b/arch/arm64/kernel/pi/idreg-override.c @@ -133,6 +133,7 @@ static const struct ftr_set_desc pfr1 __prel64_initconst = { .override = &id_aa64pfr1_override, .fields = { FIELD("bt", ID_AA64PFR1_EL1_BT_SHIFT, NULL ), + FIELD("gcs", ID_AA64PFR1_EL1_GCS_SHIFT, NULL), FIELD("mte", ID_AA64PFR1_EL1_MTE_SHIFT, NULL), FIELD("sme", ID_AA64PFR1_EL1_SME_SHIFT, pfr1_sme_filter), {} @@ -215,6 +216,7 @@ static const struct { { "arm64.nosve", "id_aa64pfr0.sve=0" }, { "arm64.nosme", "id_aa64pfr1.sme=0" }, { "arm64.nobti", "id_aa64pfr1.bt=0" }, + { "arm64.nogcs", "id_aa64pfr1.gcs=0" }, { "arm64.nopauth", "id_aa64isar1.gpi=0 id_aa64isar1.gpa=0 " "id_aa64isar1.api=0 id_aa64isar1.apa=0 " From patchwork Tue Oct 1 22:58:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832103 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5783D1CF2BE; Tue, 1 Oct 2024 23:03:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823819; cv=none; b=qmGknbq2AHtxWEOthI287xsOmwv8JZ9Nv+Iybogkpoueiv1pE3SjBK9JkdnzMVCjdTrJPBSBh0LxXTiAKwrutXhK8GE68HHrYZAyh5gRWeSWI/cFtVDOcxZDD86s7xtbiT8O8dPXwgaQf9g25nYfrvwoY0tE8wG0+71ncFif/a8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823819; c=relaxed/simple; bh=JwTS5J5Kz+XCxYDRc3FG3N1klYZC99PAyz93oxxWHns=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Xl340ZXE1vUbe1pJ2Tz33fDDnLtrnYD1w5NHEcbFdIPr7l+sniamvlpxcw3p1Za6LTPdhEpy49nX1VQBqSRMUTl8jp01KiQsXjERf3kov3ahTLbjFXN8DSAU6zlnh67HpiEX83D3qNjzb5QRrNFIxwWMBLp6sst0flRErSpVfLY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=h5QOvsyH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="h5QOvsyH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B91A5C4CEC6; Tue, 1 Oct 2024 23:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823818; bh=JwTS5J5Kz+XCxYDRc3FG3N1klYZC99PAyz93oxxWHns=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=h5QOvsyHwmwYJXTikpeXsVEp2A3UPoVjGZRUucHDnIEP8s20RwqCtfvtaEgYASJ3A 4VvT7Z9FkWpjGLRRUyAM0E57+nFfV/1smmkjKGkRZKQARGtcXv4Kr/B1Fg5EqqXPTT w7iMpC+tcgCrK+qsiDR7XnELMA4PkJgDckI9vpw8SVgbifNvpYEuLSdt70HOnbcLrh bILOGqSFtZOKo1haHLpTrtPKpk0NC0HS0Pd2sgi3K8cPhb+7HZxIGLJOcB3QJHfl0A Gos56B/AegT6wsD2YwKq8Ie1aAdRX49w3baVX3709fSna41T6qIRdgrbFb2IW97c4V 0q6PzQ7x+JAPg== From: Mark Brown Date: Tue, 01 Oct 2024 23:58:59 +0100 Subject: [PATCH v13 20/40] arm64/mm: Handle GCS data aborts Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-20-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=3733; i=broonie@kernel.org; h=from:subject:message-id; bh=JwTS5J5Kz+XCxYDRc3FG3N1klYZC99PAyz93oxxWHns=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7TVZ3jDKeIyJFS2xkZycVzFdIgkZ/2MVMSxegg PracWFmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+0wAKCRAk1otyXVSH0PPFB/ 9jOVBZ0sZYLFYt8CYSgFwth5bQAMffCYR65UIbAKTDAZV8czeEm1xDTBVqLzh2y8cJ8L9rksWGRzsa KoNNdusUYbP0WwOzwiB8f26x4k5gKBP8dzKhvHobyjoVwM9dwHJ1P+31Rnk9S2VC0HlJ12ukGOll7A CZczYl0vb1teCaug4ldvi8DJRkAAMotUN8oeVWz1J0E9GwHb+eMDvTL9igdY9wrw4EyL8kkGp2K8aF B+pBMS0HXzUFSpXDbnSX5eVKfGA1myQSsEM0KQsGSA4aaHTZ8Guexvn78tQkU7ofyEMejYY/2DYTmb IZnC/hzjnKbZohafBaYTyRcPVaFiYq X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB All GCS operations at EL0 must happen on a page which is marked as having UnprivGCS access, including read operations. If a GCS operation attempts to access a page without this then it will generate a data abort with the GCS bit set in ESR_EL1.ISS2. EL0 may validly generate such faults, for example due to copy on write which will cause the GCS data to be stored in a read only page with no GCS permissions until the actual copy happens. Since UnprivGCS allows both reads and writes to the GCS (though only through GCS operations) we need to ensure that the memory management subsystem handles GCS accesses as writes at all times. Do this by adding FAULT_FLAG_WRITE to any GCS page faults, adding handling to ensure that invalid cases are identfied as such early so the memory management core does not think they will succeed. The core cannot distinguish between VMAs which are generally writeable and VMAs which are only writeable through GCS operations. EL1 may validly write to EL0 GCS for management purposes (eg, while initialising with cap tokens). We also report any GCS faults in VMAs not marked as part of a GCS as access violations, causing a fault to be delivered to userspace if it attempts to do GCS operations outside a GCS. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/mm/fault.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 8b281cf308b3..c2f89a678ac0 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -504,6 +504,14 @@ static bool fault_from_pkey(unsigned long esr, struct vm_area_struct *vma, false); } +static bool is_gcs_fault(unsigned long esr) +{ + if (!esr_is_data_abort(esr)) + return false; + + return ESR_ELx_ISS2(esr) & ESR_ELx_GCS; +} + static bool is_el0_instruction_abort(unsigned long esr) { return ESR_ELx_EC(esr) == ESR_ELx_EC_IABT_LOW; @@ -518,6 +526,23 @@ static bool is_write_abort(unsigned long esr) return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); } +static bool is_invalid_gcs_access(struct vm_area_struct *vma, u64 esr) +{ + if (!system_supports_gcs()) + return false; + + if (unlikely(is_gcs_fault(esr))) { + /* GCS accesses must be performed on a GCS page */ + if (!(vma->vm_flags & VM_SHADOW_STACK)) + return true; + } else if (unlikely(vma->vm_flags & VM_SHADOW_STACK)) { + /* Only GCS operations can write to a GCS page */ + return esr_is_data_abort(esr) && is_write_abort(esr); + } + + return false; +} + static int __kprobes do_page_fault(unsigned long far, unsigned long esr, struct pt_regs *regs) { @@ -554,6 +579,14 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, /* It was exec fault */ vm_flags = VM_EXEC; mm_flags |= FAULT_FLAG_INSTRUCTION; + } else if (is_gcs_fault(esr)) { + /* + * The GCS permission on a page implies both read and + * write so always handle any GCS fault as a write fault, + * we need to trigger CoW even for GCS reads. + */ + vm_flags = VM_WRITE; + mm_flags |= FAULT_FLAG_WRITE; } else if (is_write_abort(esr)) { /* It was write fault */ vm_flags = VM_WRITE; @@ -587,6 +620,13 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, if (!vma) goto lock_mmap; + if (is_invalid_gcs_access(vma, esr)) { + vma_end_read(vma); + fault = 0; + si_code = SEGV_ACCERR; + goto bad_area; + } + if (!(vma->vm_flags & vm_flags)) { vma_end_read(vma); fault = 0; From patchwork Tue Oct 1 22:59:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832102 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B0E01CEEB4; Tue, 1 Oct 2024 23:04:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823843; cv=none; b=CetkYhnVkTvPRZlR8xoTbAUt89qV9i6txlugBCbL4Vl4uzis+9THv/4CPsdhTswqXKG5u1E8iOH8Cq/bZPUJEmBz4NaPuaVQO2KfpJN86EoKOw3NOdktBeQmci0R80Lw9rXWtBqfYqVzskupRECKaIvxKpOZDkXEtL73s71/two= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823843; c=relaxed/simple; bh=Us0URBRhSa1wij7R5AMlAWepe3NAZ1hCwtEWfYpyTZE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=f2dVFxk+QbMWT9t7z6St0bbK3adOZLKLX/uKvPYV72lsC9uvDkztdajkYUbxIr2qmg5n/r7etIgKQWQxFCsmRrvW6TjUw5NTZLFYtx47SeEHEgj9PFitkMQUYTZsXy2vye5KGBx3TAkdWcxAVzoR3lpooV5RRUqo1UwNh95uqBg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=s0KZ+4Dm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="s0KZ+4Dm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30237C4CEC6; Tue, 1 Oct 2024 23:03:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823843; bh=Us0URBRhSa1wij7R5AMlAWepe3NAZ1hCwtEWfYpyTZE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=s0KZ+4DmO57LtfWSEGq6YAnDihkmJYoPIG37qRni6HuPyMr3zWQ6hRuGS9/aTk65Q 1ssmYu48gex0m25/V7FJkX7yht1tdIZ9pLXNuqu09xMODWyC2cfR1yl81ePxYlLEVt s+NCCZbo5pHSrFUKyXzU0w0HFRdvw94utNVwKST1QyV8CrdIPXjXPpFWm/EwxGscL5 QMTIdumdfZ10g9iUwI/rrX3YhyVNs50mpmVRCR/98UlDvcCWH0lDyGt7Y8exwNFUIu /6UrglK/eqmhjCvlw7OcX986EexRyIR+sybhoMV7Dp/eg5+gIDhMInI70t3MmkPxhP nSOgci1rnSpZg== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:01 +0100 Subject: [PATCH v13 22/40] arm64/gcs: Ensure that new threads have a GCS Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-22-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=7454; i=broonie@kernel.org; h=from:subject:message-id; bh=Us0URBRhSa1wij7R5AMlAWepe3NAZ1hCwtEWfYpyTZE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7Vlw2a91awf5dQ51yn2oKKo++ah0gmIhormFHO IFzKoGuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+1QAKCRAk1otyXVSH0GXiB/ 4sukR3e9ueVGNs3Q1p22exSd9lbXbDllUPyFJRC1zfyQjKx2EzGvmcpMdDVmk5/goay67QMxQjKDJd WntsAEwdsM3HHSD8pAZHPYIaWcfhsMw8mU1tFplNjSWdG+PH5bOQ8dOtN8qxNrMsz+YdzlcudTCoKR UgDf5yrtZkS9eH06xyw8KTSHt6uWM6MUGZ83TexJbc/KJGL4TbD03CYvzG0PE7oAvngEQ4kavKV1QH QSdtefxEFMfWOajyTpqhateKmpXWdS8jlaoqcW3pxVRbiUKRU0e5aMcS2rO+8KTK+gXtnpUR0m+OA4 VFyI+/KUpM2Sy1v7HCZ6SdXOYftgXV X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB When a new thread is created by a thread with GCS enabled the GCS needs to be specified along with the regular stack. Unfortunately plain clone() is not extensible and existing clone3() users will not specify a stack so all existing code would be broken if we mandated specifying the stack explicitly. For compatibility with these cases and also x86 (which did not initially implement clone3() support for shadow stacks) if no GCS is specified we will allocate one so when a thread is created which has GCS enabled allocate one for it. We follow the extensively discussed x86 implementation and allocate min(RLIMIT_STACK/2, 2G). Since the GCS only stores the call stack and not any variables this should be more than sufficient for most applications. GCSs allocated via this mechanism will be freed when the thread exits. Reviewed-by: Thiago Jung Bauermann Acked-by: Yury Khrustalev Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 9 +++++ arch/arm64/include/asm/mmu_context.h | 9 +++++ arch/arm64/kernel/process.c | 32 +++++++++++++++++ arch/arm64/mm/gcs.c | 69 ++++++++++++++++++++++++++++++++++++ 4 files changed, 119 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 04594ef59dad..c1f274fdb9c0 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -8,6 +8,8 @@ #include #include +struct kernel_clone_args; + static inline void gcsb_dsync(void) { asm volatile(".inst 0xd503227f" : : : "memory"); @@ -58,6 +60,8 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) void gcs_set_el0_mode(struct task_struct *task); void gcs_free(struct task_struct *task); void gcs_preserve_current_state(void); +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); #else @@ -69,6 +73,11 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) static inline void gcs_set_el0_mode(struct task_struct *task) { } static inline void gcs_free(struct task_struct *task) { } static inline void gcs_preserve_current_state(void) { } +static inline unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + return -ENOTSUPP; +} #endif diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 7c09d47e09cb..48b3d9553b67 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -311,6 +312,14 @@ static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, return por_el0_allows_pkey(vma_pkey(vma), write, execute); } +#define deactivate_mm deactivate_mm +static inline void deactivate_mm(struct task_struct *tsk, + struct mm_struct *mm) +{ + gcs_free(tsk); +} + + #include #endif /* !__ASSEMBLY__ */ diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index aedcf332f422..fdd095480c3f 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -294,9 +294,35 @@ static void flush_gcs(void) write_sysreg_s(0, SYS_GCSPR_EL0); } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + unsigned long gcs; + + if (!system_supports_gcs()) + return 0; + + p->thread.gcs_base = 0; + p->thread.gcs_size = 0; + + gcs = gcs_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(gcs)) + return PTR_ERR((void *)gcs); + + p->thread.gcs_el0_mode = current->thread.gcs_el0_mode; + p->thread.gcs_el0_locked = current->thread.gcs_el0_locked; + + return 0; +} + #else static void flush_gcs(void) { } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + return 0; +} #endif @@ -313,6 +339,7 @@ void flush_thread(void) void arch_release_task_struct(struct task_struct *tsk) { fpsimd_release_task(tsk); + gcs_free(tsk); } int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) @@ -376,6 +403,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long stack_start = args->stack; unsigned long tls = args->tls; struct pt_regs *childregs = task_pt_regs(p); + int ret; memset(&p->thread.cpu_context, 0, sizeof(struct cpu_context)); @@ -420,6 +448,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.uw.tp_value = tls; p->thread.tpidr2_el0 = 0; } + + ret = copy_thread_gcs(p, args); + if (ret != 0) + return ret; } else { /* * A kthread has no context to ERET to, so ensure any buggy diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index f8f4f984a247..3c7a18f57ea9 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -5,9 +5,69 @@ #include #include +#include #include +#include #include +static unsigned long alloc_gcs(unsigned long addr, unsigned long size) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE; + struct mm_struct *mm = current->mm; + unsigned long mapped_addr, unused; + + if (addr) + flags |= MAP_FIXED_NOREPLACE; + + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, + VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); + mmap_write_unlock(mm); + + return mapped_addr; +} + +static unsigned long gcs_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + /* Allocate RLIMIT_STACK/2 with limits of PAGE_SIZE..2G */ + size = PAGE_ALIGN(min_t(unsigned long long, + rlimit(RLIMIT_STACK) / 2, SZ_2G)); + return max(PAGE_SIZE, size); +} + +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size; + + if (!system_supports_gcs()) + return 0; + + if (!task_gcs_el0_enabled(tsk)) + return 0; + + if ((args->flags & (CLONE_VFORK | CLONE_VM)) != CLONE_VM) { + tsk->thread.gcspr_el0 = read_sysreg_s(SYS_GCSPR_EL0); + return 0; + } + + size = args->stack_size / 2; + + size = gcs_size(size); + addr = alloc_gcs(0, size); + if (IS_ERR_VALUE(addr)) + return addr; + + tsk->thread.gcs_base = addr; + tsk->thread.gcs_size = size; + tsk->thread.gcspr_el0 = addr + size - sizeof(u64); + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. @@ -33,6 +93,15 @@ void gcs_free(struct task_struct *task) if (!system_supports_gcs()) return; + /* + * When fork() with CLONE_VM fails, the child (tsk) already + * has a GCS allocated, and exit_thread() calls this function + * to free it. In this case the parent (current) and the + * child share the same mm struct. + */ + if (!task->mm || task->mm != current->mm) + return; + if (task->thread.gcs_base) vm_munmap(task->thread.gcs_base, task->thread.gcs_size); From patchwork Tue Oct 1 22:59:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832101 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 378CD1CF5E1; Tue, 1 Oct 2024 23:04:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823867; cv=none; b=H2DaRReuFu/KrSswyMqJiYBIia1uMSBzoGHoNepP4IdcGpFPW0pVoHPpsmBKQQNnPHZACxUy7HFOS9OZHn0GXbETEzRszYPyT5PPpBDFQpwfVArc6SCKhNCqeElAO7BoLTy4Jdz1/07OFZ/Hwdf1j4VYtXmJ1sDDG3TyyY+ePRQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823867; c=relaxed/simple; bh=qUaWHBHdN5KhPSaC2shYeXLiosV3WxoA3aMO7oZsLVE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=hFDDcAhsgBFRskUKsqbn1PQLNmwAsigewg1P+i3BKyJNFzOyzhmJwJedMb+zkFeRTFsRtIUUbZZOnCyBrHN4veDH2hE5kzV79HCd7OiH7aMuqOYjXUTVJexkWtBJOLuMwPpoI0N+fpn9bPUUvJ9QLYUMr5sdvW8LVBdZluS9E14= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R5eOO/Cw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R5eOO/Cw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2731BC4CEC6; Tue, 1 Oct 2024 23:04:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823866; bh=qUaWHBHdN5KhPSaC2shYeXLiosV3WxoA3aMO7oZsLVE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=R5eOO/CwUjlApWCgKSs8nP1G9FbTW6gonLZZKFpKZtiXZTuq5ksbI0wpEZjOTU8yG xKBo6Dd0I2/GI0kHJEkkXYarMC1cRGNbblyUwlD/NHUJB0gNe2M34+Pvj+i5NMs2oW AQUVAUxEn0i8MeiNshGyJtj0LUF/KvnAe3Q76pZS78zE69xC5Ne9aYD9h3mJjXX39m WvwPuQhQeS2KJ4LBzyZMXZmT9HjML+MzgorTHGfC0IFNwyg630Fv4fYif2qm7VlHqP ieU/iLIRdW7+XBOXwFy6sWKRc9mDVgT4WTb5lVgTRBUoaFCDboJBJV5vPgPvvQHQfu x3so/N3EBojRw== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:03 +0100 Subject: [PATCH v13 24/40] arm64/mm: Implement map_shadow_stack() Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-24-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=3164; i=broonie@kernel.org; h=from:subject:message-id; bh=qUaWHBHdN5KhPSaC2shYeXLiosV3WxoA3aMO7oZsLVE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7W5jFfBYx0dD/AiVB3co7PUfmUOb1RGg81DGmy uzTdjduJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+1gAKCRAk1otyXVSH0O7/B/ 42dRKw5u6PwVHw5uyGMv0ruX389ObAvdjm3Y36WIttnDUP0tWY+nfpebp14vMHoDi1oOFtcjrptu1r ZlVTVs7h1ekurkDzRQ2XJm89Mop9C4VeausdyagksJAtlGy43yqbrsMZysKPPEPLgeMJrgL+bs0yhY hHpo0pRPoFNGUK3Jzl9WSDCsKobZbzpSDpA+7AdL8f93D4UFUH7XCRkK0oY7mo/u2UCTFOK+8fNwkL CVDcP/EVT9hqzjaMyv4mIH4Ij7rJyU6y/ZXTMMHEPZUQZUOqHuYMaoePQShnA8T4Qw8zPwyZ4dkJZ0 KnR47/XLkVLmJuFGlBMHuGsjK6mlZf X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB As discussed extensively in the changelog for the addition of this syscall on x86 ("x86/shstk: Introduce map_shadow_stack syscall") the existing mmap() and madvise() syscalls do not map entirely well onto the security requirements for guarded control stacks since they lead to windows where memory is allocated but not yet protected or stacks which are not properly and safely initialised. Instead a new syscall map_shadow_stack() has been defined which allocates and initialises a shadow stack page. Implement this for arm64. Two flags are provided, allowing applications to request that the stack be initialised with a valid cap token at the top of the stack and optionally also an end of stack marker above that. We support requesting an end of stack marker alone but since this is a NULL pointer it is indistinguishable from not initialising anything by itself. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Acked-by: Yury Khrustalev Signed-off-by: Mark Brown --- arch/arm64/mm/gcs.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index 61a80de6baf8..5c46ec527b1c 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -68,6 +68,70 @@ unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, return addr; } +SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) +{ + unsigned long alloc_size; + unsigned long __user *cap_ptr; + unsigned long cap_val; + int ret = 0; + int cap_offset; + + if (!system_supports_gcs()) + return -EOPNOTSUPP; + + if (flags & ~(SHADOW_STACK_SET_TOKEN | SHADOW_STACK_SET_MARKER)) + return -EINVAL; + + if (!PAGE_ALIGNED(addr)) + return -EINVAL; + + if (size == 8 || !IS_ALIGNED(size, 8)) + return -EINVAL; + + /* + * An overflow would result in attempting to write the restore token + * to the wrong location. Not catastrophic, but just return the right + * error code and block it. + */ + alloc_size = PAGE_ALIGN(size); + if (alloc_size < size) + return -EOVERFLOW; + + addr = alloc_gcs(addr, alloc_size); + if (IS_ERR_VALUE(addr)) + return addr; + + /* + * Put a cap token at the end of the allocated region so it + * can be switched to. + */ + if (flags & SHADOW_STACK_SET_TOKEN) { + /* Leave an extra empty frame as a top of stack marker? */ + if (flags & SHADOW_STACK_SET_MARKER) + cap_offset = 2; + else + cap_offset = 1; + + cap_ptr = (unsigned long __user *)(addr + size - + (cap_offset * sizeof(unsigned long))); + cap_val = GCS_CAP(cap_ptr); + + put_user_gcs(cap_val, cap_ptr, &ret); + if (ret != 0) { + vm_munmap(addr, size); + return -EFAULT; + } + + /* + * Ensure the new cap is ordered before standard + * memory accesses to the same location. + */ + gcsb_dsync(); + } + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. From patchwork Tue Oct 1 22:59:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832100 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DCDF1CF29F; Tue, 1 Oct 2024 23:04:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823889; cv=none; b=Wapw3E83HgKsbL2q2J8jrdvUrJSihmmjT/8TC4l18oHJ42tAK28f9NcR72qaJ1bThPFrL5DmcgRlaQPu5aqQYkz4I+qoFsyPqSGH/6t88HtGQYC66vLDZQb6yt0/YyuEHrTGg8t0ntlH4VQAEn+re075Yub7biRpJKrLv/GJH9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823889; c=relaxed/simple; bh=iheYVdOI1mRov1x2wnyFTq0nLuVXvcgzNFY1vvTX4L4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YfhpK0ahtwtyOgUY8RoK5Qh/SNdWfWgq3EQo8V20IRe9I0ACnJwZUPSHDKN9lB2ctWbykpbRCNrY8uE/pZVkV6uUYCNszGrRAeDDsubufyx/JO8MERPa1hIkA+wt4fE9UMBaKIwIIbICtg1Y2bu/tifZOmV+WU/k9GOE1xLKrX8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BwIW27cY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BwIW27cY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8BA5C4CEC6; Tue, 1 Oct 2024 23:04:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823889; bh=iheYVdOI1mRov1x2wnyFTq0nLuVXvcgzNFY1vvTX4L4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BwIW27cYksAnySxigWOYt0SrEnqSg8DQYIaOnWhOA0RmnhGPkZNsPdLvpGjXLHoFy x9mBOmkAgbznrCwLJIwQXwgbSpQgLgda83m7E8XTGR/hpwmO9+3OPAi1uU0TCEeGyM 9JiEyK/6y5TPZbGoIs2Q4X1QSD6nD+29VI2lTNHh20WKw6hGcZlOF91t4tyvyGGUud Er3KBwhnr6s/S5WpZBGIZsV6T9/+OZ8Q9UeC9TufeeP6UnCEheXFhvzHTC1pWfDdE7 blzYCs94us/bd2Nqjy+3llEwkSsm9Kk0bQaxm2WUjd4zYSbW2gZTb+pX3qgl8J3TvX ZINL9rlEvPKdg== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:05 +0100 Subject: [PATCH v13 26/40] arm64/signal: Expose GCS state in signal frames Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-26-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=6331; i=broonie@kernel.org; h=from:subject:message-id; bh=iheYVdOI1mRov1x2wnyFTq0nLuVXvcgzNFY1vvTX4L4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7Yz8d8MeWJ1s7RbiNfQuHzARatODnl4idncq8R cuNHIXOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+2AAKCRAk1otyXVSH0IwMB/ 9jNDjhhihZZ4eg1B5HLLhXzEXv8N0aVlgGh/C0HpKuP1JRpb9vWvmXYmEb9b+OED6jwkbAaDTq34DO tIgwn8Hxp1xHgkhwqJPTA/ZlaSRZsLMpjQGYO50gNphK4ZTqB8UsmomfvnmqpiHRG2Tvn9yfF6ctdn 6jCSM3nAAnS9z/iZyVIRdD+DJ6uqNygJh5d+zzcrtYXSjxfHHUH6WFm4VP9CsuiY/zjMTz3cQCR+GD Dkyz+qG+3r8mAsYatUuQH1MZvCZPtuexKj1rWX0DbluY1QzNURLAZ+Wzhk6r8UfrQ2C7Dpaj+1j8fk vzEPmC/B62LJ0D69GBKu8xUJsBBlQj X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add a context for the GCS state and include it in the signal context when running on a system that supports GCS. We reuse the same flags that the prctl() uses to specify which GCS features are enabled and also provide the current GCS pointer. We do not support enabling GCS via signal return, there is a conflict between specifying GCSPR_EL0 and allocation of a new GCS and this is not an ancticipated use case. We also enforce GCS configuration locking on signal return. Reviewed-by: Catalin Marinas Reviewed-by: Thiago Jung Bauermann Acked-by: Yury Khrustalev Signed-off-by: Mark Brown --- arch/arm64/include/uapi/asm/sigcontext.h | 9 +++ arch/arm64/kernel/signal.c | 109 +++++++++++++++++++++++++++++++ 2 files changed, 118 insertions(+) diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h index bb7af77a30a7..d42f7a92238b 100644 --- a/arch/arm64/include/uapi/asm/sigcontext.h +++ b/arch/arm64/include/uapi/asm/sigcontext.h @@ -183,6 +183,15 @@ struct zt_context { __u16 __reserved[3]; }; +#define GCS_MAGIC 0x47435300 + +struct gcs_context { + struct _aarch64_ctx head; + __u64 gcspr; + __u64 features_enabled; + __u64 reserved; +}; + #endif /* !__ASSEMBLY__ */ #include diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index b5ab0e229a78..62d666278264 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -66,6 +66,7 @@ struct rt_sigframe_user_layout { unsigned long fpsimd_offset; unsigned long esr_offset; + unsigned long gcs_offset; unsigned long sve_offset; unsigned long tpidr2_offset; unsigned long za_offset; @@ -198,6 +199,8 @@ struct user_ctxs { u32 fpmr_size; struct poe_context __user *poe; u32 poe_size; + struct gcs_context __user *gcs; + u32 gcs_size; }; static int preserve_fpsimd_context(struct fpsimd_context __user *ctx) @@ -643,6 +646,82 @@ extern int restore_zt_context(struct user_ctxs *user); #endif /* ! CONFIG_ARM64_SME */ +#ifdef CONFIG_ARM64_GCS + +static int preserve_gcs_context(struct gcs_context __user *ctx) +{ + int err = 0; + u64 gcspr = read_sysreg_s(SYS_GCSPR_EL0); + + /* + * If GCS is enabled we will add a cap token to the frame, + * include it in the GCSPR_EL0 we report to support stack + * switching via sigreturn if GCS is enabled. We do not allow + * enabling via sigreturn so the token is only relevant for + * threads with GCS enabled. + */ + if (task_gcs_el0_enabled(current)) + gcspr -= 8; + + __put_user_error(GCS_MAGIC, &ctx->head.magic, err); + __put_user_error(sizeof(*ctx), &ctx->head.size, err); + __put_user_error(gcspr, &ctx->gcspr, err); + __put_user_error(0, &ctx->reserved, err); + __put_user_error(current->thread.gcs_el0_mode, + &ctx->features_enabled, err); + + return err; +} + +static int restore_gcs_context(struct user_ctxs *user) +{ + u64 gcspr, enabled; + int err = 0; + + if (user->gcs_size != sizeof(*user->gcs)) + return -EINVAL; + + __get_user_error(gcspr, &user->gcs->gcspr, err); + __get_user_error(enabled, &user->gcs->features_enabled, err); + if (err) + return err; + + /* Don't allow unknown modes */ + if (enabled & ~PR_SHADOW_STACK_SUPPORTED_STATUS_MASK) + return -EINVAL; + + err = gcs_check_locked(current, enabled); + if (err != 0) + return err; + + /* Don't allow enabling */ + if (!task_gcs_el0_enabled(current) && + (enabled & PR_SHADOW_STACK_ENABLE)) + return -EINVAL; + + /* If we are disabling disable everything */ + if (!(enabled & PR_SHADOW_STACK_ENABLE)) + enabled = 0; + + current->thread.gcs_el0_mode = enabled; + + /* + * We let userspace set GCSPR_EL0 to anything here, we will + * validate later in gcs_restore_signal(). + */ + write_sysreg_s(gcspr, SYS_GCSPR_EL0); + + return 0; +} + +#else /* ! CONFIG_ARM64_GCS */ + +/* Turn any non-optimised out attempts to use these into a link error: */ +extern int preserve_gcs_context(void __user *ctx); +extern int restore_gcs_context(struct user_ctxs *user); + +#endif /* ! CONFIG_ARM64_GCS */ + static int parse_user_sigframe(struct user_ctxs *user, struct rt_sigframe __user *sf) { @@ -661,6 +740,7 @@ static int parse_user_sigframe(struct user_ctxs *user, user->zt = NULL; user->fpmr = NULL; user->poe = NULL; + user->gcs = NULL; if (!IS_ALIGNED((unsigned long)base, 16)) goto invalid; @@ -777,6 +857,17 @@ static int parse_user_sigframe(struct user_ctxs *user, user->fpmr_size = size; break; + case GCS_MAGIC: + if (!system_supports_gcs()) + goto invalid; + + if (user->gcs) + goto invalid; + + user->gcs = (struct gcs_context __user *)head; + user->gcs_size = size; + break; + case EXTRA_MAGIC: if (have_extra_context) goto invalid; @@ -896,6 +987,9 @@ static int restore_sigframe(struct pt_regs *regs, err = restore_fpsimd_context(&user); } + if (err == 0 && system_supports_gcs() && user.gcs) + err = restore_gcs_context(&user); + if (err == 0 && system_supports_tpidr2() && user.tpidr2) err = restore_tpidr2_context(&user); @@ -1029,6 +1123,15 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user, return err; } +#ifdef CONFIG_ARM64_GCS + if (system_supports_gcs() && (add_all || current->thread.gcspr_el0)) { + err = sigframe_alloc(user, &user->gcs_offset, + sizeof(struct gcs_context)); + if (err) + return err; + } +#endif + if (system_supports_sve() || system_supports_sme()) { unsigned int vq = 0; @@ -1136,6 +1239,12 @@ static int setup_sigframe(struct rt_sigframe_user_layout *user, __put_user_error(current->thread.fault_code, &esr_ctx->esr, err); } + if (system_supports_gcs() && err == 0 && user->gcs_offset) { + struct gcs_context __user *gcs_ctx = + apply_user_offset(user, user->gcs_offset); + err |= preserve_gcs_context(gcs_ctx); + } + /* Scalable Vector Extension state (including streaming), if present */ if ((system_supports_sve() || system_supports_sme()) && err == 0 && user->sve_offset) { From patchwork Tue Oct 1 22:59:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832099 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FC2B1CDFD9; Tue, 1 Oct 2024 23:05:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823911; cv=none; b=bf12NfmZqP9pZ1+HswggeWBJ8mUKNIRgodjkKq9AEjYB8ogU8uCNjCjqZU4XfBdVMiyQmPqUXYr835w4Bif47T1FjRNEhFHvexwxlukYa+/zXTyq8ems+pg0mzCClmtV9nVGH3cYha4+JXRDieNsO0uEmSaTcGwCqrYybMrZsgw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823911; c=relaxed/simple; bh=atDZEtbemNIieqZC6RtXrtbJMuxF2tLKf2K/TSJvVes=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZYxvXC6vITjiBI2CWJsx0RMhqQzbn2yaOB/lYI5yPlE5BZ4NWdY/020vUo7MMlOwLUcLQCg0EejeLgGumCRkl2H8lIVAQfIKpN1T/je2xZ4AvGbRdhXhMyodUXl3gRyhmDhWBqJL7FCeLd9bOY/NmA21QwPENKNN7DL45V6GBhw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=B5J7eMmH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="B5J7eMmH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2430C4CEC6; Tue, 1 Oct 2024 23:04:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823910; bh=atDZEtbemNIieqZC6RtXrtbJMuxF2tLKf2K/TSJvVes=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=B5J7eMmHKHd97SZEJxiVYhbNX/Ks8kJn5Nue5+50O9KQLxkG1E+ZShZeg7g74vkcV A18Gq9rmhXe19Z4kUcF+pJs+Qtvsqr91+6+b7Kbsshk5rbEYcPSxMF6dM0k1CoOe1h e7cF+4+kslmzvSIQSqXfflUfAQAyfMhFn+CLChMvKBH4y0HQkYfZBLgHsx4waUTth1 FFWq7cK+cnEGEWp6Vnhnn8l3ARtW5Mw5+nZSWMAvmkBEWfgsMfYi/qYMdxye19J2jn B7Sl2jtRZqa10PTgSCq8QLQWkDF2ckA/wZ0Me+T/OCoSkGoPz9AVJA8AU9mCB+UO5i YC+XhEF1TuzLw== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:07 +0100 Subject: [PATCH v13 28/40] arm64: Add Kconfig for Guarded Control Stack (GCS) Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-28-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=1477; i=broonie@kernel.org; h=from:subject:message-id; bh=atDZEtbemNIieqZC6RtXrtbJMuxF2tLKf2K/TSJvVes=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7Zc7rKkETgD5/LTaxxdr7BGJRS0qSsZjNw/txV r2Wuw/+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+2QAKCRAk1otyXVSH0M+qB/ 91ODENwPWdKuLlElj/7OYeWGSCjN5ITwHJ4ppzWwF1BP5tpBhvEs9TuGenH2UXxcohCfUT6yjbeNU7 BqHi1YY0+wQUtr4ulVRIVOnj1gq7q8h/tS/NBM0kzuoPNW/h3FxRY8l5bCnYAo5i4/0MKAznj5S5pL tj2aRrRznvG5Pj2t4m43nTUIxrTduG05Cdqisl6Txjh+LJSLhBOsiLtFeWGetiiTrHDcpNisfrWB/g INyNQ3jEJ06Kni4oXAI1kFQBiWcbUVqmeRHJg4jEpObnjwLN4GS3NWbsDfDtfdFgL9q0FM2zOz1vfP x+uZvUmrp6KnGE2kvCpjKmctHavSdF X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Provide a Kconfig option allowing the user to select if GCS support is built into the kernel. Reviewed-by: Thiago Jung Bauermann Reviewed-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/Kconfig | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 3e29b44d2d7b..dcb12f041c13 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2178,6 +2178,27 @@ config ARCH_PKEY_BITS endmenu # "ARMv8.9 architectural features" +menu "v9.4 architectural features" + +config ARM64_GCS + bool "Enable support for Guarded Control Stack (GCS)" + default y + select ARCH_HAS_USER_SHADOW_STACK + select ARCH_USES_HIGH_VMA_FLAGS + depends on !UPROBES + help + Guarded Control Stack (GCS) provides support for a separate + stack with restricted access which contains only return + addresses. This can be used to harden against some attacks + by comparing return address used by the program with what is + stored in the GCS, and may also be used to efficiently obtain + the call stack for applications such as profiling. + + The feature is detected at runtime, and will remain disabled + if the system does not implement the feature. + +endmenu # "v9.4 architectural features" + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y From patchwork Tue Oct 1 22:59:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832098 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76C3D1CF7D1; Tue, 1 Oct 2024 23:05:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823934; cv=none; b=ifJuV3sGb95EoNA4LwM5yXLf1993+rQyx6HT1086ek+t4PnHuFDDxKK/FQlh3AJckLfpRxwy0BOrbRztAZcoZQqSeK2Xw+bppenrnBaslG8OuTjoLG0uAGDDNilarB0DVOzV5fkotlgxd1GgpiBHz+uueIzobAmzXBpgGkg7X0k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823934; c=relaxed/simple; bh=qbIXmU2eUtmemMq8xp+h1yj3ghFTmpSN4hiSxbQmL24=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cqjhJijQiEKei/4eOx6pSznT3KHOjObRX9Fpvqpr5JUcRFEGv2ahoYAEi8IMB+ESe1g3MBsw5nl2mcHriBoxX3/shVecNg8Vd+qq4UTBezPz6m/xBhwivPooeyhBqquA9qlx6JX4jRxqnRFDjUdToA1U56yeVV+n3P0EjxSlMdA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=iLfN8hK6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="iLfN8hK6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4A385C4CEC6; Tue, 1 Oct 2024 23:05:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823933; bh=qbIXmU2eUtmemMq8xp+h1yj3ghFTmpSN4hiSxbQmL24=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=iLfN8hK6fHJsCL43iSYWHnlQe8lmB/0kpA5kef24n6vIHD3puN6AUc0DPk4sOqABb wWAjZyCkgL0G9BIFMcx8vXrgyMxvMHHfYAHX+x+mB0u7AfBeK8aepBzLUdkq6pOEZv N56GqM7jshutguOWelLOOIe09qymjx/165hlaMxOg7nQT69Losj8yjFsr85AwUTXyZ c+lc9rAWrd+ijT8JfypMqXpIHR89Q94sffXErVyiSHR+KNkso6Qb3wgi5CScjjwN3+ cA7pnI/kGEugcQvkMWCHyU01V4KUQOzIthpcHlAIkQa2BGUJojXKuFps2EShSHPk/9 U6SlBWyZTQfgw== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:09 +0100 Subject: [PATCH v13 30/40] kselftest/arm64: Add GCS as a detected feature in the signal tests Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-30-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=1892; i=broonie@kernel.org; h=from:subject:message-id; bh=qbIXmU2eUtmemMq8xp+h1yj3ghFTmpSN4hiSxbQmL24=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7b2/iy2LdLJp3um+baJpFqFpKYtxnFOd3PCELd fKwZqKaJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+2wAKCRAk1otyXVSH0NHEB/ 9EKBm0Nlsi2M1QMfMGnwSQGrHLQyo0Ec2/WoKudVqFQEvsSyGbPTSFYvGmQ1dt68l8+uMZJF6ZolQ6 +p9amzeznqBLv2udaMcOQCbVRNUVkhWWEVPH03tjMMJ2JCfAhJiu5tWmU+AO3dNrD0+IKLhlvAy7a4 ytAjYf8dykZi4yMOOBEZVFM2y0oDkNBzrnLNudtDx1M0m5EGTdCOtsOuOgbcYE3iCawBsDuPo5PVii W2sqXjLa0a3JOYmFrJ+qR3PhmfLgNx6tlv0SDPPQuPqi6bStN5a3v662I9gyQ3nFzHQQPFLgjdhlGT zbzEjrNGMmUur+kCBdWhwHlyVS3mx9 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for testing GCS related signal handling add it as a feature we check for in the signal handling support code. Reviewed-by: Thiago Jung Bauermann Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/signal/test_signals.h | 2 ++ tools/testing/selftests/arm64/signal/test_signals_utils.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/tools/testing/selftests/arm64/signal/test_signals.h b/tools/testing/selftests/arm64/signal/test_signals.h index 1e6273d81575..7ada43688c02 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.h +++ b/tools/testing/selftests/arm64/signal/test_signals.h @@ -35,6 +35,7 @@ enum { FSME_BIT, FSME_FA64_BIT, FSME2_BIT, + FGCS_BIT, FMAX_END }; @@ -43,6 +44,7 @@ enum { #define FEAT_SME (1UL << FSME_BIT) #define FEAT_SME_FA64 (1UL << FSME_FA64_BIT) #define FEAT_SME2 (1UL << FSME2_BIT) +#define FEAT_GCS (1UL << FGCS_BIT) /* * A descriptor used to describe and configure a test case. diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c index 0dc948db3a4a..dcc49e3ce1eb 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.c +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c @@ -30,6 +30,7 @@ static char const *const feats_names[FMAX_END] = { " SME ", " FA64 ", " SME2 ", + " GCS ", }; #define MAX_FEATS_SZ 128 @@ -329,6 +330,8 @@ int test_init(struct tdescr *td) td->feats_supported |= FEAT_SME_FA64; if (getauxval(AT_HWCAP2) & HWCAP2_SME2) td->feats_supported |= FEAT_SME2; + if (getauxval(AT_HWCAP) & HWCAP_GCS) + td->feats_supported |= FEAT_GCS; if (feats_ok(td)) { if (td->feats_required & td->feats_supported) fprintf(stderr, From patchwork Tue Oct 1 22:59:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832097 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73F6B1CFEAD; Tue, 1 Oct 2024 23:05:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823954; cv=none; b=DQQ9QKLgDuJN/ulr+VngChhQIerQH4wkaDFCfqMWMzHdJaF1O1B0UWgGlJq5f3tHl9Fe1qRdI/At6kIxoVtTNJHOZWJfhyFyjbi4HO5VuRuZRmhW1mCi6bCkOe+uLeP8SZYStP3/ZYRbQt7pdEsaB9dJWD9FvepnmF9fyXbcubE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823954; c=relaxed/simple; bh=HmmAMUZllaI31DzUS2UptX/TOKKFhFSrPJJKYwJO4R8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=G1X8e1a92VMYAPCquAOziGzK4OFkvfLinr0rJn50JuOYOmVXPSVfzTXagWoXgMgc65fuJKBc9Jsa7YIhakEOU63XAB4HV6L+/K3I4KZBcjGW2ZtY1hY0J5QsTQiQgziTJ341mVNu0jBZjDpP4zhajnOzIB+3YJLFr236l2TwmeY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oWWWef7q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oWWWef7q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5472C4CEC6; Tue, 1 Oct 2024 23:05:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823954; bh=HmmAMUZllaI31DzUS2UptX/TOKKFhFSrPJJKYwJO4R8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=oWWWef7qAoDsa3O284WmbA0+4gtsVF9fQGw5bErbwRfW/G/3F2SnO82AUhRfOPPKp 8iSUpwDGzBViwQtHsL3bQEW8BkhBnpj9tsYQ9f2AEG1HMszClmQxtjptGjzvfdp9TR 2tnBMAP2zHB7RwsCEnx1Q9Jw/Er87rPJHsfadtyl3WUB9Qp8Z5LArX/rxGdsCM4okJ AwQM6Sb3okhzC6Iz/NkZ7AYbUJHbyTX09NUjSYuyaWLQV++JhEiAu5b3rjgLU71fgb Qm6GFQeB2KZVXCXH7y0lGxAjEK9TD15YOJxqXZCZHRmKZNCIA0RDT7Xp5ibX4kyD39 l4XZfCeHsT78w== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:11 +0100 Subject: [PATCH v13 32/40] kselftest/arm64: Allow signals tests to specify an expected si_code Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-32-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=2693; i=broonie@kernel.org; h=from:subject:message-id; bh=HmmAMUZllaI31DzUS2UptX/TOKKFhFSrPJJKYwJO4R8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7d5yGKXzr5saRN3DNRkwpkNhqMTqqjdMk4lgiz LOIonX6JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+3QAKCRAk1otyXVSH0JaKB/ 91iK7iv9R+ueyloiMWoxyPRjComH147TYmfbIX9Ykm8qRaL4igLeAchB/ATQueCmLNC6Pxpeowl4QZ NYwqAhZdyqAlYXQZc7c4+Rg4CRUKUKUsWKWX7sfqAgnoXhUYaitLkAb34xofi2MRlJ6FLRrENO1XVf DSLs8fbdKAurGonLAbqeHFbVzI5LfwqHtakqeDAH8dvk2F9Tmt7xxdlnRP3qt8APEzJfneZp5Vf5io QROYJbPR8axNhug6Vn8nmgIjd6adwj+/ZoxkkVwwNWTu6IcwnjuV7KdVL6Z0H1siVkldER5/d0RtbK j4xQwxJqxUFu2TYrOofQwMa8i38WEN X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Currently we ignore si_code unless the expected signal is a SIGSEGV, in which case we enforce it being SEGV_ACCERR. Allow test cases to specify exactly which si_code should be generated so we can validate this, and test for other segfault codes. Reviewed-by: Thiago Jung Bauermann Signed-off-by: Mark Brown --- .../testing/selftests/arm64/signal/test_signals.h | 4 +++ .../selftests/arm64/signal/test_signals_utils.c | 29 ++++++++++++++-------- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/arm64/signal/test_signals.h b/tools/testing/selftests/arm64/signal/test_signals.h index 7ada43688c02..ee75a2c25ce7 100644 --- a/tools/testing/selftests/arm64/signal/test_signals.h +++ b/tools/testing/selftests/arm64/signal/test_signals.h @@ -71,6 +71,10 @@ struct tdescr { * Zero when no signal is expected on success */ int sig_ok; + /* + * expected si_code for sig_ok, or 0 to not check + */ + int sig_ok_code; /* signum expected on unsupported CPU features. */ int sig_unsupp; /* a timeout in second for test completion */ diff --git a/tools/testing/selftests/arm64/signal/test_signals_utils.c b/tools/testing/selftests/arm64/signal/test_signals_utils.c index dcc49e3ce1eb..5d3621921cfe 100644 --- a/tools/testing/selftests/arm64/signal/test_signals_utils.c +++ b/tools/testing/selftests/arm64/signal/test_signals_utils.c @@ -143,16 +143,25 @@ static bool handle_signal_ok(struct tdescr *td, "current->token ZEROED...test is probably broken!\n"); abort(); } - /* - * Trying to narrow down the SEGV to the ones generated by Kernel itself - * via arm64_notify_segfault(). This is a best-effort check anyway, and - * the si_code check may need to change if this aspect of the kernel - * ABI changes. - */ - if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { - fprintf(stdout, - "si_code != SEGV_ACCERR...test is probably broken!\n"); - abort(); + if (td->sig_ok_code) { + if (si->si_code != td->sig_ok_code) { + fprintf(stdout, "si_code is %d not %d\n", + si->si_code, td->sig_ok_code); + abort(); + } + } else { + /* + * Trying to narrow down the SEGV to the ones + * generated by Kernel itself via + * arm64_notify_segfault(). This is a best-effort + * check anyway, and the si_code check may need to + * change if this aspect of the kernel ABI changes. + */ + if (td->sig_ok == SIGSEGV && si->si_code != SEGV_ACCERR) { + fprintf(stdout, + "si_code != SEGV_ACCERR...test is probably broken!\n"); + abort(); + } } td->pass = 1; /* From patchwork Tue Oct 1 22:59:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832096 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB8E51CF7D6; Tue, 1 Oct 2024 23:06:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823976; cv=none; b=RJRiEf0/U9StnPdn5vK8a/ZVJAm1KJEUPlnOrZkbKHIwd5HLdYqGZFgU9c4SoyiqQZ/PnvezAFX2kEq2y9IeE6UWmwLrs9WHJ7kByriD73HmLCOgVeuYCXnleWx3/XtiwefVr6Q8LeiSpGWDOM35lzBjfHx4X/OU/ferLJkLLDI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823976; c=relaxed/simple; bh=Q17T+yWBmbeXt5IFTELe5+0SvI5sZQYMONrJzT8GYVk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kRMPHcbSYbo13MKcx8vx+zugg/2M2HgHBpTulziLTqydAtOVPCE2NZPlj6t+hNY9wM5SqzrkSr6Oz0e1v9D4+WKT5iPPIi6IUHydv+lFFSQNCKWqjSXwi2tY9SP6rfICvXpq+fN4W1/8yNfJnsWeSEtn3VydsgjTQb3HIt+ntXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YkFNSpYQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YkFNSpYQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A025C4CECD; Tue, 1 Oct 2024 23:06:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823976; bh=Q17T+yWBmbeXt5IFTELe5+0SvI5sZQYMONrJzT8GYVk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=YkFNSpYQeheyRp/N67f4dTfhLKXJEhHMycTSTF3CBcydocntNO5yVgSUbG6pLZ+Bd UP6P1YxDY8UwOkkUGwW50AE7NFURvXzc18llA6fzDVXVWyVWwKGHLM9WJ1kMvYpSuQ gkXl0fmwFUcKMOUpXgrv+V+NxK2JIq4AHuSmFclAK5KNUzPWM0gq/YxzuV/3yZduz/ 3DFKwVMmet18n6cmZgNhsxsgSX/8cRxRCW4bJilPl2Ct4NyBQIXO8+jvk7ocEhknD8 mG6mbBtO+UH79ssG9U0or9su377IpqyXyKOfFNH74ErV0EovWRZ+Krfg/7S1EGmZ6Y yJ5ELXtWGMlEw== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:13 +0100 Subject: [PATCH v13 34/40] kselftest/arm64: Add very basic GCS test program Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-34-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=13394; i=broonie@kernel.org; h=from:subject:message-id; bh=Q17T+yWBmbeXt5IFTELe5+0SvI5sZQYMONrJzT8GYVk=; b=owGbwMvMwMWocq27KDak/QLjabUkhrQ/dfcMi/V2LnBafPKHKOvyryFTisX4bRS/ibrkx1o1WL34 9uRUJ6MxCwMjF4OsmCLL2mcZq9LDJbbOfzT/FcwgViaQKQxcnAIwkTIt9v/u+579nS1+778BW7KLqA JPXkqboN8Eva89r0KsDSfr6t/REfu7fp+n/daPAs2d5wxE1XY1FAdnPlpStmTJoZteSt83/NvAyPi1 vbHi6APphWo5iX0HLuuXnnE56BfjXF8vaJRq75uYPuFLoUr35cPbK1plrV1eJCqmMPx3WbKz8NWzPx PkBX87GC6sqV2Qpb1a5pXh+sKkC/fSfm8pVM2tKS/iLXo4z0vllj+P3KtNWkp7RZlz+F2f6li0u6w6 9jgn6+M0l30VPRvl8+4z5f4v3iAtraVcdSPCcZZAfvJtuSk+L5xLaueterlG/HHRddOuAo/qAxxOrL k/p5+68DGoxadPsiU0xFSjniEPAA== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB This test program just covers the basic GCS ABI, covering aspects of the ABI as standalone features without attempting to integrate things. Reviewed-by: Thiago Jung Bauermann Tested-by: Thiago Jung Bauermann Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/Makefile | 2 +- tools/testing/selftests/arm64/gcs/.gitignore | 1 + tools/testing/selftests/arm64/gcs/Makefile | 18 ++ tools/testing/selftests/arm64/gcs/basic-gcs.c | 357 ++++++++++++++++++++++++++ tools/testing/selftests/arm64/gcs/gcs-util.h | 90 +++++++ 5 files changed, 467 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile index 28b93cab8c0d..22029e60eff3 100644 --- a/tools/testing/selftests/arm64/Makefile +++ b/tools/testing/selftests/arm64/Makefile @@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi +ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs else ARM64_SUBTARGETS := endif diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore new file mode 100644 index 000000000000..0e5e695ecba5 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -0,0 +1 @@ +basic-gcs diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile new file mode 100644 index 000000000000..61a30f483429 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 ARM Limited +# +# In order to avoid interaction with the toolchain and dynamic linker the +# portions of these tests that interact with the GCS are implemented using +# nolibc. +# + +TEST_GEN_PROGS := basic-gcs + +include ../../lib.mk + +$(OUTPUT)/basic-gcs: basic-gcs.c + $(CC) -g -fno-asynchronous-unwind-tables -fno-ident -s -Os -nostdlib \ + -static -include ../../../../include/nolibc/nolibc.h \ + -I../../../../../usr/include \ + -std=gnu99 -I../.. -g \ + -ffreestanding -Wall $^ -o $@ -lgcc diff --git a/tools/testing/selftests/arm64/gcs/basic-gcs.c b/tools/testing/selftests/arm64/gcs/basic-gcs.c new file mode 100644 index 000000000000..3fb9742342a3 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/basic-gcs.c @@ -0,0 +1,357 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Limited. + */ + +#include +#include + +#include + +#include +#include +#include + +#include "kselftest.h" +#include "gcs-util.h" + +/* nolibc doesn't have sysconf(), just hard code the maximum */ +static size_t page_size = 65536; + +static __attribute__((noinline)) void valid_gcs_function(void) +{ + /* Do something the compiler can't optimise out */ + my_syscall1(__NR_prctl, PR_SVE_GET_VL); +} + +static inline int gcs_set_status(unsigned long mode) +{ + bool enabling = mode & PR_SHADOW_STACK_ENABLE; + int ret; + unsigned long new_mode; + + /* + * The prctl takes 1 argument but we need to ensure that the + * other 3 values passed in registers to the syscall are zero + * since the kernel validates them. + */ + ret = my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, mode, + 0, 0, 0); + + if (ret == 0) { + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &new_mode, 0, 0, 0); + if (ret == 0) { + if (new_mode != mode) { + ksft_print_msg("Mode set to %lx not %lx\n", + new_mode, mode); + ret = -EINVAL; + } + } else { + ksft_print_msg("Failed to validate mode: %d\n", ret); + } + + if (enabling != chkfeat_gcs()) { + ksft_print_msg("%senabled by prctl but %senabled in CHKFEAT\n", + enabling ? "" : "not ", + chkfeat_gcs() ? "" : "not "); + ret = -EINVAL; + } + } + + return ret; +} + +/* Try to read the status */ +static bool read_status(void) +{ + unsigned long state; + int ret; + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &state, 0, 0, 0); + if (ret != 0) { + ksft_print_msg("Failed to read state: %d\n", ret); + return false; + } + + return state & PR_SHADOW_STACK_ENABLE; +} + +/* Just a straight enable */ +static bool base_enable(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE failed %d\n", ret); + return false; + } + + return true; +} + +/* Check we can read GCSPR_EL0 when GCS is enabled */ +static bool read_gcspr_el0(void) +{ + unsigned long *gcspr_el0; + + ksft_print_msg("GET GCSPR\n"); + gcspr_el0 = get_gcspr(); + ksft_print_msg("GCSPR_EL0 is %p\n", gcspr_el0); + + return true; +} + +/* Also allow writes to stack */ +static bool enable_writeable(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE writeable failed: %d\n", ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +/* Also allow writes to stack */ +static bool enable_push_pop(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE with push failed: %d\n", + ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +/* Enable GCS and allow everything */ +static bool enable_all(void) +{ + int ret; + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH | + PR_SHADOW_STACK_WRITE); + if (ret) { + ksft_print_msg("PR_SHADOW_STACK_ENABLE with everything failed: %d\n", + ret); + return false; + } + + ret = gcs_set_status(PR_SHADOW_STACK_ENABLE); + if (ret) { + ksft_print_msg("failed to restore plain enable %d\n", ret); + return false; + } + + return true; +} + +static bool enable_invalid(void) +{ + int ret = gcs_set_status(ULONG_MAX); + if (ret == 0) { + ksft_print_msg("GCS_SET_STATUS %lx succeeded\n", ULONG_MAX); + return false; + } + + return true; +} + +/* Map a GCS */ +static bool map_guarded_stack(void) +{ + int ret; + uint64_t *buf; + uint64_t expected_cap; + int elem; + bool pass = true; + + buf = (void *)my_syscall3(__NR_map_shadow_stack, 0, page_size, + SHADOW_STACK_SET_MARKER | + SHADOW_STACK_SET_TOKEN); + if (buf == MAP_FAILED) { + ksft_print_msg("Failed to map %lu byte GCS: %d\n", + page_size, errno); + return false; + } + ksft_print_msg("Mapped GCS at %p-%p\n", buf, + (void *)((uint64_t)buf + page_size)); + + /* The top of the newly allocated region should be 0 */ + elem = (page_size / sizeof(uint64_t)) - 1; + if (buf[elem]) { + ksft_print_msg("Last entry is 0x%llx not 0x0\n", buf[elem]); + pass = false; + } + + /* Then a valid cap token */ + elem--; + expected_cap = ((uint64_t)buf + page_size - 16); + expected_cap &= GCS_CAP_ADDR_MASK; + expected_cap |= GCS_CAP_VALID_TOKEN; + if (buf[elem] != expected_cap) { + ksft_print_msg("Cap entry is 0x%llx not 0x%llx\n", + buf[elem], expected_cap); + pass = false; + } + ksft_print_msg("cap token is 0x%llx\n", buf[elem]); + + /* The rest should be zeros */ + for (elem = 0; elem < page_size / sizeof(uint64_t) - 2; elem++) { + if (!buf[elem]) + continue; + ksft_print_msg("GCS slot %d is 0x%llx not 0x0\n", + elem, buf[elem]); + pass = false; + } + + ret = munmap(buf, page_size); + if (ret != 0) { + ksft_print_msg("Failed to unmap %ld byte GCS: %d\n", + page_size, errno); + pass = false; + } + + return pass; +} + +/* A fork()ed process can run */ +static bool test_fork(void) +{ + unsigned long child_mode; + int ret, status; + pid_t pid; + bool pass = true; + + pid = fork(); + if (pid == -1) { + ksft_print_msg("fork() failed: %d\n", errno); + pass = false; + goto out; + } + if (pid == 0) { + /* In child, make sure we can call a function, read + * the GCS pointer and status and then exit */ + valid_gcs_function(); + get_gcspr(); + + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &child_mode, 0, 0, 0); + if (ret == 0 && !(child_mode & PR_SHADOW_STACK_ENABLE)) { + ksft_print_msg("GCS not enabled in child\n"); + ret = -EINVAL; + } + + exit(ret); + } + + /* + * In parent, check we can still do function calls then block + * for the child. + */ + valid_gcs_function(); + + ksft_print_msg("Waiting for child %d\n", pid); + + ret = waitpid(pid, &status, 0); + if (ret == -1) { + ksft_print_msg("Failed to wait for child: %d\n", + errno); + return false; + } + + if (!WIFEXITED(status)) { + ksft_print_msg("Child exited due to signal %d\n", + WTERMSIG(status)); + pass = false; + } else { + if (WEXITSTATUS(status)) { + ksft_print_msg("Child exited with status %d\n", + WEXITSTATUS(status)); + pass = false; + } + } + +out: + + return pass; +} + +typedef bool (*gcs_test)(void); + +static struct { + char *name; + gcs_test test; + bool needs_enable; +} tests[] = { + { "read_status", read_status }, + { "base_enable", base_enable, true }, + { "read_gcspr_el0", read_gcspr_el0 }, + { "enable_writeable", enable_writeable, true }, + { "enable_push_pop", enable_push_pop, true }, + { "enable_all", enable_all, true }, + { "enable_invalid", enable_invalid, true }, + { "map_guarded_stack", map_guarded_stack }, + { "fork", test_fork }, +}; + +int main(void) +{ + int i, ret; + unsigned long gcs_mode; + + ksft_print_header(); + + /* + * We don't have getauxval() with nolibc so treat a failure to + * read GCS state as a lack of support and skip. + */ + ret = my_syscall5(__NR_prctl, PR_GET_SHADOW_STACK_STATUS, + &gcs_mode, 0, 0, 0); + if (ret != 0) + ksft_exit_skip("Failed to read GCS state: %d\n", ret); + + if (!(gcs_mode & PR_SHADOW_STACK_ENABLE)) { + gcs_mode = PR_SHADOW_STACK_ENABLE; + ret = my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + gcs_mode, 0, 0, 0); + if (ret != 0) + ksft_exit_fail_msg("Failed to enable GCS: %d\n", ret); + } + + ksft_set_plan(ARRAY_SIZE(tests)); + + for (i = 0; i < ARRAY_SIZE(tests); i++) { + ksft_test_result((*tests[i].test)(), "%s\n", tests[i].name); + } + + /* One last test: disable GCS, we can do this one time */ + my_syscall5(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, 0, 0, 0, 0); + if (ret != 0) + ksft_print_msg("Failed to disable GCS: %d\n", ret); + + ksft_finished(); + + return 0; +} diff --git a/tools/testing/selftests/arm64/gcs/gcs-util.h b/tools/testing/selftests/arm64/gcs/gcs-util.h new file mode 100644 index 000000000000..1ae6864d3f86 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-util.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2023 ARM Limited. + */ + +#ifndef GCS_UTIL_H +#define GCS_UTIL_H + +#include + +#ifndef __NR_map_shadow_stack +#define __NR_map_shadow_stack 453 +#endif + +#ifndef __NR_prctl +#define __NR_prctl 167 +#endif + +/* Shadow Stack/Guarded Control Stack interface */ +#define PR_GET_SHADOW_STACK_STATUS 74 +#define PR_SET_SHADOW_STACK_STATUS 75 +#define PR_LOCK_SHADOW_STACK_STATUS 76 + +# define PR_SHADOW_STACK_ENABLE (1UL << 0) +# define PR_SHADOW_STACK_WRITE (1UL << 1) +# define PR_SHADOW_STACK_PUSH (1UL << 2) + +#define PR_SHADOW_STACK_ALL_MODES \ + PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE | PR_SHADOW_STACK_PUSH + +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ +#define SHADOW_STACK_SET_MARKER (1ULL << 1) /* Set up a top of stack merker in the shadow stack */ + +#define GCS_CAP_ADDR_MASK (0xfffffffffffff000UL) +#define GCS_CAP_TOKEN_MASK (0x0000000000000fffUL) +#define GCS_CAP_VALID_TOKEN 1 +#define GCS_CAP_IN_PROGRESS_TOKEN 5 + +#define GCS_CAP(x) (((unsigned long)(x) & GCS_CAP_ADDR_MASK) | \ + GCS_CAP_VALID_TOKEN) + +static inline unsigned long *get_gcspr(void) +{ + unsigned long *gcspr; + + asm volatile( + "mrs %0, S3_3_C2_C5_1" + : "=r" (gcspr) + : + : "cc"); + + return gcspr; +} + +static inline void __attribute__((always_inline)) gcsss1(unsigned long *Xt) +{ + asm volatile ( + "sys #3, C7, C7, #2, %0\n" + : + : "rZ" (Xt) + : "memory"); +} + +static inline unsigned long __attribute__((always_inline)) *gcsss2(void) +{ + unsigned long *Xt; + + asm volatile( + "SYSL %0, #3, C7, C7, #3\n" + : "=r" (Xt) + : + : "memory"); + + return Xt; +} + +static inline bool chkfeat_gcs(void) +{ + register long val __asm__ ("x16") = 1; + + /* CHKFEAT x16 */ + asm volatile( + "hint #0x28\n" + : "=r" (val) + : "r" (val)); + + return val != 1; +} + +#endif From patchwork Tue Oct 1 22:59:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832095 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3D721CFEC8; Tue, 1 Oct 2024 23:06:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823998; cv=none; b=Nv4RmoTznbRDhz2d9kK1wfp6Fz7fMesqq/21XbqsbJxGNOn7alUDG35/1e9vb/w5CBRe6mtjcDTbOWKMeXW1drag6AKLlmlx2MIoa+9DWFJwueH3X9p9SrbDDIlcb4aHDENOIYVByQeXuf3lqUNIjTluU7kMSox0CwHJjiLfmUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727823998; c=relaxed/simple; bh=Keu8ANmZTdMHjv5XQqqLSm0bGXVuBnTuWvLw/oRshns=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iNo7VerWb+oFqRw1INqsTO1bJELYFXhfxulRaG5q9xUlIp96qbxa+F/eVQ6fnxEeQFga0z9T24zDCJOgYsaIQN9ubynnM8kSGO1Kma8IsRkMPDS2lr+cbDG4XgTcdKkiiJAxqmTJsHaUtVMoA8PRsQ9xfOIPrtFQXIoMXKArw1I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sF0vzapk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sF0vzapk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 774DAC4CED2; Tue, 1 Oct 2024 23:06:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823998; bh=Keu8ANmZTdMHjv5XQqqLSm0bGXVuBnTuWvLw/oRshns=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=sF0vzapkZ6OWVZ0k/kSzj3dKXchXzPS8npUmWJm1MibYtXim0Q9mxZwMWt97Di6xU yda6vZmOzJlaTRdQT2nxKlx9PcxcmeUxFRTv3juYtpJgOn1vaCcg1jaqPL1PPqxujd 7TuUsHctTr/bSf6uZlOYdv0qiVEXBHPousZcV8ZuvmzC10O3rKJQqPnnjo6ANTQbSq oV5u++OT6okSC9IE/37dGxczAWt268dY6guvk/M0eAqQ9wDSMOLN53HqS5ZVAv20Dp v9NkRpNaV2Bs19uxW+xQ4VJoATkfLh9VT9jvZwgQVU5xHwRdyg3XlTKeFsJ5OgwiAF LO80OecjvMfQw== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:15 +0100 Subject: [PATCH v13 36/40] kselftest/arm64: Add test coverage for GCS mode locking Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-36-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=7444; i=broonie@kernel.org; h=from:subject:message-id; bh=Keu8ANmZTdMHjv5XQqqLSm0bGXVuBnTuWvLw/oRshns=; b=owGbwMvMwMWocq27KDak/QLjabUkhrQ/dQ9seMOy9MQMBIJ+ba6S1nVc7f8h0+Ve3ik2vRO8cwRe Oe/sZDRmYWDkYpAVU2RZ+yxjVXq4xNb5j+a/ghnEygQyhYGLUwAmIhbG/r9AVzTE7VlHEu8a91ecHA f2KbDWB+VsursoMLVvTcYLlk+TfV4sU2EK2eZb+qFWYNlJhx1qW20EVdMUejhU2SyiF8/fPz/D9Zlx +34bz/r7F5TTAk3OyMwqbJHjeH805R3XSf4XORIiKYnbc5VVrXsy/x1qnOVUf9fS/9lxyat9/gsfrY 6pD4r+7X1PQFL+Ma/MgpKjnSszynmzP1TKKoh4yMRsL1r0/XXChhtO8wUWCy9ybnwY1pzsPPe6SYpb xO7VO6RmC/5d8TbIu3+awIT5M9YZPinXiZYTm/67mPm6iGDzU3P+DU+rUzPL8459OGb06qyu0y+2Aj /NK9WPfTL53i+SePD25PyIo/McAQ== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Verify that we can lock individual GCS mode bits, that other modes aren't affected and as a side effect also that every combination of modes can be enabled. Normally the inability to reenable GCS after disabling it would be an issue with testing but fortunately the kselftest_harness runs each test within a fork()ed child. This can be inconvenient for some kinds of testing but here it means that each test is in a separate thread and therefore won't be affected by other tests in the suite. Once we get toolchains with support for enabling GCS by default we will need to take care to not do that in the build system but there are no such toolchains yet so it is not yet an issue. Reviewed-by: Thiago Jung Bauermann Tested-by: Thiago Jung Bauermann Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/gcs/.gitignore | 1 + tools/testing/selftests/arm64/gcs/Makefile | 2 +- tools/testing/selftests/arm64/gcs/gcs-locking.c | 200 ++++++++++++++++++++++++ 3 files changed, 202 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore index 5810c4a163d4..0c86f53f68ad 100644 --- a/tools/testing/selftests/arm64/gcs/.gitignore +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -1,2 +1,3 @@ basic-gcs libc-gcs +gcs-locking diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile index a8fdf21e9a47..2173d6275956 100644 --- a/tools/testing/selftests/arm64/gcs/Makefile +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -6,7 +6,7 @@ # nolibc. # -TEST_GEN_PROGS := basic-gcs libc-gcs +TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking LDLIBS+=-lpthread diff --git a/tools/testing/selftests/arm64/gcs/gcs-locking.c b/tools/testing/selftests/arm64/gcs/gcs-locking.c new file mode 100644 index 000000000000..989f75a491b7 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-locking.c @@ -0,0 +1,200 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Limited. + * + * Tests for GCS mode locking. These tests rely on both having GCS + * unconfigured on entry and on the kselftest harness running each + * test in a fork()ed process which will have it's own mode. + */ + +#include + +#include +#include + +#include + +#include "kselftest_harness.h" + +#include "gcs-util.h" + +#define my_syscall2(num, arg1, arg2) \ +({ \ + register long _num __asm__ ("x8") = (num); \ + register long _arg1 __asm__ ("x0") = (long)(arg1); \ + register long _arg2 __asm__ ("x1") = (long)(arg2); \ + register long _arg3 __asm__ ("x2") = 0; \ + register long _arg4 __asm__ ("x3") = 0; \ + register long _arg5 __asm__ ("x4") = 0; \ + \ + __asm__ volatile ( \ + "svc #0\n" \ + : "=r"(_arg1) \ + : "r"(_arg1), "r"(_arg2), \ + "r"(_arg3), "r"(_arg4), \ + "r"(_arg5), "r"(_num) \ + : "memory", "cc" \ + ); \ + _arg1; \ +}) + +/* No mode bits are rejected for locking */ +TEST(lock_all_modes) +{ + int ret; + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, ULONG_MAX, 0, 0, 0); + ASSERT_EQ(ret, 0); +} + +FIXTURE(valid_modes) +{ +}; + +FIXTURE_VARIANT(valid_modes) +{ + unsigned long mode; +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable) +{ + .mode = PR_SHADOW_STACK_ENABLE, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_write) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_push) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_PUSH, +}; + +FIXTURE_VARIANT_ADD(valid_modes, enable_write_push) +{ + .mode = PR_SHADOW_STACK_ENABLE | PR_SHADOW_STACK_WRITE | + PR_SHADOW_STACK_PUSH, +}; + +FIXTURE_SETUP(valid_modes) +{ +} + +FIXTURE_TEARDOWN(valid_modes) +{ +} + +/* We can set the mode at all */ +TEST_F(valid_modes, set) +{ + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + _exit(0); +} + +/* Enabling, locking then disabling is rejected */ +TEST_F(valid_modes, enable_lock_disable) +{ + unsigned long mode; + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, 0); + ASSERT_EQ(ret, -EBUSY); + + _exit(0); +} + +/* Locking then enabling is rejected */ +TEST_F(valid_modes, lock_enable) +{ + unsigned long mode; + int ret; + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, -EBUSY); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, 0); + + _exit(0); +} + +/* Locking then changing other modes is fine */ +TEST_F(valid_modes, lock_enable_disable_others) +{ + unsigned long mode; + int ret; + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + ret = prctl(PR_LOCK_SHADOW_STACK_STATUS, variant->mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + PR_SHADOW_STACK_ALL_MODES); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, PR_SHADOW_STACK_ALL_MODES); + + + ret = my_syscall2(__NR_prctl, PR_SET_SHADOW_STACK_STATUS, + variant->mode); + ASSERT_EQ(ret, 0); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + ASSERT_EQ(ret, 0); + ASSERT_EQ(mode, variant->mode); + + _exit(0); +} + +int main(int argc, char **argv) +{ + unsigned long mode; + int ret; + + if (!(getauxval(AT_HWCAP) & HWCAP_GCS)) + ksft_exit_skip("SKIP GCS not supported\n"); + + ret = prctl(PR_GET_SHADOW_STACK_STATUS, &mode, 0, 0, 0); + if (ret) { + ksft_print_msg("Failed to read GCS state: %d\n", ret); + return EXIT_FAILURE; + } + + if (mode & PR_SHADOW_STACK_ENABLE) { + ksft_print_msg("GCS was enabled, test unsupported\n"); + return KSFT_SKIP; + } + + return test_harness_run(argc, argv); +} From patchwork Tue Oct 1 22:59:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832094 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D8E81D0413; Tue, 1 Oct 2024 23:07:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727824022; cv=none; b=MUb91v6ZPWuAzv1kgqHghaweC0MZsqML8imo3/IsrUIdRCYkb1Cq7IDmvOPeOUIHGsmAoIpx5Kn1Q/JAqBV6C7WSDnW7PjVu9AdKL54yk7DBkUpL9MYTIMwQ9EoLsQnUzzwSufrrqS20GtoDd3TsgeSDyvWvgKR66M/lTuDtX9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727824022; c=relaxed/simple; bh=y2JDbrLAwwW8QwfmD140EDad9A018uPPmRKANTo668U=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Iav+xh2skxROXv7t7woPc1FJXVZwSVYp0msjMB420a64vjyzGMqGjvlYW7F6dyEAhLzOWdI+pJHatSr5gvu+h+S5IfgDWcHuuBIHfXbfZ7aLHpNq/3MbUPvZ4bCbKoQ6TUPZRSvb0SQyDGh9vsIw18nQ0p18YJhEs/I/C/xSoIY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nozd5hPc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nozd5hPc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA355C4CEC6; Tue, 1 Oct 2024 23:06:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727824022; bh=y2JDbrLAwwW8QwfmD140EDad9A018uPPmRKANTo668U=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nozd5hPcT6FL+5TdSKS+gT6iveAYwbb8gEl2Rn3kEZkSpRwYuDbaDNCi4Hy8QNhxA yuiAqGgyhXVgpl4CeNS4XF7LJRv3S6BmdRij5P1oosq1PqxAHXoh26ZhO7CXr2H88h kItDRzX5DNF0UkWArKNe/j8YlfOwVsrdoe8hsZkhcLnpXRx8+y1Qf0AqSukDz6QHzS zwQ50gAR3Z2aBxG77QW08H9fjwlf/nOpoME7GF+NwwYs9RTW9gWxZEMNjbYGCv9Yt/ xm4WqHBTw3S3nz8wdInLadPu95t73UtlMipca9iXcBmtcv5uCpioIS+2u+ZI+hfvxQ GMAnz8eJS05sQ== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:17 +0100 Subject: [PATCH v13 38/40] kselftest/arm64: Add a GCS stress test Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-38-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=21256; i=broonie@kernel.org; h=from:subject:message-id; bh=y2JDbrLAwwW8QwfmD140EDad9A018uPPmRKANTo668U=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7hmnxCWdb1zqbs2xx4abtsmcUvLZCrVdBcfUl1 ict8qtKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+4QAKCRAk1otyXVSH0MNHB/ sHXfp1r/TQZrYOp5s01GLwIk0xJIzqyUTV5H/uZCC9xnb+lD6j2D8jPz1huQkAjtnz+1ovI1wP9I3r SHIPks4Z5XxTuf8vmPeVyke9r4VjujXML6y1+FFFhS3onFu/o6dPz+XiLzM0mPojbbxK9xJKgEeNin XyU/e6Wc5JFuRiXQDb3KdJtsoj5F4UtOD+e9vonAczCzWOllpaoH7yGeYVuj8xLcZQS9Vd6Q+P5YAH YgN+2ZCtDSNT32PcyJDsRIGKxPwrhmy72v4UGdcAioRDXvw018H1sACMola6MVZl0E+CCVgR/QIihz PQl4GRRw8AQglIm7mU9fXkKm1qu//j X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Add a stress test which runs one more process than we have CPUs spinning through a very recursive function with frequent syscalls immediately prior to return and signals being injected every 100ms. The goal is to flag up any scheduling related issues, for example failure to ensure that barriers are inserted when moving a GCS using task to another CPU. The test runs for a configurable amount of time, defaulting to 10 seconds. Reviewed-by: Thiago Jung Bauermann Tested-by: Thiago Jung Bauermann Signed-off-by: Mark Brown --- tools/testing/selftests/arm64/gcs/.gitignore | 2 + tools/testing/selftests/arm64/gcs/Makefile | 6 +- tools/testing/selftests/arm64/gcs/asm-offsets.h | 0 .../selftests/arm64/gcs/gcs-stress-thread.S | 311 ++++++++++++ tools/testing/selftests/arm64/gcs/gcs-stress.c | 530 +++++++++++++++++++++ 5 files changed, 848 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/gcs/.gitignore b/tools/testing/selftests/arm64/gcs/.gitignore index 0c86f53f68ad..1e8d1f6b27f2 100644 --- a/tools/testing/selftests/arm64/gcs/.gitignore +++ b/tools/testing/selftests/arm64/gcs/.gitignore @@ -1,3 +1,5 @@ basic-gcs libc-gcs gcs-locking +gcs-stress +gcs-stress-thread diff --git a/tools/testing/selftests/arm64/gcs/Makefile b/tools/testing/selftests/arm64/gcs/Makefile index 2173d6275956..d8b06ca51e22 100644 --- a/tools/testing/selftests/arm64/gcs/Makefile +++ b/tools/testing/selftests/arm64/gcs/Makefile @@ -6,7 +6,8 @@ # nolibc. # -TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking +TEST_GEN_PROGS := basic-gcs libc-gcs gcs-locking gcs-stress +TEST_GEN_PROGS_EXTENDED := gcs-stress-thread LDLIBS+=-lpthread @@ -18,3 +19,6 @@ $(OUTPUT)/basic-gcs: basic-gcs.c -I../../../../../usr/include \ -std=gnu99 -I../.. -g \ -ffreestanding -Wall $^ -o $@ -lgcc + +$(OUTPUT)/gcs-stress-thread: gcs-stress-thread.S + $(CC) -nostdlib $^ -o $@ diff --git a/tools/testing/selftests/arm64/gcs/asm-offsets.h b/tools/testing/selftests/arm64/gcs/asm-offsets.h new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S b/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S new file mode 100644 index 000000000000..b88b25217da5 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-stress-thread.S @@ -0,0 +1,311 @@ +// Program that loops for ever doing lots of recursions and system calls, +// intended to be used as part of a stress test for GCS context switching. +// +// Copyright 2015-2023 Arm Ltd + +#include + +#define sa_sz 32 +#define sa_flags 8 +#define sa_handler 0 +#define sa_mask_sz 8 + +#define si_code 8 + +#define SIGINT 2 +#define SIGABRT 6 +#define SIGUSR1 10 +#define SIGSEGV 11 +#define SIGUSR2 12 +#define SIGTERM 15 +#define SEGV_CPERR 10 + +#define SA_NODEFER 1073741824 +#define SA_SIGINFO 4 +#define ucontext_regs 184 + +#define PR_SET_SHADOW_STACK_STATUS 75 +# define PR_SHADOW_STACK_ENABLE (1UL << 0) + +#define GCSPR_EL0 S3_3_C2_C5_1 + +.macro function name + .macro endfunction + .type \name, @function + .purgem endfunction + .endm +\name: +.endm + +// Print a single character x0 to stdout +// Clobbers x0-x2,x8 +function putc + str x0, [sp, #-16]! + + mov x0, #1 // STDOUT_FILENO + mov x1, sp + mov x2, #1 + mov x8, #__NR_write + svc #0 + + add sp, sp, #16 + ret +endfunction +.globl putc + +// Print a NUL-terminated string starting at address x0 to stdout +// Clobbers x0-x3,x8 +function puts + mov x1, x0 + + mov x2, #0 +0: ldrb w3, [x0], #1 + cbz w3, 1f + add x2, x2, #1 + b 0b + +1: mov w0, #1 // STDOUT_FILENO + mov x8, #__NR_write + svc #0 + + ret +endfunction +.globl puts + +// Utility macro to print a literal string +// Clobbers x0-x4,x8 +.macro puts string + .pushsection .rodata.str1.1, "aMS", @progbits, 1 +.L__puts_literal\@: .string "\string" + .popsection + + ldr x0, =.L__puts_literal\@ + bl puts +.endm + +// Print an unsigned decimal number x0 to stdout +// Clobbers x0-x4,x8 +function putdec + mov x1, sp + str x30, [sp, #-32]! // Result can't be > 20 digits + + mov x2, #0 + strb w2, [x1, #-1]! // Write the NUL terminator + + mov x2, #10 +0: udiv x3, x0, x2 // div-mod loop to generate the digits + msub x0, x3, x2, x0 + add w0, w0, #'0' + strb w0, [x1, #-1]! + mov x0, x3 + cbnz x3, 0b + + ldrb w0, [x1] + cbnz w0, 1f + mov w0, #'0' // Print "0" for 0, not "" + strb w0, [x1, #-1]! + +1: mov x0, x1 + bl puts + + ldr x30, [sp], #32 + ret +endfunction +.globl putdec + +// Print an unsigned decimal number x0 to stdout, followed by a newline +// Clobbers x0-x5,x8 +function putdecn + mov x5, x30 + + bl putdec + mov x0, #'\n' + bl putc + + ret x5 +endfunction +.globl putdecn + +// Fill x1 bytes starting at x0 with 0. +// Clobbers x1, x2. +function memclr + mov w2, #0 +endfunction +.globl memclr + // fall through to memfill + +// Trivial memory fill: fill x1 bytes starting at address x0 with byte w2 +// Clobbers x1 +function memfill + cmp x1, #0 + b.eq 1f + +0: strb w2, [x0], #1 + subs x1, x1, #1 + b.ne 0b + +1: ret +endfunction +.globl memfill + +// w0: signal number +// x1: sa_action +// w2: sa_flags +// Clobbers x0-x6,x8 +function setsignal + str x30, [sp, #-((sa_sz + 15) / 16 * 16 + 16)]! + + mov w4, w0 + mov x5, x1 + mov w6, w2 + + add x0, sp, #16 + mov x1, #sa_sz + bl memclr + + mov w0, w4 + add x1, sp, #16 + str w6, [x1, #sa_flags] + str x5, [x1, #sa_handler] + mov x2, #0 + mov x3, #sa_mask_sz + mov x8, #__NR_rt_sigaction + svc #0 + + cbz w0, 1f + + puts "sigaction failure\n" + b abort + +1: ldr x30, [sp], #((sa_sz + 15) / 16 * 16 + 16) + ret +endfunction + + +function tickle_handler + // Perhaps collect GCSPR_EL0 here in future? + ret +endfunction + +function terminate_handler + mov w21, w0 + mov x20, x2 + + puts "Terminated by signal " + mov w0, w21 + bl putdec + puts ", no error\n" + + mov x0, #0 + mov x8, #__NR_exit + svc #0 +endfunction + +function segv_handler + // stash the siginfo_t * + mov x20, x1 + + // Disable GCS, we don't want additional faults logging things + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x1, xzr + mov x2, xzr + mov x3, xzr + mov x4, xzr + mov x5, xzr + mov x8, #__NR_prctl + svc #0 + + puts "Got SIGSEGV code " + + ldr x21, [x20, #si_code] + mov x0, x21 + bl putdec + + // GCS faults should have si_code SEGV_CPERR + cmp x21, #SEGV_CPERR + bne 1f + + puts " (GCS violation)" +1: + mov x0, '\n' + bl putc + b abort +endfunction + +// Recurse x20 times +.macro recurse id +function recurse\id + stp x29, x30, [sp, #-16]! + mov x29, sp + + cmp x20, 0 + beq 1f + sub x20, x20, 1 + bl recurse\id + +1: + ldp x29, x30, [sp], #16 + + // Do a syscall immediately prior to returning to try to provoke + // scheduling and migration at a point where coherency issues + // might trigger. + mov x8, #__NR_getpid + svc #0 + + ret +endfunction +.endm + +// Generate and use two copies so we're changing the GCS contents +recurse 1 +recurse 2 + +.globl _start +function _start + // Run with GCS + mov x0, PR_SET_SHADOW_STACK_STATUS + mov x1, PR_SHADOW_STACK_ENABLE + mov x2, xzr + mov x3, xzr + mov x4, xzr + mov x5, xzr + mov x8, #__NR_prctl + svc #0 + cbz x0, 1f + puts "Failed to enable GCS\n" + b abort +1: + + mov w0, #SIGTERM + adr x1, terminate_handler + mov w2, #SA_SIGINFO + bl setsignal + + mov w0, #SIGUSR1 + adr x1, tickle_handler + mov w2, #SA_SIGINFO + orr w2, w2, #SA_NODEFER + bl setsignal + + mov w0, #SIGSEGV + adr x1, segv_handler + mov w2, #SA_SIGINFO + orr w2, w2, #SA_NODEFER + bl setsignal + + puts "Running\n" + +loop: + // Small recursion depth so we're frequently flipping between + // the two recursors and changing what's on the stack + mov x20, #5 + bl recurse1 + mov x20, #5 + bl recurse2 + b loop +endfunction + +abort: + mov x0, #255 + mov x8, #__NR_exit + svc #0 diff --git a/tools/testing/selftests/arm64/gcs/gcs-stress.c b/tools/testing/selftests/arm64/gcs/gcs-stress.c new file mode 100644 index 000000000000..bdec7ee8cfd5 --- /dev/null +++ b/tools/testing/selftests/arm64/gcs/gcs-stress.c @@ -0,0 +1,530 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2022-3 ARM Limited. + */ + +#define _GNU_SOURCE +#define _POSIX_C_SOURCE 199309L + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../kselftest.h" + +struct child_data { + char *name, *output; + pid_t pid; + int stdout; + bool output_seen; + bool exited; + int exit_status; + int exit_signal; +}; + +static int epoll_fd; +static struct child_data *children; +static struct epoll_event *evs; +static int tests; +static int num_children; +static bool terminate; + +static int startup_pipe[2]; + +static int num_processors(void) +{ + long nproc = sysconf(_SC_NPROCESSORS_CONF); + if (nproc < 0) { + perror("Unable to read number of processors\n"); + exit(EXIT_FAILURE); + } + + return nproc; +} + +static void start_thread(struct child_data *child) +{ + int ret, pipefd[2], i; + struct epoll_event ev; + + ret = pipe(pipefd); + if (ret != 0) + ksft_exit_fail_msg("Failed to create stdout pipe: %s (%d)\n", + strerror(errno), errno); + + child->pid = fork(); + if (child->pid == -1) + ksft_exit_fail_msg("fork() failed: %s (%d)\n", + strerror(errno), errno); + + if (!child->pid) { + /* + * In child, replace stdout with the pipe, errors to + * stderr from here as kselftest prints to stdout. + */ + ret = dup2(pipefd[1], 1); + if (ret == -1) { + fprintf(stderr, "dup2() %d\n", errno); + exit(EXIT_FAILURE); + } + + /* + * Duplicate the read side of the startup pipe to + * FD 3 so we can close everything else. + */ + ret = dup2(startup_pipe[0], 3); + if (ret == -1) { + fprintf(stderr, "dup2() %d\n", errno); + exit(EXIT_FAILURE); + } + + /* + * Very dumb mechanism to clean open FDs other than + * stdio. We don't want O_CLOEXEC for the pipes... + */ + for (i = 4; i < 8192; i++) + close(i); + + /* + * Read from the startup pipe, there should be no data + * and we should block until it is closed. We just + * carry on on error since this isn't super critical. + */ + ret = read(3, &i, sizeof(i)); + if (ret < 0) + fprintf(stderr, "read(startp pipe) failed: %s (%d)\n", + strerror(errno), errno); + if (ret > 0) + fprintf(stderr, "%d bytes of data on startup pipe\n", + ret); + close(3); + + ret = execl("gcs-stress-thread", "gcs-stress-thread", NULL); + fprintf(stderr, "execl(gcs-stress-thread) failed: %d (%s)\n", + errno, strerror(errno)); + + exit(EXIT_FAILURE); + } else { + /* + * In parent, remember the child and close our copy of the + * write side of stdout. + */ + close(pipefd[1]); + child->stdout = pipefd[0]; + child->output = NULL; + child->exited = false; + child->output_seen = false; + + ev.events = EPOLLIN | EPOLLHUP; + ev.data.ptr = child; + + ret = asprintf(&child->name, "Thread-%d", child->pid); + if (ret == -1) + ksft_exit_fail_msg("asprintf() failed\n"); + + ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, child->stdout, &ev); + if (ret < 0) { + ksft_exit_fail_msg("%s EPOLL_CTL_ADD failed: %s (%d)\n", + child->name, strerror(errno), errno); + } + } + + ksft_print_msg("Started %s\n", child->name); + num_children++; +} + +static bool child_output_read(struct child_data *child) +{ + char read_data[1024]; + char work[1024]; + int ret, len, cur_work, cur_read; + + ret = read(child->stdout, read_data, sizeof(read_data)); + if (ret < 0) { + if (errno == EINTR) + return true; + + ksft_print_msg("%s: read() failed: %s (%d)\n", + child->name, strerror(errno), + errno); + return false; + } + len = ret; + + child->output_seen = true; + + /* Pick up any partial read */ + if (child->output) { + strncpy(work, child->output, sizeof(work) - 1); + cur_work = strnlen(work, sizeof(work)); + free(child->output); + child->output = NULL; + } else { + cur_work = 0; + } + + cur_read = 0; + while (cur_read < len) { + work[cur_work] = read_data[cur_read++]; + + if (work[cur_work] == '\n') { + work[cur_work] = '\0'; + ksft_print_msg("%s: %s\n", child->name, work); + cur_work = 0; + } else { + cur_work++; + } + } + + if (cur_work) { + work[cur_work] = '\0'; + ret = asprintf(&child->output, "%s", work); + if (ret == -1) + ksft_exit_fail_msg("Out of memory\n"); + } + + return false; +} + +static void child_output(struct child_data *child, uint32_t events, + bool flush) +{ + bool read_more; + + if (events & EPOLLIN) { + do { + read_more = child_output_read(child); + } while (read_more); + } + + if (events & EPOLLHUP) { + close(child->stdout); + child->stdout = -1; + flush = true; + } + + if (flush && child->output) { + ksft_print_msg("%s: %s\n", child->name, child->output); + free(child->output); + child->output = NULL; + } +} + +static void child_tickle(struct child_data *child) +{ + if (child->output_seen && !child->exited) + kill(child->pid, SIGUSR1); +} + +static void child_stop(struct child_data *child) +{ + if (!child->exited) + kill(child->pid, SIGTERM); +} + +static void child_cleanup(struct child_data *child) +{ + pid_t ret; + int status; + bool fail = false; + + if (!child->exited) { + do { + ret = waitpid(child->pid, &status, 0); + if (ret == -1 && errno == EINTR) + continue; + + if (ret == -1) { + ksft_print_msg("waitpid(%d) failed: %s (%d)\n", + child->pid, strerror(errno), + errno); + fail = true; + break; + } + + if (WIFEXITED(status)) { + child->exit_status = WEXITSTATUS(status); + child->exited = true; + } + + if (WIFSIGNALED(status)) { + child->exit_signal = WTERMSIG(status); + ksft_print_msg("%s: Exited due to signal %d\n", + child->name); + fail = true; + child->exited = true; + } + } while (!child->exited); + } + + if (!child->output_seen) { + ksft_print_msg("%s no output seen\n", child->name); + fail = true; + } + + if (child->exit_status != 0) { + ksft_print_msg("%s exited with error code %d\n", + child->name, child->exit_status); + fail = true; + } + + ksft_test_result(!fail, "%s\n", child->name); +} + +static void handle_child_signal(int sig, siginfo_t *info, void *context) +{ + int i; + bool found = false; + + for (i = 0; i < num_children; i++) { + if (children[i].pid == info->si_pid) { + children[i].exited = true; + children[i].exit_status = info->si_status; + found = true; + break; + } + } + + if (!found) + ksft_print_msg("SIGCHLD for unknown PID %d with status %d\n", + info->si_pid, info->si_status); +} + +static void handle_exit_signal(int sig, siginfo_t *info, void *context) +{ + int i; + + /* If we're already exiting then don't signal again */ + if (terminate) + return; + + ksft_print_msg("Got signal, exiting...\n"); + + terminate = true; + + /* + * This should be redundant, the main loop should clean up + * after us, but for safety stop everything we can here. + */ + for (i = 0; i < num_children; i++) + child_stop(&children[i]); +} + +/* Handle any pending output without blocking */ +static void drain_output(bool flush) +{ + int ret = 1; + int i; + + while (ret > 0) { + ret = epoll_wait(epoll_fd, evs, tests, 0); + if (ret < 0) { + if (errno == EINTR) + continue; + ksft_print_msg("epoll_wait() failed: %s (%d)\n", + strerror(errno), errno); + } + + for (i = 0; i < ret; i++) + child_output(evs[i].data.ptr, evs[i].events, flush); + } +} + +static const struct option options[] = { + { "timeout", required_argument, NULL, 't' }, + { } +}; + +int main(int argc, char **argv) +{ + int seen_children; + bool all_children_started = false; + int gcs_threads; + int timeout = 10; + int ret, cpus, i, c; + struct sigaction sa; + + while ((c = getopt_long(argc, argv, "t:", options, NULL)) != -1) { + switch (c) { + case 't': + ret = sscanf(optarg, "%d", &timeout); + if (ret != 1) + ksft_exit_fail_msg("Failed to parse timeout %s\n", + optarg); + break; + default: + ksft_exit_fail_msg("Unknown argument\n"); + } + } + + cpus = num_processors(); + tests = 0; + + if (getauxval(AT_HWCAP) & HWCAP_GCS) { + /* One extra thread, trying to trigger migrations */ + gcs_threads = cpus + 1; + tests += gcs_threads; + } else { + gcs_threads = 0; + } + + ksft_print_header(); + ksft_set_plan(tests); + + ksft_print_msg("%d CPUs, %d GCS threads\n", + cpus, gcs_threads); + + if (!tests) + ksft_exit_skip("No tests scheduled\n"); + + if (timeout > 0) + ksft_print_msg("Will run for %ds\n", timeout); + else + ksft_print_msg("Will run until terminated\n"); + + children = calloc(sizeof(*children), tests); + if (!children) + ksft_exit_fail_msg("Unable to allocate child data\n"); + + ret = epoll_create1(EPOLL_CLOEXEC); + if (ret < 0) + ksft_exit_fail_msg("epoll_create1() failed: %s (%d)\n", + strerror(errno), ret); + epoll_fd = ret; + + /* Create a pipe which children will block on before execing */ + ret = pipe(startup_pipe); + if (ret != 0) + ksft_exit_fail_msg("Failed to create startup pipe: %s (%d)\n", + strerror(errno), errno); + + /* Get signal handers ready before we start any children */ + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = handle_exit_signal; + sa.sa_flags = SA_RESTART | SA_SIGINFO; + sigemptyset(&sa.sa_mask); + ret = sigaction(SIGINT, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGINT handler: %s (%d)\n", + strerror(errno), errno); + ret = sigaction(SIGTERM, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGTERM handler: %s (%d)\n", + strerror(errno), errno); + sa.sa_sigaction = handle_child_signal; + ret = sigaction(SIGCHLD, &sa, NULL); + if (ret < 0) + ksft_print_msg("Failed to install SIGCHLD handler: %s (%d)\n", + strerror(errno), errno); + + evs = calloc(tests, sizeof(*evs)); + if (!evs) + ksft_exit_fail_msg("Failed to allocated %d epoll events\n", + tests); + + for (i = 0; i < gcs_threads; i++) + start_thread(&children[i]); + + /* + * All children started, close the startup pipe and let them + * run. + */ + close(startup_pipe[0]); + close(startup_pipe[1]); + + timeout *= 10; + for (;;) { + /* Did we get a signal asking us to exit? */ + if (terminate) + break; + + /* + * Timeout is counted in 100ms with no output, the + * tests print during startup then are silent when + * running so this should ensure they all ran enough + * to install the signal handler, this is especially + * useful in emulation where we will both be slow and + * likely to have a large set of VLs. + */ + ret = epoll_wait(epoll_fd, evs, tests, 100); + if (ret < 0) { + if (errno == EINTR) + continue; + ksft_exit_fail_msg("epoll_wait() failed: %s (%d)\n", + strerror(errno), errno); + } + + /* Output? */ + if (ret > 0) { + for (i = 0; i < ret; i++) { + child_output(evs[i].data.ptr, evs[i].events, + false); + } + continue; + } + + /* Otherwise epoll_wait() timed out */ + + /* + * If the child processes have not produced output they + * aren't actually running the tests yet. + */ + if (!all_children_started) { + seen_children = 0; + + for (i = 0; i < num_children; i++) + if (children[i].output_seen || + children[i].exited) + seen_children++; + + if (seen_children != num_children) { + ksft_print_msg("Waiting for %d children\n", + num_children - seen_children); + continue; + } + + all_children_started = true; + } + + ksft_print_msg("Sending signals, timeout remaining: %d00ms\n", + timeout); + + for (i = 0; i < num_children; i++) + child_tickle(&children[i]); + + /* Negative timeout means run indefinitely */ + if (timeout < 0) + continue; + if (--timeout == 0) + break; + } + + ksft_print_msg("Finishing up...\n"); + terminate = true; + + for (i = 0; i < tests; i++) + child_stop(&children[i]); + + drain_output(false); + + for (i = 0; i < tests; i++) + child_cleanup(&children[i]); + + drain_output(true); + + ksft_finished(); +} From patchwork Tue Oct 1 22:59:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 832093 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 015911CEEA2; Tue, 1 Oct 2024 23:07:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727824042; cv=none; b=j3UvTOomEGxxSSLyJR9MO5TT6fZLKUIRy+Drh6z9IM/XlkMajz2OeeFPthah6kdbfzROyznzTUq9pOSTKyJhIonv8wmrJQovLjMEgUFN7PcEy38H9vi/iYSDSuvM5WRKqT/FQosNhpQl3nvfs99WV86HIJktOCI5cuwh/eaahGI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727824042; c=relaxed/simple; bh=j6IIdqnHFeQum6z66KX3fHgl2RzDdY03fDwrd4bo9J4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Rjf0peIDpWcVtoyBAv4dsblFRtoUGL4da306FGSSm+U6IAqmYsi+C6tndyIJu8gBoBkcKy/3jCLGLnDgB/5ftuUFTSR4+LNYy9ExUHdjDmuScM1JmGYSbuh9sNlV1S1dISGiZyZosW8Pp4grLTAS0IVxCY16j4IdWY5gY7ZJT+g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MlMuheBU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MlMuheBU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E2416C4CEC6; Tue, 1 Oct 2024 23:07:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727824041; bh=j6IIdqnHFeQum6z66KX3fHgl2RzDdY03fDwrd4bo9J4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=MlMuheBUz9TDrNZYXlk1//cCagyMhWpLR/SGxdIhq25FU+GmsM9wO7eQvhmd0z5nE 83lXiVJxTeqY5wHk8bzg6jShROnWKZ+lB6WEmLzR1fjQKCqx2ZVf9Cv8cFnFILwoiA F/1RU4x4fX9e6v/AIlXDtOx/CrYpVJNB99SHnzBBZE8Kg7Wgb7LCN8VJ311tPLDfz2 etBgdgfLrtXLe/uFUkHHHOwYXUgCE7taIIEUJv7aGC9IqZKm6Rdx6AAuJ6gaeGMwAA YX9XTcwUh9AMMrlMQRkCsGpUNfuR2D1s6RYLyjvTtWewz3gL6ch+5sTU0a7Tsv5Wlh a07Ozo2sKVw2w== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:19 +0100 Subject: [PATCH v13 40/40] KVM: selftests: arm64: Add GCS registers to get-reg-list Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-40-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=2416; i=broonie@kernel.org; h=from:subject:message-id; bh=j6IIdqnHFeQum6z66KX3fHgl2RzDdY03fDwrd4bo9J4=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7jXlZsyr1+7roR/ZRU6a6ost0JSw7SFgAbAQjO xwJbQc+JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+4wAKCRAk1otyXVSH0I//B/ 4vOMUsdK63kNQyVzSZDRn7qmBg3F1T4vHXcC4pbPIxVJCkLTtjsMmQyQC4O5xFSsh3MdZ2n/zkY+RB OGjrdX1UWguJggO+skYEssBRt3LCT1U4DkVsT7HBHEHvwbwyiNDwJvke6m7i/vrm8OvLEunaT7mku4 hkGqiVlSrXxsje+lpEJews5xOhtfs+x0tz5eRbsj8EYyRPNpU8nV30/XAF+lQSfam5chnnMBTF1LH/ adX/YvsJtRLVtJVhfHJtoj69nD/7FJekPVdjsN5+81ApUOobQNwSxHvsE6bbVmRm+KxZl9R5lWIQdj yXgi4EeOV0ewNakXwRY7QF3rT0VO6A X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB GCS adds new registers GCSCR_EL1, GCSCRE0_EL1, GCSPR_EL1 and GCSPR_EL0. Add these to those validated by get-reg-list. Reviewed-by: Thiago Jung Bauermann Signed-off-by: Mark Brown --- tools/testing/selftests/kvm/aarch64/get-reg-list.c | 28 ++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tools/testing/selftests/kvm/aarch64/get-reg-list.c b/tools/testing/selftests/kvm/aarch64/get-reg-list.c index d43fb3f49050..c17451069a15 100644 --- a/tools/testing/selftests/kvm/aarch64/get-reg-list.c +++ b/tools/testing/selftests/kvm/aarch64/get-reg-list.c @@ -29,6 +29,24 @@ static struct feature_id_reg feat_id_regs[] = { 0, 1 }, + { + ARM64_SYS_REG(3, 0, 2, 5, 0), /* GCSCR_EL1 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 44, + 1 + }, + { + ARM64_SYS_REG(3, 0, 2, 5, 1), /* GCSPR_EL1 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 44, + 1 + }, + { + ARM64_SYS_REG(3, 0, 2, 5, 2), /* GCSCRE0_EL1 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 44, + 1 + }, { ARM64_SYS_REG(3, 0, 10, 2, 2), /* PIRE0_EL1 */ ARM64_SYS_REG(3, 0, 0, 7, 3), /* ID_AA64MMFR3_EL1 */ @@ -52,6 +70,12 @@ static struct feature_id_reg feat_id_regs[] = { ARM64_SYS_REG(3, 0, 0, 7, 3), /* ID_AA64MMFR3_EL1 */ 16, 1 + }, + { + ARM64_SYS_REG(3, 3, 2, 5, 1), /* GCSPR_EL0 */ + ARM64_SYS_REG(3, 0, 0, 4, 1), /* ID_AA64PFR1_EL1 */ + 44, + 1 } }; @@ -472,6 +496,9 @@ static __u64 base_regs[] = { ARM64_SYS_REG(3, 0, 2, 0, 1), /* TTBR1_EL1 */ ARM64_SYS_REG(3, 0, 2, 0, 2), /* TCR_EL1 */ ARM64_SYS_REG(3, 0, 2, 0, 3), /* TCR2_EL1 */ + ARM64_SYS_REG(3, 0, 2, 5, 0), /* GCSCR_EL1 */ + ARM64_SYS_REG(3, 0, 2, 5, 1), /* GCSPR_EL1 */ + ARM64_SYS_REG(3, 0, 2, 5, 2), /* GCSCRE0_EL1 */ ARM64_SYS_REG(3, 0, 5, 1, 0), /* AFSR0_EL1 */ ARM64_SYS_REG(3, 0, 5, 1, 1), /* AFSR1_EL1 */ ARM64_SYS_REG(3, 0, 5, 2, 0), /* ESR_EL1 */ @@ -488,6 +515,7 @@ static __u64 base_regs[] = { ARM64_SYS_REG(3, 0, 13, 0, 4), /* TPIDR_EL1 */ ARM64_SYS_REG(3, 0, 14, 1, 0), /* CNTKCTL_EL1 */ ARM64_SYS_REG(3, 2, 0, 0, 0), /* CSSELR_EL1 */ + ARM64_SYS_REG(3, 3, 2, 5, 1), /* GCSPR_EL0 */ ARM64_SYS_REG(3, 3, 10, 2, 4), /* POR_EL0 */ ARM64_SYS_REG(3, 3, 13, 0, 2), /* TPIDR_EL0 */ ARM64_SYS_REG(3, 3, 13, 0, 3), /* TPIDRRO_EL0 */