From patchwork Fri Feb 9 06:40:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_M=C3=BCllner?= X-Patchwork-Id: 771684 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88BBF657BE for ; Fri, 9 Feb 2024 06:41:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707460862; cv=none; b=AGFv+NwGEPwn3CI+/EsT8SzYlb/z9TsYy4+Lhf4veT7iR8yxFkaQqR3RIP7BBGKMhb/B1Y7Pf/S84vs/TDp39MzOMj4boHn3ie9j5d+HRTBb9JtYePSuSwWjy+kAvSToWApXPAO4o3fsb6V1LeYFAcf/e7zZsqZ2+LZut3Gx0C8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707460862; c=relaxed/simple; bh=ifyOm/Jff2D4hC+7lPpykba0FTTiyuZ30Z/Vma6f/bw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ahopSIar1Lw65ZytlM7f3/5XY5jEEWPkymkGgNHn/GGhDr/oZM46JbvmJpt4H//STUBrpEcMAwYUR+FXDm+E+WIv5isXuNBAKS7WtD3GHh8W3g/vdG6y3IQO3O55BnmDsBVC1HU5jTby/SqRRFFwM/jAZL/4JMdxpotUYu0MNs0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=vrull.eu; spf=pass smtp.mailfrom=vrull.eu; dkim=pass (2048-bit key) header.d=vrull.eu header.i=@vrull.eu header.b=mJ65RxN3; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=vrull.eu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vrull.eu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vrull.eu header.i=@vrull.eu header.b="mJ65RxN3" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-a3810e975aaso66933166b.3 for ; Thu, 08 Feb 2024 22:41:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1707460859; x=1708065659; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ka0BwVIa89CrbsNMJkPh2Vgcr3mcWFHipPKQFS87pYA=; b=mJ65RxN3sYP77x4R1N9FyTwmjQpcX+8J6BJ1K55yW+ueU5YyUCggc205fy85RFGi9q JsgndWapcHw/TGlhWzG2IQwE7CzhYhViPGJFqfUfuS+BkQq1cOYvUIFzzy5Q7K9adp+W bhTMbuBQjrO4qlw8PmgSDEPBrYBdqL/YSArDFgh/ZYxyRNfjYgUMxZXxOy6ZJJovH7mT dIHzWyH90VmBlRK+S60LMPy+knjP7Rp2TQxZKfIIdPV8lkgcC90+C5dfaCUExvlcwgP8 3oKVQxfe0Q6Me5hwkOTHeIotrvSF2YdAp3shyxkF8EmLIXxdFS6VXgGj97NPzbc37ajF g6YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707460859; x=1708065659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ka0BwVIa89CrbsNMJkPh2Vgcr3mcWFHipPKQFS87pYA=; b=WZLl++Xs252kXGWSryHfyNbspgx5uhOK9MpalYp0cWO1cGwMy8NzjD4p8GFKQyZnp/ HEJd/qd+IlsyRfrC0maNwS70ap3vEuhCuPrwV5HrNR6OiPfESyNT3Nenn/w7ScjIVLsV eTMDUMce0sevvpz3/eBDrNSXgAPWGm98zEhRhSj6TrPp/FNXb+9IjTz8sPBpJU55dOh7 oI5/hnuuzHiQmosyNH8pwEkHIjP+z5MG6Pkro3L00BNTlz3eNZiyceYxftu+8fjmIMSF I3z7EUfHvPvo7YuhZYGHixma12AWNhS/UW+nZXTFAzfekGsgqgbQqYJcjNHOEpPvTJdW zpgw== X-Gm-Message-State: AOJu0Yywicj2C5SkGKngrrnldC2xlm6sWVBo6nEqQVcmA7VdoeM8GL1I e2UDwD03LusrENwvQ+8Uzzcv4H2j7Y4dwAOHEVyFTCzWC5MokcnSO0U+xe/MGpo= X-Google-Smtp-Source: AGHT+IFEN03if+6niuHDJ7y6KQ3qH470BeXh3gKz+Sb465eM/2K3LAMO8sCRGXg+1jKiB4FL7oHnjw== X-Received: by 2002:a17:906:2e91:b0:a37:7b4:9538 with SMTP id o17-20020a1709062e9100b00a3707b49538mr530857eji.33.1707460858580; Thu, 08 Feb 2024 22:40:58 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCUGzRsAZwmfQ3VpSld4acgsFVB8Z2BpFxm5NYoupbUP/wDHZUcHWD0xshhR8SzfywjNN/6uiHCGHWB3luBB7eAro95MsHdnxJbi/UsaoOWSi/QX7Yt3BGzNKdp5rZMZI4ZCaJrcG3SSADnGciJhN51oKuauuTPK+/oaFDQEq9fFxnkTdCcgWwxBRc2/qEVCRnTSSXWdN9Dhwxos973APpZ8ybYob1XLGih9Ej/b67MnvWZBSQzE/otQIR76P/XvNjg/eVczYytDfWZXVLUrGDTLltumZbxt+67DG9vbcnDKew0X/chAbTQfYrTYpkLRlj0kwsodyv04SaDJ1gad2bOJTPEQtGLPKz2/YsnmOLREsvkoageVxwSbJhiDF30C7qkgV++HU2WIkmb+wOIuU3Ck3GzZIMla0sFstAz8dXyRNm+yDeUoEflVK0q1lKALfz8tkUBEGc6omFxBsljE/KqQwVHtBdygQ8fNqUfMEWoT03c74tkohgGFw1IsWg7MdAVF9WedboaLk5CYiTXW9RAWxMEBDsDVChXyTdHCz2NF7PPk1QYS+zBnkZyypbUI02tKIE7oorhJtzbg0Vkuslqhe7u5XzLFZLe8WE7GD35XxR1jGkVb1zdSYPBxCzO/3LmeKOWXN12YDTlnDMFgGmKCvKNAs5IipyIg22VgBDA/GW0iNeAax58E0aap/Swu9mxIkfNzrAZOSnJRYq6ZR6wASlI8grPksL2Sabilobn98XkAJ1WuIQ5fOHUl+iRQMsuItpiI724RZwSPd/uKOZnbK/M+PBA+0pYuXqXUK9gz+7NrViKr8a+JhLSAFif+ZnT4CZaNekMrga7Hjfyh9xRRxpJzHHYAXrXnPSDDbJWFZw== Received: from beast.fritz.box (62-178-148-172.cable.dynamic.surfer.at. [62.178.148.172]) by smtp.gmail.com with ESMTPSA id vb9-20020a170907d04900b00a370a76d3a0sm441180ejc.123.2024.02.08.22.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 22:40:58 -0800 (PST) From: =?utf-8?q?Christoph_M=C3=BCllner?= To: linux-riscv@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Jonas Oberhauser , Peter Zijlstra , Palmer Dabbelt , Paul Walmsley , Albert Ou , Andrew Morton , Shuah Khan , Jonathan Corbet , Anup Patel , Philipp Tomsich , Andrew Jones , Guo Ren , Daniel Henrique Barboza , Conor Dooley , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Alan Stern , Will Deacon , Daniel Lustig , Brendan Sweeney , Andrew Waterman , Brendan Sweeney , Andrea Parri , Hans Boehm Cc: =?utf-8?q?Christoph_M=C3=BCllner?= Subject: [RFC PATCH v2 2/6] uapi: prctl: Add new prctl call to set/get the memory consistency model Date: Fri, 9 Feb 2024 07:40:46 +0100 Message-ID: <20240209064050.2746540-3-christoph.muellner@vrull.eu> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240209064050.2746540-1-christoph.muellner@vrull.eu> References: <20240209064050.2746540-1-christoph.muellner@vrull.eu> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch defines a prctl uAPI for switching the active memory consistency model of user-space processes. The implementation follows the way other prctl calls are implemented by disabling them unless arch-specific code provides the relevant macros. Signed-off-by: Christoph Müllner --- .../mm/dynamic-memory-consistency-model.rst | 27 +++++++++++++++++++ include/uapi/linux/prctl.h | 3 +++ kernel/sys.c | 12 +++++++++ 3 files changed, 42 insertions(+) diff --git a/Documentation/mm/dynamic-memory-consistency-model.rst b/Documentation/mm/dynamic-memory-consistency-model.rst index 3117c3d82b2b..1fce855a1fad 100644 --- a/Documentation/mm/dynamic-memory-consistency-model.rst +++ b/Documentation/mm/dynamic-memory-consistency-model.rst @@ -47,3 +47,30 @@ at run-time: * Only switching from a weaker to a stronger model is safe. * The stronger memory model affects all threads of a process, when running in user mode. * Forked processes derive their active memory model from their parents. + +User API via prctl +================== + +Two prctl calls are defined to get/set the active memory consistency model: + +* prctl(PR_GET_MEMORY_CONSISTENCY_MODEL) + + Returns the active memory consistency model for the calling process/thread. + If the architecture does not support dynamic memory consistency models, + then -1 is returned, and errno is set to EINVAL. + +* prctl(PR_SET_MEMORY_CONSISTENCY_MODEL, unsigned long new_model) + + Switches the memory consistency model for the calling process/thread + to the given model. If the architecture does not support dynamic + memory consistency models, or does not support the provided model, or + does not allow to switch to the proveided model then -1 is returned, + and errno is set to EINVAL. + +Supported memory consistency models +=================================== + +This section defines the memory consistency models which are supported +by the prctl interface. + + diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 370ed14b1ae0..579662731eaa 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -306,4 +306,7 @@ struct prctl_mm_map { # define PR_RISCV_V_VSTATE_CTRL_NEXT_MASK 0xc # define PR_RISCV_V_VSTATE_CTRL_MASK 0x1f +#define PR_SET_MEMORY_CONSISTENCY_MODEL 71 +#define PR_GET_MEMORY_CONSISTENCY_MODEL 72 + #endif /* _LINUX_PRCTL_H */ diff --git a/kernel/sys.c b/kernel/sys.c index e219fcfa112d..a1b92a38f889 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -146,6 +146,12 @@ #ifndef RISCV_V_GET_CONTROL # define RISCV_V_GET_CONTROL() (-EINVAL) #endif +#ifndef SET_MEMORY_CONSISTENCY_MODEL +# define SET_MEMORY_CONSISTENCY_MODEL(a) (-EINVAL) +#endif +#ifndef GET_MEMORY_CONSISTENCY_MODEL +# define GET_MEMORY_CONSISTENCY_MODEL() (-EINVAL) +#endif /* * this is where the system-wide overflow UID and GID are defined, for @@ -2743,6 +2749,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, case PR_RISCV_V_GET_CONTROL: error = RISCV_V_GET_CONTROL(); break; + case PR_SET_MEMORY_CONSISTENCY_MODEL: + error = SET_MEMORY_CONSISTENCY_MODEL(arg2); + break; + case PR_GET_MEMORY_CONSISTENCY_MODEL: + error = GET_MEMORY_CONSISTENCY_MODEL(); + break; default: error = -EINVAL; break; From patchwork Fri Feb 9 06:40:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_M=C3=BCllner?= X-Patchwork-Id: 771683 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50A04657D4 for ; Fri, 9 Feb 2024 06:41:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707460865; cv=none; b=DB+X5W77dZb5ZoL4454bMOFfas8/rpWpFiJhT2K7xDw2mhnKuBxuqvdgP+nA/aOfjY4nsy3ZsPCdFuIx3+wy1vkT+J+n0hFmfBm09hV0QeprAxr4jICPM0LVMfrmmpzWnEmaiCi8Bv6WI1fkpjdWEUFvOL408xxqHct+ve7e6KI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707460865; c=relaxed/simple; bh=uU0PO3OZf2iUz0gGhE0y9kQtipSBlOMMES8f3o99Jp4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OVBjuFl9dW0IUjqXE3MBEf0WUtq7baV5+ZJvDSO6YMU4YtRzeJTlWGQ/tEw1BHYs7xORcPNcNAc3uUE4xcBBUiONXGoIuTjnSljrOg/fn2eQPtcCJugc97HVzR4kCswrYcc8laG2MXQYpgeD3wed+iEz8weRIYiRLVlbBSexXMM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=vrull.eu; spf=pass smtp.mailfrom=vrull.eu; dkim=pass (2048-bit key) header.d=vrull.eu header.i=@vrull.eu header.b=EzJmaNkv; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=vrull.eu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vrull.eu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vrull.eu header.i=@vrull.eu header.b="EzJmaNkv" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-a293f2280c7so78994466b.1 for ; Thu, 08 Feb 2024 22:41:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1707460860; x=1708065660; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+Iy+qb8cqxclx2b4u4iSMkolO0A5gDplvFNecdg0ef0=; b=EzJmaNkv78fjYfXOcvcx0CeYhMxaz2ligpdFFkx+tA4xXxpSDcRxzPhLXmzzhOYVrZ XORjP5Gjm366mkfHS4H9t8JHTNio7Af2cHm9acDukzzAm446D+L2t6xnh2k8H3d6A4U9 x543gy5sfddYkFiBFVvUHm+RZbh6hUy6NM9ysNz7y9QkF1DN+Lh0hrXP2ZqufT/YUMZX w3/gpFXVf2RMh+iNTy4rSd2q1qQPwXSjryAJjykvO3WaNTot0mOUJogq2VJGEVH2cD15 IuHQvlO08RirGxzFvKU8dTbmx9a49SzVIc1Wx1PbTc0sPU4sX9pDVRBrCZibq84EkFnk gAyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707460860; x=1708065660; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+Iy+qb8cqxclx2b4u4iSMkolO0A5gDplvFNecdg0ef0=; b=DaDdvKCuhHCqjgKQzIpztUPZRl3SiMpk2G5t+bjYZ7NmqjPxgnwJHGwPY2X3AENvbs r0Xpm5QzwEEwoUhxkKJ93ha//hXB1WjokiMeOlwSLnaP+prShf8o289FymZ9d/uPsnJO 61b8Tmw4RdgN221N9ZHsFkOXn353UgTmBIdX0NjCODDR51t0lW6XddjAH1cEYqNXidTa N5xeX16zEAH5pszPwGyd9HEIp5gLpc5sVdc5OF9A8ACEsbuU1+dZihvPOti6zzfz8JTW w/hc4WyfcLNF+2ZsjY5Z5ieKqDp0T2zwiuJ1bWatpqS7x92XIcznNzL32d2iT4btsic5 Jc7A== X-Gm-Message-State: AOJu0YwANud2HOB6WXMZQK+wYZ/sg0Q5G3/38GsSj+ezb+b/bihViCXc t3bNxGBH2iMrP7gIUMJYvvNk0EKqnd+2DBhPxpNcD2TUb2OC2GYl5caYe+SfvAMFNEdV+C8a21V 9o04= X-Google-Smtp-Source: AGHT+IGWG9nd+Kfmz7RZ7epqow+tvQ3hOS+FSn8n3iqZKNodfZG8lzKrxgzFpIiKDWC28PJLXwIyaw== X-Received: by 2002:a17:906:4a45:b0:a37:625d:397d with SMTP id a5-20020a1709064a4500b00a37625d397dmr485869ejv.44.1707460860511; Thu, 08 Feb 2024 22:41:00 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCX71W5p9A9pSxDne91E1S+4dp8HZwpU0zSjneWfoUp5Vt++iXRw5L1lGgzZko6taQEC94C0rij3mTasjZ0imQ33SelhsJS8vXg4MFZtDCv3a9Gu1ybl3KXUIVa/k6uPIBveTIa7wX2YUYYpaN/AncLrj6jhN7f4uDuH3v0+HXBHBXVavl0bmm2QP2neV+G7RfmU6WvDDb68V6ZsmTsUexb0+hxKOgQTUyOGKVs/ElNaFdvJpmrYOWVlHO/+fINXm02jH5erk/ApvVYwhuWTS83Z1B2DnGQy50tkRqCxt4mvAZptPoEZCqEd6XOTmHSOSzwz3AsaNcBpM4dGGBcNxAsOghL32JGvy6vXlrPeegNpxjHokJdnMSwPOyx7T8mEdpY0CuITCe+uhB2WiyuEJ3NZEysAY/GF2uFXej92BT6tDCuuFJJ4uEwVNTsD9De0/gTCETvzUMtB0D0KUoCuMGitSAdyiKvG0doTkND9n9FXzJ3Nle8XkRSutC6b6qdobtfR0tbHInlRn9BG9rEjmMisBCVeW4ZHYzTUS3iMwDwLxFTyrHfHtpuzOUWPvLrcMUgHOMqGTbJRM+8NNjzOr/cX6QKD5lg3uGrTc4TWWFgxvJM2dVt9n3YpvTQ2QGbVPL6BvDdbUEfkLaq+86GBL0CjzxWC0Kj9KubMYkgcqf6gmMYCY8MPnSvP3UBJU8na4T4zRLNMQtPrpqC+kX3yyoGzBNXpldu4u9aoCrp+KpWj/EZtCX057UdjBZclhwV+K+RRcmRfBL3EInLWnKOO9xfh9FMp9OsQqlp4hK6GphEw5217o/PLI9FDc6/pJl5Jv8xPfNoOB3RtxRkwBx541UsO373RLlu50htB1kQEZIushg== Received: from beast.fritz.box (62-178-148-172.cable.dynamic.surfer.at. [62.178.148.172]) by smtp.gmail.com with ESMTPSA id vb9-20020a170907d04900b00a370a76d3a0sm441180ejc.123.2024.02.08.22.40.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 22:40:59 -0800 (PST) From: =?utf-8?q?Christoph_M=C3=BCllner?= To: linux-riscv@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Jonas Oberhauser , Peter Zijlstra , Palmer Dabbelt , Paul Walmsley , Albert Ou , Andrew Morton , Shuah Khan , Jonathan Corbet , Anup Patel , Philipp Tomsich , Andrew Jones , Guo Ren , Daniel Henrique Barboza , Conor Dooley , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Alan Stern , Will Deacon , Daniel Lustig , Brendan Sweeney , Andrew Waterman , Brendan Sweeney , Andrea Parri , Hans Boehm Cc: =?utf-8?q?Christoph_M=C3=BCllner?= Subject: [RFC PATCH v2 3/6] RISC-V: Enable dynamic memory consistency model support with Ssdtso Date: Fri, 9 Feb 2024 07:40:47 +0100 Message-ID: <20240209064050.2746540-4-christoph.muellner@vrull.eu> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240209064050.2746540-1-christoph.muellner@vrull.eu> References: <20240209064050.2746540-1-christoph.muellner@vrull.eu> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch implements dynamic memory consistency switching on RISC-V using the Ssdtso ISA extension. Ssdtso is a RISC-V ISA extension, which allows to switch the memory consistency model of less privileged modes from RVWMO to TSO at runtime. The active model is controlled by a DTSO bit in the {m,h,s}envcfg CSRs (per-hart state). TSO is a stronger memory ordering than RVWMO, which means that executing software that was written for RVWMO can also run under TSO without causing memory consistency issues. Since RVWMO is the default model, switching to TSO is safe. The patch introduces Ssdtso basic support: * enable dynamic memory consistency switching if Ssdtso support is enabled in the kernel config * define the relevant envcfg bits * add the relevant code to store/restore the DTSO state * register the the extension in hwcap/cpufeatures * extend task_struct to keep the state across context switches * add a Kconfig symbol to disable Ssdtso support Signed-off-by: Christoph Müllner --- arch/riscv/Kconfig | 11 ++++ arch/riscv/include/asm/csr.h | 1 + arch/riscv/include/asm/dtso.h | 97 ++++++++++++++++++++++++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/asm/switch_to.h | 3 + arch/riscv/kernel/asm-offsets.c | 3 + arch/riscv/kernel/cpufeature.c | 1 + 7 files changed, 117 insertions(+) create mode 100644 arch/riscv/include/asm/dtso.h diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index bffbd869a068..1b26797e7cdd 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -23,6 +23,7 @@ config RISCV select ARCH_HAS_DEBUG_VIRTUAL if MMU select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DEBUG_WX + select ARCH_HAS_DYNAMIC_MEMORY_CONSISTENCY_MODEL if RISCV_ISA_SSDTSO select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL select ARCH_HAS_GIGANTIC_PAGE @@ -480,6 +481,16 @@ config RISCV_ISA_C If you don't know what to do here, say Y. +config RISCV_ISA_SSDTSO + bool "Ssdtso extension support for dynamic TSO memory ordering" + default y + help + Adds support to dynamically detect the presence of the Ssdtso + ISA-extension and allows user-space processes to activate/deactivate + the TSO memory ordering model at run-time. + + If you don't know what to do here, say Y. + config RISCV_ISA_SVNAPOT bool "Svnapot extension support for supervisor mode NAPOT pages" depends on 64BIT && MMU diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 510014051f5d..83e5737d720d 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -194,6 +194,7 @@ /* xENVCFG flags */ #define ENVCFG_STCE (_AC(1, ULL) << 63) #define ENVCFG_PBMTE (_AC(1, ULL) << 62) +#define ENVCFG_DTSO (_AC(1, UL) << 8) #define ENVCFG_CBZE (_AC(1, UL) << 7) #define ENVCFG_CBCFE (_AC(1, UL) << 6) #define ENVCFG_CBIE_SHIFT 4 diff --git a/arch/riscv/include/asm/dtso.h b/arch/riscv/include/asm/dtso.h new file mode 100644 index 000000000000..25f9bb30884e --- /dev/null +++ b/arch/riscv/include/asm/dtso.h @@ -0,0 +1,97 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (c) 2023 Christoph Muellner + */ + +#ifndef __ASM_RISCV_DTSO_H +#define __ASM_RISCV_DTSO_H + +#define RISCV_MEMORY_CONSISTENCY_MODEL_WMO 0 +#define RISCV_MEMORY_CONSISTENCY_MODEL_TSO 1 + +#ifdef CONFIG_RISCV_ISA_SSDTSO + +#include +#include +#include + +static __always_inline bool has_dtso(void) +{ + return riscv_has_extension_unlikely(RISCV_ISA_EXT_SSDTSO); +} + +static __always_inline bool has_ztso(void) +{ + return riscv_has_extension_unlikely(RISCV_ISA_EXT_ZTSO); +} + +static inline bool dtso_is_enabled(void) +{ + if (has_dtso()) + return csr_read(CSR_SENVCFG) & ENVCFG_DTSO; + return 0; +} + +static inline void dtso_disable(void) +{ + if (has_dtso() && !has_ztso()) + csr_clear(CSR_SENVCFG, ENVCFG_DTSO); +} + +static inline void dtso_enable(void) +{ + if (has_dtso() && !has_ztso()) + csr_set(CSR_SENVCFG, ENVCFG_DTSO); +} + +static inline unsigned long get_memory_consistency_model( + struct task_struct *task) +{ + return task->memory_consistency_model; +} + +static inline void set_memory_consitency_model(struct task_struct *task, + unsigned long model) +{ + task->memory_consistency_model = model; +} + +static inline void dtso_restore(struct task_struct *task) +{ + unsigned long cur_model = get_memory_consistency_model(task); + + if (cur_model == RISCV_MEMORY_CONSISTENCY_MODEL_TSO) + dtso_enable(); + else + dtso_disable(); +} + +static inline void __switch_to_dtso(struct task_struct *prev, + struct task_struct *next) +{ + struct pt_regs *regs; + + regs = task_pt_regs(prev); + + /* + * We don't need to save the DTSO bit, because we don't expect it to + * change. So any mechanism that changes the DTSO bit, needs to take + * care to write to task->memory_consistency_model (and reschedule + * all threads of the process). + */ + + dtso_restore(next); +} + +#else /* ! CONFIG_RISCV_ISA_SSDTSO */ + +static __always_inline bool has_dtso(void) { return false; } +static __always_inline bool dtso_is_enabled(void) { return false; } +#define dtso_disable() do { } while (0) +#define dtso_enable() do { } while (0) +#define dtso_restore(task) do { } while (0) +#define __switch_to_dtso(prev, next) do { } while (0) + +#endif /* CONFIG_RISCV_ISA_SSDTSO */ + +#endif /* ! __ASM_RISCV_DTSO_H */ diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 5340f818746b..88740f419d13 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -80,6 +80,7 @@ #define RISCV_ISA_EXT_ZFA 71 #define RISCV_ISA_EXT_ZTSO 72 #define RISCV_ISA_EXT_ZACAS 73 +#define RISCV_ISA_EXT_SSDTSO 74 #define RISCV_ISA_EXT_MAX 128 #define RISCV_ISA_EXT_INVALID U32_MAX diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h index 7efdb0584d47..bedf7fe12c1d 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -80,6 +81,8 @@ do { \ __switch_to_fpu(__prev, __next); \ if (has_vector()) \ __switch_to_vector(__prev, __next); \ + if (has_dtso()) \ + __switch_to_dtso(__prev, __next); \ ((last) = __switch_to(__prev, __next)); \ } while (0) diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index a03129f40c46..b5dc39788c41 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -80,6 +80,9 @@ void asm_offsets(void) #ifdef CONFIG_STACKPROTECTOR OFFSET(TSK_STACK_CANARY, task_struct, stack_canary); #endif +#ifdef CONFIG_DYNAMIC_MEMORY_CONSISTENCY_MODEL + OFFSET(TASK_MEM_CONSISTENCY_MODEL, task_struct, memory_consistency_model); +#endif DEFINE(PT_SIZE, sizeof(struct pt_regs)); OFFSET(PT_EPC, pt_regs, epc); diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 89920f84d0a3..b63d6b699238 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -303,6 +303,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_DATA(smstateen, RISCV_ISA_EXT_SMSTATEEN), __RISCV_ISA_EXT_DATA(ssaia, RISCV_ISA_EXT_SSAIA), __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), + __RISCV_ISA_EXT_DATA(ssdtso, RISCV_ISA_EXT_SSDTSO), __RISCV_ISA_EXT_DATA(sstc, RISCV_ISA_EXT_SSTC), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL), __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT), From patchwork Fri Feb 9 06:40:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_M=C3=BCllner?= X-Patchwork-Id: 771682 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF6F765BCF for ; Fri, 9 Feb 2024 06:41:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707460867; cv=none; b=k/C1b2MHWoP9ZXt6wmtw+l9lMcg/gA3jhRSH3y7zhQZ2fhSKTdeudenvNkh3Hc/DHGh2iGHiicuTfGVtPgslvHCaWRANC3FRKSBHD7iolBflvbe62mg5e/8VNHqoub4jCLgXc1eyd+/uBKHjx9XofVVaC6ZV2yaLQztjb6DteM4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707460867; c=relaxed/simple; bh=6yt58djLr4K/j01BKAd/9T/omJcD7zSGCQ9LoJyod5E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=E6kkIlc9ntt+5jC9y0JGxX8XeUpvlhhwgX8FZcl2CF5KcQpGjTfUzPv0Ptqz67jEZnslOA3LYexo2IqHVtun0uPjyt6NisFqcV2jIUca51hp4q0nv9o7VhOICNPn9tfFavVBHM4SAjljsCm3JPryYMpz8lT6ExiQAZKr1E0TZHI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=vrull.eu; spf=pass smtp.mailfrom=vrull.eu; dkim=pass (2048-bit key) header.d=vrull.eu header.i=@vrull.eu header.b=KaUBIE9Z; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=vrull.eu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vrull.eu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vrull.eu header.i=@vrull.eu header.b="KaUBIE9Z" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a3864258438so110914366b.0 for ; Thu, 08 Feb 2024 22:41:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1707460864; x=1708065664; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bqv1kMDueOeDBLAFFzoizkbUOaos7eIuqxmLJzXo0DE=; b=KaUBIE9ZycXKC6w0BlYJpvU8IW9r9lg3foVfRuwpff1ZVB8lMYQ5/3iPVzmWA7TvCE XXGqsFS1qx6kvOJh6POfDh0wQwfRN43FqQaET35DtSfsq9mZYVZmwWDT5vF7fcnM2KG5 tKRgOHyLemM/xAB7gWca/oDqoo3K8qpyo5f6breWDhsjiBJyYVmolLcq3gwXbT78WqSS TqgQwwIhe+O5vfc5xIviAWOVCjfE2TBcmOfTrN8GKWkSJ2YoRV07apANoA/1jdN7Fc23 NWWdG+HI757tsvniT66AgRV81sfAiQi/blA2vx+6DX5fG5hQmYDdAcOeot/irCrVRAu2 NfQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707460864; x=1708065664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bqv1kMDueOeDBLAFFzoizkbUOaos7eIuqxmLJzXo0DE=; b=X19RewJLFdYmcCUO8pnvQcsK96LMK4lfmD0CK9acmzFyXxtMoahMOWfsc+GF3IgPdC 7juZW4D+MJWv3B7gX/2ILHTN3Pyqoq9xO+XChmN6/zlM0XZ68OK8FiBwBGKdS0g87oyl c5dM6mTe5Tm3qI94po3fztvGyPY4EicjkwgYVSOESJ++32jPxsShAMMEkUC2SxhDCSMJ MXZ3pSmFT1I6KV8KM1PuoRYp5x4oGpo/GWM3RtA0UmP6tHaIAkist30tlZ+EFv+cSSjZ cjAnGeqBzlQxeJJM9wylTFY01oArrtk+acJu1Tk3GdY2JJBRrx4CHTa75XpdNtLIgtx+ xsSA== X-Forwarded-Encrypted: i=1; AJvYcCUwBuNqQH/a6fvyBhZri+RZN98jDEyAw63IE8dgIk3YrFHjulOWojQ0xsEf/Gq9XpCbHwDenBFXCAbNPeA5zJNr6boGMDsElNz8NWVMKaFC X-Gm-Message-State: AOJu0Yznqim2NX4sXRxpt5Dm88eyTNxvCDlwqhwTTYcqAh3j3tIrkMud 3y1jJhOpLrk1Oo2hKw8iyqu35730/iYeogS/UHJ8CLTE6tDrlemx+pTkdsu7icw= X-Google-Smtp-Source: AGHT+IEqYoT9P4yGxxV/Ao8vom6ObPUGe8xNfmYsrj9mQrupH69Vmjj5Fc9hQWs8Ax3BJgQO4kgnZg== X-Received: by 2002:a17:906:5913:b0:a3b:e92b:2a41 with SMTP id h19-20020a170906591300b00a3be92b2a41mr102530ejq.13.1707460864179; Thu, 08 Feb 2024 22:41:04 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVqA02Hms1GnUHhoCeUGdttK/loHEZjbx5QHKXwtPOxxttGYOksuXyK3muEb0xliNnRKF6szoFyOSFwsz9PcaJko6qKNjXTFB2fAnAyUpnMoVD+BQG/lDIsKGVkKuNnMy7wzKsP39NZoAmg06wjeoSHXyfFAN5cEbS/6EnYBTKMlzhMZrDk5pfPc0uIXGdiLw/6BaMIhcMFITP5pb19J+1RzF1urgzUtqlh9B3W/WQuGvzEkbSKup3S6Ti9kxG8XeGvyWsAft1kmMY1VJ6TJbTHHR1wRt2CPbV9ydGIIqh/nQ9xc08W/kJCzxUtTilHW2TunSFgIeJzQ3lp9ayfyWx2182k/ndXiSX8IaP8KBgE8qY5GSVV+bdIJ5yQSQN9sfvgp75UBDTSZI2KE/8h2zApaSfOvP1gqYu/l6w5sOHk5uXmxwNtsDKPg/gvNYpxPQv7s2Gux4RWuG/wgzzPenpKcbPz33H0ukQHd5hP61bQyFOi56bWkPzdKg5iSEHJwmX+hUwCtRf1aFZLCk8jDQBfrf6u7hMgQG4Fzd/mXKTkwTYwYGWEuSXv/GdHqagQcbufcOpMEPv8GUrFRZPXbb+AgJhrcf6yK9tzrRxobSk9dGB+n3QindAud8ArE7rKrG9g6aIUWMuYYYRNaST1o7FvBqxYWk3FP3yx7itHhzpZNOWGtzCqfkGtd8D6UmbZdt4aCSQKBYYs0yt3Y4BwVSmaxyAQ8wD7McQfbqmqIViPqNjTuTbrEnVlJxms4USHpISc/HAc/2iE4gnsIRzEnNdlnrvq5C/hROjT2FFt8cZ7PhawJvwaZWx3U3nXNWMQeSN47hTgcGZa24eiYOELFMolR5ikwa245mlzkn2LoKfrnA== Received: from beast.fritz.box (62-178-148-172.cable.dynamic.surfer.at. [62.178.148.172]) by smtp.gmail.com with ESMTPSA id vb9-20020a170907d04900b00a370a76d3a0sm441180ejc.123.2024.02.08.22.41.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 22:41:03 -0800 (PST) From: =?utf-8?q?Christoph_M=C3=BCllner?= To: linux-riscv@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Jonas Oberhauser , Peter Zijlstra , Palmer Dabbelt , Paul Walmsley , Albert Ou , Andrew Morton , Shuah Khan , Jonathan Corbet , Anup Patel , Philipp Tomsich , Andrew Jones , Guo Ren , Daniel Henrique Barboza , Conor Dooley , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Alan Stern , Will Deacon , Daniel Lustig , Brendan Sweeney , Andrew Waterman , Brendan Sweeney , Andrea Parri , Hans Boehm Cc: =?utf-8?q?Christoph_M=C3=BCllner?= Subject: [RFC PATCH v2 5/6] RISC-V: Expose Ssdtso via hwprobe API Date: Fri, 9 Feb 2024 07:40:49 +0100 Message-ID: <20240209064050.2746540-6-christoph.muellner@vrull.eu> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240209064050.2746540-1-christoph.muellner@vrull.eu> References: <20240209064050.2746540-1-christoph.muellner@vrull.eu> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch adds Ssdtso to the list of extensions which are announced to user-space using te hwprobe API. Signed-off-by: Christoph Müllner --- Documentation/arch/riscv/hwprobe.rst | 3 +++ arch/riscv/include/uapi/asm/hwprobe.h | 1 + arch/riscv/kernel/sys_hwprobe.c | 2 ++ 3 files changed, 6 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index b2bcc9eed9aa..d881d56eba37 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -188,6 +188,9 @@ The following keys are defined: manual starting from commit 95cf1f9 ("Add changes requested by Ved during signoff") + * :c:macro:`RISCV_HWPROBE_EXT_SSDTSO`: The Ssdtso extension is supported, as + in version v1.0-draft3 of the corresponding extension. + * :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: A bitmask that contains performance information about the selected set of processors. diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index 9f2a8e3ff204..ee6e830abe4d 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -59,6 +59,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_EXT_ZTSO (1ULL << 33) #define RISCV_HWPROBE_EXT_ZACAS (1ULL << 34) #define RISCV_HWPROBE_EXT_ZICOND (1ULL << 35) +#define RISCV_HWPROBE_EXT_SSDTSO (1ULL << 36) #define RISCV_HWPROBE_KEY_CPUPERF_0 5 #define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) #define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index a7c56b41efd2..9024061e9193 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -132,6 +132,8 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair, EXT_KEY(ZFHMIN); EXT_KEY(ZFA); } + + EXT_KEY(SSDTSO); #undef EXT_KEY }